From patchwork Thu May 18 10:44:45 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shahaf Shuler X-Patchwork-Id: 24391 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [IPv6:::1]) by dpdk.org (Postfix) with ESMTP id A2C1B374F; Thu, 18 May 2017 12:44:55 +0200 (CEST) Received: from EUR01-DB5-obe.outbound.protection.outlook.com (mail-db5eur01on0081.outbound.protection.outlook.com [104.47.2.81]) by dpdk.org (Postfix) with ESMTP id 8C3E1374E; Thu, 18 May 2017 12:44:53 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=aTuprClNMRmIOHr/Mux5cY7yEP1LrQfQMbfu25Gnyu0=; b=GRVwi3aSMs92kLnLzDj23egp93yuELVBC7GTvKtwHQOIF+G7VsY3Z60UuqbyF9YC4LWfuNNLhU1cSRTgYm/1u2lkjtGwIhLsiJdfm94YY334PDXsDda4N7lDavqd4aLspnqH+lUYp3MY00ulI96ZNB1QL5Yq7tM5gL/Lk9BFZGc= Authentication-Results: 6wind.com; dkim=none (message not signed) header.d=none; 6wind.com; dmarc=none action=none header.from=mellanox.com; Received: from mellanox.com (82.166.227.17) by HE1PR05MB1516.eurprd05.prod.outlook.com (10.164.49.150) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1084.16; Thu, 18 May 2017 10:44:50 +0000 From: Shahaf Shuler To: , CC: , Date: Thu, 18 May 2017 13:44:45 +0300 Message-ID: <20170518104445.44358-1-shahafs@mellanox.com> X-Mailer: git-send-email 2.12.0 MIME-Version: 1.0 X-Originating-IP: [82.166.227.17] X-ClientProxiedBy: VI1PR0501CA0011.eurprd05.prod.outlook.com (10.172.9.149) To HE1PR05MB1516.eurprd05.prod.outlook.com (10.164.49.150) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 865c00f1-db3f-408c-fce8-08d49ddae9a1 X-MS-Office365-Filtering-HT: Tenant X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001)(48565401081)(201703131423075)(201703031133081); SRVR:HE1PR05MB1516; X-Microsoft-Exchange-Diagnostics: 1; HE1PR05MB1516; 3:y7f4qdaKHzIdnnFXBYQoLuQNCLly4QT8k5Lrlcab+5ZQ8zCqDMn5b+3mrkS7rhW6bHc5sNwZBuJQFl4Mz12w7bD4F4LgKpgRY3GIfv//tGNPn2ZdkfLiXlJXv+VCiGZMzXvUqFoaJGCyUXeUdFqqsu5jUI7DR/FdikKuNYrFuycFXE6BsWSIT6Ftx79Oc03ALceUzDTPfBRV+16Q6REGszgD9SJY1h9FaKFaZzck9HDT8FIqeFjysd7vc9WF20ujcj5JwtIZhCis21jcwzeiLyR/aP85O/6cOG1ZTjFb72aqsuq/WbDv/q5Z4iRJ50z1bC7S9k4uxVofd21ZV3fnHsnRC5T3wNIOv8ByhZsiLAc=; 25:+ggwCZ0yR5iwooxfZlWXf387NqjcexY/HH6lE2XyJzTMAzGdxIgIBf3K6UYm7TAShQPeGPe74FfOfjIel1SbBUh2vpzhjs3Te7KKGNfVD0HGtLoFE6ndscNDje3R03eApm/ClGLpC7mL2Jdostg5akXQugaflKfylana+yI4kqMplouEUu54tKDeaC4t4QNq8T5OMIGOa37JFilXPvZXdpGr9JgCGKsB6D4ajuPS5JRZK8WaW3taDmZDx5sQogiD/cjo7x86HibVGqx1JVoxI5QDKV+ZU7E0aub9KnlcPW5PnAGoEaHyY2Ge2FFHu+IXQDFX4mW4QldxMO/pXb2D8SwbjM9+qkfcDk7Dmwek2JXVSqP/CnbKCRx+Q0Qgiei7AmhHEY+QfPL1KcsHQFO6ol8TzM4T2Mkw3g7sclms+5fpAk2v8XIR6cdTR/pdkuCqoCJar5aPV3KW/GVthuF0JluwypmCEx++ngWon72esHo= X-LD-Processed: a652971c-7d2e-4d9b-a6a4-d149256f461b,ExtAddr X-Microsoft-Exchange-Diagnostics: 1; HE1PR05MB1516; 31:l8MBBK0kR9OdizDYXzDWAkd49X5zCiEYpqEmN8ODvSvFntLmRbOREt2wyy1pN2QI2lMm/IV22P7C++KkovIa5VHAqdhBYNp8Rmf2jVDOPFVwyQwHaRi7X3Lsh/7giJ6D7VgY3hMKxgEsxh1L7lbM6vZ4otNMNQ+Y2K/U0+CjKLRQml1Siz9pWfoeoCl8axrYER3P1ZAammpempRSgFk3+Cl5or7VpoGjz3Z0R2vYOqA=; 20:V2GNYA1soPurv4sAISYSM0ffh34IZnoZ7UP33bH7G/P+C2yrkKRWIeYPspL9jl85lwBgh3J8jyLb3zxU1TcdYM/IPBi6X/5wg1bd1txoHuE3owQad9CgJki44SEWmc50AFePdsg8Axlu3lnwvJHPYmiaxcB1yAiwmXu0Py3Qen3G0+lcvZ2tSiIxBfCrCbOiM76WBaGy31o195U+zLLB4Tfh+pdmy6mCYR/c6skyY8glteADgFaHrdQIRz/TBot5LGqnIwQCTRwibBpK2277VueuLQ9RcED+cAZXrYOHQk+TJ17YHSOKOousxxslytuoDbrecZR9IcpAEmPw/MgEiwtYrmT5ogLDV2uI8NfDiKqNeilATNB/ZW1/4w1txNnMegmfNGx+F4rY1xigFf37gSC9uhGMXJ253RtR1IB0teYlNDjAbrT3dxCmUzV39NqN5gHDYh0alMpAmOHU4vd1jS8yf0+hbq1CjeYEs+BfsiV+9XseTcMIf36Kx9DKdqL/ X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(6040450)(601004)(2401047)(8121501046)(5005006)(10201501046)(3002001)(93006095)(93001095)(6055026)(6041248)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123560025)(20161123564025)(20161123562025)(20161123558100)(20161123555025)(6072148); SRVR:HE1PR05MB1516; BCL:0; PCL:0; RULEID:; SRVR:HE1PR05MB1516; X-Microsoft-Exchange-Diagnostics: 1; HE1PR05MB1516; 4:VJfOp1UB7dWpvHk8NZ0De61640xrtv9Hgd0WOLf/ElItPaMCK28Wluxd5drgEJ34R0TV5Ui9RmPpY3rjZDuRBinUkoftvKd95trFxcipeJMuVHEryW8lG/PUySestY57T8kMmpKKiVHLpQ5cIJ0pv/bjGvArmDAJvEJ82P4f6Pkv/X9ONmL0jS9yjfySSzboJnPptpe0Roylf6aSeYYpOF5Su2LSG59Jmgh9i4maGiKJ15GgQKgNjTkn+iuduML+8JHK2Nw6Bf0KVDgQGLFgCrS92qiNj4CiAxmamFI0Em/VW0dG22cU1rLYuA7bIm5NlZpUPVPxEHJYQey9n3KOfapenhGKXivHgmSmnD0yy5gc70hFmFA4+p9Z8+P5sdqGQb9boym5KeoOC3zCU+I5RW24xupdXXSWk8DVWVWP8MeFEIuA/Fmiw3AzGjmMuewBi8WjsHpaupfHIwx2Ci5Fohz+uWD1SHH7TkwARlJnl5673bjCPyp8EQwscy3TwLJsXbdC3Cixt5g7Iegni3yf1d19gAxEolJDGiddF14ARHsJROS5kb+HgNFWVWLDga5CVP7PPKt9v8xKb5sNbIp2lyv8wl5a6pRsF080viJyL1EzneLcfVrkk3ufPYBMOdrIpKPIkJU8ItuK/O/1VOXGKB3fbWcuLlbPrg9MrCX/h05LzzMqeO2RLZxIr5VyIF10e3cCm2qjexJpTe2jJR5jCh84IWlNY7pA71yW22ZgzyE4O4RqqgkpNuDLINfjT97r5HeV1apsLprJwpytcNQgHtXDtpSfzr0ImYQtR+/Ctl4/+nIPC3QslKYe0psVYdTx X-Forefront-PRVS: 0311124FA9 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(4630300001)(6009001)(39400400002)(39850400002)(39450400003)(39860400002)(39410400002)(39840400002)(478600001)(5660300001)(189998001)(6666003)(50986999)(42186005)(25786009)(33646002)(5003940100001)(38730400002)(50466002)(48376002)(8676002)(81166006)(50226002)(86362001)(3846002)(1076002)(66066001)(4326008)(6116002)(55016002)(7736002)(36756003)(54906002)(33026002)(53936002); DIR:OUT; SFP:1101; SCL:1; SRVR:HE1PR05MB1516; H:mellanox.com; FPR:; SPF:None; MLV:sfv; LANG:en; X-Microsoft-Exchange-Diagnostics: 1; HE1PR05MB1516; 23:qZ0hZ75K1UNfF0j5b+bcsKrkvyLh7UuPUGi0SMKGluD71NHUDQRtg2AwBAbDQEG8iA4xazvJNrbeD4+DaAI43h+ZZvF+gaijqIdrYT9akPIFxIg3hj1/QYK7FnXYeoP+l8wUz2NBHNWPZl3Veq5S1/dWnhQm5YlMA/ObDjWqN4moB3EXUTmT+FxKTeye4bAEI/wp0qNDFiHG7TJzm0VM6GtwP+NrnzHFCon1ptO6F+YAGGVZBzGQPZx9nZBDIjoAUlzyTOaVr67VprG6mZL8TgbIzvuzcxLoW0K1JBMMXSwVhu3FIdrg6b1naQlHk90RpfJ4PLuRZDw072NlwK6Yyx6pcu1PtXah3SqMKCvhZrAr5iMkBotPcennaYuXSSg+gkeCt8R6Gc3vc1RPsg5VpsxSvoJx0YFpLr3aT2Vi02tsOB9i/IptrmiruEsNcKNISZyH2DE7ilptNV7l06swPsGCeewekYesrFp0PgDVXdMjfqYb7FnH9PzNMbUbx6rEW+aSHxc/dQc3O3OY2plJiXqVpssFU6MPDalrEOIDD9RrkivQ7x07B/iISXVH1sfKnIx4xE8eCU2OZBmvA/YGuNMSlsOReRt1A6mVP4tT5OIq45/79Dx4XtMhCvrp1LKynoWXpzCuKqbAZFbbdVJjqVQsT8EI8sahfhw4O9H4PJaO9Fl4d6Zhb2IQEOA42vz3f3BM73ZmxqYeDdMlgYTZMmM3gUIqyrK+om7dVnUZyIx8bZhkDg8hSq44IyApUWgSON1100YDyRCtRw8CEQ9trWkXb3Suzze5oxP8EzZ5mJtJ6rOsVyNS/NeGNvaysTiv1h0IORZaqovP+wJ3flpywE/8DlCGrn7cqDyZ+1VNsBKfwGXZjRTucjdGjPr29NUZUWYDp1wmP7s3TPthg0BliRx+84cXcdPBQpT/puuoyIc= X-Microsoft-Exchange-Diagnostics: 1; HE1PR05MB1516; 6:cDawqsSduGUR2zUqfa4euwBVNhBw8b44VQ2F5wEcibrNHb6wZ2/3h62QnVDAmiR/WkRGB6iSAD5gynSPhYYyBijGY5FgjrlM1jCXbluYzuptccRX+g4w1hNcSeZl0ISlMkC5XpHLL35PdVcptsVNDrEnE43sdNDVKNxeAXRqyuZwY+/c0gSzMk4cQ7S4bXcHCT79anyfga8+YyeFII16sm8cnsxCrl0HZJPpThwGkaHFIlqFkSOvhYgnAGna9Q+B6lytFbjWjVRx30pSPAIb+eXO4/UffB3EOPcg35lpTpbB2NPA+Zq3fzkTOXgvSX4lON3Ckfe0IY8XtcqutLzymZtymNbpWfKS+HrNnqOCyUhQGOFVzm6BitmfrjIJLmDimRFC7d4VyWdfhIsvQggmFNa5tAMd/ec/H2z9ybB2+sYjpah/gJ3OLvpShiOSbqkG7whbD2Wen1WqNzVThSvB4kDjt0CU/U93uBhITPJwFMvGPm8EFbGmMwV8SbQZv3Mpf7LkQBWhhMnZ12XIfOi6chnFgbFYx8qfjyPOLrtmbes=; 5:4MShERMFnQR2CTRp0F2MfD4L/e+qQOkhOrOyrVAsWWeOhHUoqYqpdM2w1KtHSMACQF3r7xf3DaWsYQQAnK6HGSJCnmV/50V+O+lsHSRxxhE9iPSzhbh/6YjCTue2OfPZCKW+UparxGV9nvrw+qaxgQ==; 24:LQuolZHMFAZdp5xA8g3zz/PuHuFYM8WNh1t66F67fb2TG3plxQKLTkudqMoqbCkNQG/AqI4wxS4i/ji271xguCSvnuUG7uhugHJTKqeMxWY= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; HE1PR05MB1516; 7:MMJWi9ePDaOjCQLrnvOLULfdTC5Oi0ILJ9AJmg5T4UiBitAFDEzAYRq4zVQmpC50hwUS+V9ZwDPTU2GWMfMg5z1zj9yBFW6vREq9J9Eghw92wDfQMflUMvW+ZOmsjW2sB9Nc4f4DYKE2p7MNkhjJAtNB1d/iu7Hmnf4rblnWGqB3OMubKoDRcr3YgsMyyKc4LovSyIrn7kX9f2EsedwZ2LChpv8rP+G+Tns4Xyef4Jak//FJ4lZlC+Z0krH0iBDB+7YL/f041BFI5IEeg9rzFz3CrhBIopqAiI3vM7V9m11RzTsz7ab6IV5Uz4ZNOsv+cHSAFvH6zXlB/O1swflRIQ== X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 18 May 2017 10:44:50.5324 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: HE1PR05MB1516 Subject: [dpdk-dev] [PATCH] net/mlx5: fix wrong completion buffer size X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Completion buffer size was computed wrongly, causing completion polling to wraparound too early and miss entries. Fixing it by using Direct Verbs to query the CQ info. Fixes: 6218063b39a6 ("net/mlx5: refactor Rx data path") Fixes: 1d88ba171942 ("net/mlx5: refactor Tx data path") Cc: stable@dpdk.org Signed-off-by: Shahaf Shuler Acked-by: Yongseok Koh Acked-by: Nelio Laranjeiro --- drivers/net/mlx5/mlx5_rxq.c | 14 +++++++++----- drivers/net/mlx5/mlx5_txq.c | 14 +++++++++----- 2 files changed, 18 insertions(+), 10 deletions(-) diff --git a/drivers/net/mlx5/mlx5_rxq.c b/drivers/net/mlx5/mlx5_rxq.c index 8b7823360..2a2683985 100644 --- a/drivers/net/mlx5/mlx5_rxq.c +++ b/drivers/net/mlx5/mlx5_rxq.c @@ -838,12 +838,16 @@ static inline int rxq_setup(struct rxq_ctrl *tmpl) { struct ibv_cq *ibcq = tmpl->cq; - struct mlx5_cq *cq = to_mxxx(cq, cq); + struct ibv_mlx5_cq_info cq_info; struct mlx5_rwq *rwq = container_of(tmpl->wq, struct mlx5_rwq, wq); struct rte_mbuf *(*elts)[1 << tmpl->rxq.elts_n] = rte_calloc_socket("RXQ", 1, sizeof(*elts), 0, tmpl->socket); - if (cq->cqe_sz != RTE_CACHE_LINE_SIZE) { + if (ibv_mlx5_exp_get_cq_info(ibcq, &cq_info)) { + ERROR("Unable to query CQ info. check your OFED."); + return ENOTSUP; + } + if (cq_info.cqe_size != RTE_CACHE_LINE_SIZE) { ERROR("Wrong MLX5_CQE_SIZE environment variable value: " "it should be set to %u", RTE_CACHE_LINE_SIZE); return EINVAL; @@ -851,16 +855,16 @@ rxq_setup(struct rxq_ctrl *tmpl) if (elts == NULL) return ENOMEM; tmpl->rxq.rq_db = rwq->rq.db; - tmpl->rxq.cqe_n = log2above(ibcq->cqe); + tmpl->rxq.cqe_n = log2above(cq_info.cqe_cnt); tmpl->rxq.cq_ci = 0; tmpl->rxq.rq_ci = 0; - tmpl->rxq.cq_db = cq->dbrec; + tmpl->rxq.cq_db = cq_info.dbrec; tmpl->rxq.wqes = (volatile struct mlx5_wqe_data_seg (*)[]) (uintptr_t)rwq->rq.buff; tmpl->rxq.cqes = (volatile struct mlx5_cqe (*)[]) - (uintptr_t)cq->active_buf->buf; + (uintptr_t)cq_info.buf; tmpl->rxq.elts = elts; return 0; } diff --git a/drivers/net/mlx5/mlx5_txq.c b/drivers/net/mlx5/mlx5_txq.c index de7e28be6..6c1387e56 100644 --- a/drivers/net/mlx5/mlx5_txq.c +++ b/drivers/net/mlx5/mlx5_txq.c @@ -173,23 +173,27 @@ txq_setup(struct txq_ctrl *tmpl, struct txq_ctrl *txq_ctrl) { struct mlx5_qp *qp = to_mqp(tmpl->qp); struct ibv_cq *ibcq = tmpl->cq; - struct mlx5_cq *cq = to_mxxx(cq, cq); + struct ibv_mlx5_cq_info cq_info; - if (cq->cqe_sz != RTE_CACHE_LINE_SIZE) { + if (ibv_mlx5_exp_get_cq_info(ibcq, &cq_info)) { + ERROR("Unable to query CQ info. check your OFED."); + return ENOTSUP; + } + if (cq_info.cqe_size != RTE_CACHE_LINE_SIZE) { ERROR("Wrong MLX5_CQE_SIZE environment variable value: " "it should be set to %u", RTE_CACHE_LINE_SIZE); return EINVAL; } - tmpl->txq.cqe_n = log2above(ibcq->cqe); + tmpl->txq.cqe_n = log2above(cq_info.cqe_cnt); tmpl->txq.qp_num_8s = qp->ctrl_seg.qp_num << 8; tmpl->txq.wqes = qp->gen_data.sqstart; tmpl->txq.wqe_n = log2above(qp->sq.wqe_cnt); tmpl->txq.qp_db = &qp->gen_data.db[MLX5_SND_DBR]; tmpl->txq.bf_reg = qp->gen_data.bf->reg; - tmpl->txq.cq_db = cq->dbrec; + tmpl->txq.cq_db = cq_info.dbrec; tmpl->txq.cqes = (volatile struct mlx5_cqe (*)[]) - (uintptr_t)cq->active_buf->buf; + (uintptr_t)cq_info.buf; tmpl->txq.elts = (struct rte_mbuf *(*)[1 << tmpl->txq.elts_n]) ((uintptr_t)txq_ctrl + sizeof(*txq_ctrl));