From patchwork Wed Dec 6 14:48:12 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matan Azrad X-Patchwork-Id: 31952 Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 9A9BB1B195; Wed, 6 Dec 2017 15:49:03 +0100 (CET) Received: from EUR01-HE1-obe.outbound.protection.outlook.com (mail-he1eur01on0054.outbound.protection.outlook.com [104.47.0.54]) by dpdk.org (Postfix) with ESMTP id 1BD817D19 for ; Wed, 6 Dec 2017 15:48:59 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=DPgTTxOJcoAfO7OluxcD+1eOWCjseV1R51oTG0YFmfk=; b=jHWrL1o10xYG9WQu0vo0SmvLbGuxZFMWjbXUo7kFg6ukIFxvaMqzgI0VzZiPWZEjANUnk95aMMLbFlgcWrcLyJsZy1E3+kBpavXM6bvFnOaHMg0bpY9uT9z+f/EgksCRMNCfv3FN25JIn+TYw9DCKmoWJNyzZ3CIxSn8mrQmX7A= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=matan@mellanox.com; Received: from mellanox.com (37.142.13.130) by HE1PR0502MB3659.eurprd05.prod.outlook.com (2603:10a6:7:85::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.282.5; Wed, 6 Dec 2017 14:48:57 +0000 From: Matan Azrad To: Adrien Mazarguil Cc: dev@dpdk.org Date: Wed, 6 Dec 2017 14:48:12 +0000 Message-Id: <1512571693-15338-8-git-send-email-matan@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1512571693-15338-1-git-send-email-matan@mellanox.com> References: <1511871570-16826-1-git-send-email-matan@mellanox.com> <1512571693-15338-1-git-send-email-matan@mellanox.com> MIME-Version: 1.0 X-Originating-IP: [37.142.13.130] X-ClientProxiedBy: HE1P191CA0003.EURP191.PROD.OUTLOOK.COM (2603:10a6:3:cf::13) To HE1PR0502MB3659.eurprd05.prod.outlook.com (2603:10a6:7:85::17) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: a88743e0-d892-40eb-f825-08d53cb87b29 X-MS-Office365-Filtering-HT: Tenant X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(5600026)(4604075)(4534020)(4602075)(4627115)(201703031133081)(201702281549075)(48565401081)(2017052603286); SRVR:HE1PR0502MB3659; X-Microsoft-Exchange-Diagnostics: 1; HE1PR0502MB3659; 3:mK9gKjIDklOKcA4DxCWjgOApTC1WN+FWcPaGSzOyiZ95F7xMt1DSIP9AH9QNKnq8+0br6VSi3+eIBu9Uag3lAz56g0uHbAj3sRcseu2VkGLUMektbzdOSF4f9MyZLDr3SmuNWQ2w6EtVf5rT+MHNQByUtj5TIEGUTGOROrMWZ2Vdb/+TKZZbhJIYuAvmYtu/JTJQT6xLoQFReSORi+m1JbaYRDwTQet3i5nFLqx4wBLzgeLlJEsb4lxOwwWyKEj4; 25:BQHLp4Wc4U2kIs6sxbQDVAfPyh2mszaZ76/ypTb6DoZPy20QNONGI7Az6k4N0+11Qo5kcnZH6V/CdurmpOLcrg161wtS7PAAvdyovOmLuieJ8M0Q2SdPKzNhfW5/7w03e+EBDicCeRY8fFSAlq2kQBJhGtMmQPm7g+VQst0rsGdftl3cBH50iLRLZKm2ewHBfwuyz+BVXMn6qRlY9maF5YriuGcE+gCi3Lc3Vb3hvmNyBkQ/+1QMmS22kmjI6O20zEL/JHgFcbPrvyQfkuaWNabZXrQgkKlzHm2b0F81p7m6n/GC9RXjscWSzk1KVtfH3ITOF0hURk0bGaSS7f8rgw==; 31:QuFzBlScGpmhfn1jEjm2N2dYHsxFmh/OaxKirmU1NJYQz7+igvxL5HOBB3AWXQBsqc8MqKsnGL9BcZ6wfzylt5PuLCiNHoC1z/L4QT1mAdgmkB/WojcSEXpOOjOz3fVjzcCRh+ouBLZ3X9KY3IhwbnVT6IhGsTUn11g3qu2RgiHB37za0zXHcdUJ7+A7Rdy2sVw/9UmtWk1eyWk+oJ+oc7k9t605BzCH7FLYhUbu8VY= X-MS-TrafficTypeDiagnostic: HE1PR0502MB3659: X-LD-Processed: a652971c-7d2e-4d9b-a6a4-d149256f461b,ExtAddr X-Microsoft-Exchange-Diagnostics: 1; HE1PR0502MB3659; 20:7o9rjzfoxmSvIRukEFs/s/ZA61nAhw1MaQue3av4IHi8xswCaXF8DOvr781DmUMMYPIIJsMzwuoD/vDDe2e63N/oD6anfRQXmMihHousLF6FFCGGo7YM5eHe87zB6GLi8V8xk9H8ybSZC28Nihq6GjRz+SD+GDymN+aLTVkTd3qwbXoMjWvyvkaRMuipEJ+iHphZyqpP3z0n98sB+cJlhRpt/X6O+asxRgOLQSTcVVyG+at6tnXo7gaWPeUl0psj7bE66p93Rpsd4kE9bHpL6tVvnuePKrYPiV6/A3S0z3aN7JxnuO4QaG/azN6iWAolG8kYUxNVE/QtL5exQ0fNFYwrv/jfJk6T/u6lS5fQqPHE9qax4Dt2r7FSL6TMM3luNqnyEU9LCYQySUm2HO3LdO8dxHi1D+UwbPfQFWz6jBJe8h6ML32kPL+bxcDYsdQwy1PRnHw2NQb+ELIe53FYlGa4xdUxDyH6IclehqvStPxNGhxu1zrqrDdue9msiqnZ; 4:FRIAsfkzTZndC1lfaMFSbf8Qhs29Hi07uZqiwvHmC7MqiP7QzYqrH0DIhWZmM3g18g4ZFJ9m55rBhmc3z0W+4ehWXRfPEWdIBdSGtmL9BLiJkGAlYraMNUhCqCD1YAdPIbwfgEePLIF3XH8noNJywgeDKhPHnxGI53iJEjrUL9mdCoSNAA0PwNMd6gvJdU8z848222FpNAHMXKY+QOL0qnW3NuPpY6U01OSa1P5kqyhizlTL6llyKU4pP9j2gUbJdljxeDL6nDGF0SpWQo5EIbuxAVfvMK1oLywX9bWNVlRVtA23facnv1DjBKtUeNjG X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(131327999870524); X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(6040450)(2401047)(5005006)(8121501046)(3231022)(10201501046)(3002001)(93006095)(93001095)(6055026)(6041248)(20161123562025)(20161123564025)(201703131423075)(201703011903075)(201702281528075)(201703061421075)(20161123555025)(20161123560025)(20161123558100)(6072148)(201708071742011); SRVR:HE1PR0502MB3659; BCL:0; PCL:0; RULEID:(100000803101)(100110400095); SRVR:HE1PR0502MB3659; X-Forefront-PRVS: 05134F8B4F X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(7966004)(366004)(39860400002)(346002)(376002)(189003)(199004)(6916009)(2950100002)(6116002)(4326008)(16526018)(6666003)(3846002)(47776003)(8936002)(76176011)(508600001)(8676002)(81166006)(101416001)(97736004)(33026002)(86362001)(55016002)(4720700003)(21086003)(81156014)(105586002)(7736002)(66066001)(25786009)(2906002)(69596002)(305945005)(5660300001)(106356001)(33646002)(50226002)(51416003)(50466002)(16586007)(7696005)(52116002)(316002)(53936002)(68736007)(36756003)(48376002); DIR:OUT; SFP:1101; SCL:1; SRVR:HE1PR0502MB3659; H:mellanox.com; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; Received-SPF: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; HE1PR0502MB3659; 23:Dw6PJ2Y5AXrXceSPBYW8lWKR+VR7FqPyqNjal3u?= OZrXSN7H0MvumXYLpnsWW040AFlZB2K51EvIRXdrDKIyDsHvXRyC67ncP53tKr4YV23Z33mDAUec4/RIh3Iq/izWsqH7J1Xapt6YR+dBjXdMh3zfQnImzNvCrPT2M+PFh/fvLZsflycsQFvuT6Ob0SMByjGpJQGfH3O5GZhkBZlQmB5OrNamu/ua1S9RExxevxd1Dsg59UkZP34qaMtopAauCMUd44y8XBCfEsTBMnsoLMjFTafVram0Fle7QOxGbF7UiZaaSO33xgZAQcVuedgMfv11hKj8BIY+eNFMsTapKU8NpSrP2Tnlm9MOHCeVouettGHl4GsmXwOKUdfnjNihXEHb0QwQA8gJOTHwYVNbEX5MfYEUNCmQijvyAaUz6XRTrvmqMr136tm4w6PVsAgWxNb3hH1RlLGfC97xWLoQsFe6Trw70H8cNyEH2xky32K0mUJ1MJdeoMCWp0pYGwyCwkM4+GLsJ7Kye2Bktt2JqpoHmShnGGWfzDIgWr9iuZr93xdUkG9iQetkEJfldUUAtU/p38brgmAvGt4gydC+RL2AWSKnu58F2BcmIGrYgOfCWaYLDKtUabBKK+UZWED1FV0vamb6oUdltC3OsElnOvhIfNub5WJRUgt88UxmdSzXiS0YVhOcahqR5RXLBkultXVVQQ2JsokyUG2Ngg17djKAJvd+/EhUaAEIXwmpGV9MBKHEln5mFli0+borF4XqdRc/LXyb2yGN/rPNMWDIvSGw3NsdVkX57+6HGQBzVhXnTVYrnlJSH6Ynj5PL9IBrskpm+t8vJVHd66ekqIilR7dXN9VMjbxMJFzAm09YERjM4b5/G2DCA9j3kZglwsZ0oo/oeDJY79hPcLLUGyY+ImgplNuI5+8FU78N+kgzKn6LDczTggtWZBATi76Qj8r0xNJb+bEqDsKZ8WDNhehUhS3Gn06k9pAEdcGPetzTggzkqKn1u01CH+pcfyr7v3XorLkOia/ENNPdZ83XkXLPvTtRt7JqTg9nKaK0iYYINQjwsYanKNcAwE3LpUDB8cobYLOq6QtIFNz5u+V67R5W+E8fpYGLCs/oNeCj2yFv7ZQHmZchEKsmfLlsysHQCjRTv X-Microsoft-Exchange-Diagnostics: 1; HE1PR0502MB3659; 6:ABVXJUHf+0q71wnt7lIgdoz/5nY3oWHLFqlW+JiEWGkvPBSbliEhGyf4K9xOHfnovdiIZQoPRvmE1rtDGhWvhWFymB5GYxChOa+JZhgnvlrjSmWfZff17tyb/wK8yUaBA9ZDTL/NEs0JKWoKI05Z+nLvRlhZKDvoZ9gjk+sAlB/nRRGEiRTrXI2NhVisZuyO94T6Ctz7qHsj7hHjPU2NHiWzxbuIcEJvsmyO7Zm4KxInJRchVljjqNgye51A9JZa1GqzlLDRY903wGDgwFbuw872dZdXTmnJq7jxJ/RR6lU7pmfGeSAKoGAfqL7GLcQJwFr93LmJtRHjtvbvFOiwH1eLdMLJIXPS1t3GfPQ06po=; 5:08I9KBChKlID/P5V40PTJLlAVE4iWNj2SoLg0Ys5EuxufsXJvo2TEw0ZYnBOvf45cdFRUcq1OmEqTrWJzFDue8LTkYAmbTN8/bOYLHwG6HTe8JHiPinSUdPiCnodZQO0nsQf1AhHgp60FjU6ECU3GMLtA0UNK4GXZwtRaoZXWmk=; 24:YEIB60HCkZ2qGvPVdto/0sPAHLF+ScsaHvzalS6Xth5DHOdtnZ6zhxzv2VkR+8XHNumGvPQqJd74Ea5MzStrXm6+1dBB8KctXsLLqYsDVX0=; 7:gWuN2LGb0wevhpTNiXt8ifYOyqsTonQrIUScF5uHX2MFGvKlP+dSXm6rNSnhKz82ydgpsX9jMSUbIWvWdi2aC/3S7SS1XQHxsfIXw8X40N2EVRIVFmeI7dTdP1GoUR5KmxD09XNDOd4Qe0yIGsn4uUvXIW/h50l1gC64qg+PmwzT1N3kMrtFqz8j3fpbNXcKt/VoJNnzrsX/2wT0OeXYtmyotOiaEDC4Q6yh3wwzi2paManTELa9S59UJu5AUmRa SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Dec 2017 14:48:57.5477 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: a88743e0-d892-40eb-f825-08d53cb87b29 X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-Transport-CrossTenantHeadersStamped: HE1PR0502MB3659 Subject: [dpdk-dev] [PATCH v2 7/8] net/mlx4: align Tx descriptors number X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Using power of 2 descriptors number makes the ring management easier and allows to use mask operation instead of wraparound conditions. Adjust Tx descriptor number to be power of 2 and change calculation to use mask accordingly. Signed-off-by: Matan Azrad --- drivers/net/mlx4/mlx4_rxtx.c | 28 +++++++++++++--------------- drivers/net/mlx4/mlx4_txq.c | 13 +++++++++---- 2 files changed, 22 insertions(+), 19 deletions(-) diff --git a/drivers/net/mlx4/mlx4_rxtx.c b/drivers/net/mlx4/mlx4_rxtx.c index 8b8d95e..14192fe 100644 --- a/drivers/net/mlx4/mlx4_rxtx.c +++ b/drivers/net/mlx4/mlx4_rxtx.c @@ -312,10 +312,14 @@ struct pv { * * @param txq * Pointer to Tx queue structure. + * @param sq + * Pointer to the SQ structure. + * @param elts_m + * Tx elements number mask. */ static void -mlx4_txq_complete(struct txq *txq, const unsigned int elts_n, - struct mlx4_sq *sq) +mlx4_txq_complete(struct txq *txq, const unsigned int elts_m, + struct mlx4_sq *sq) { unsigned int elts_tail = txq->elts_tail; struct mlx4_cq *cq = &txq->mcq; @@ -355,13 +359,11 @@ struct pv { if (unlikely(!completed)) return; /* First stamping address is the end of the last one. */ - first_txbb = (&(*txq->elts)[elts_tail])->eocb; + first_txbb = (&(*txq->elts)[elts_tail & elts_m])->eocb; elts_tail += completed; - if (elts_tail >= elts_n) - elts_tail -= elts_n; /* The new tail element holds the end address. */ sq->remain_size += mlx4_txq_stamp_freed_wqe(sq, first_txbb, - (&(*txq->elts)[elts_tail])->eocb); + (&(*txq->elts)[elts_tail & elts_m])->eocb); /* Update CQ consumer index. */ cq->cons_index = cons_index; *cq->set_ci_db = rte_cpu_to_be_32(cons_index & MLX4_CQ_DB_CI_MASK); @@ -580,6 +582,7 @@ struct pv { struct txq *txq = (struct txq *)dpdk_txq; unsigned int elts_head = txq->elts_head; const unsigned int elts_n = txq->elts_n; + const unsigned int elts_m = elts_n - 1; unsigned int bytes_sent = 0; unsigned int i; unsigned int max; @@ -589,24 +592,20 @@ struct pv { assert(txq->elts_comp_cd != 0); if (likely(txq->elts_comp != 0)) - mlx4_txq_complete(txq, elts_n, sq); + mlx4_txq_complete(txq, elts_m, sq); max = (elts_n - (elts_head - txq->elts_tail)); - if (max > elts_n) - max -= elts_n; assert(max >= 1); assert(max <= elts_n); /* Always leave one free entry in the ring. */ --max; if (max > pkts_n) max = pkts_n; - elt = &(*txq->elts)[elts_head]; + elt = &(*txq->elts)[elts_head & elts_m]; /* First Tx burst element saves the next WQE control segment. */ ctrl = elt->wqe; for (i = 0; (i != max); ++i) { struct rte_mbuf *buf = pkts[i]; - unsigned int elts_head_next = - (((elts_head + 1) == elts_n) ? 0 : elts_head + 1); - struct txq_elt *elt_next = &(*txq->elts)[elts_head_next]; + struct txq_elt *elt_next = &(*txq->elts)[++elts_head & elts_m]; uint32_t owner_opcode = sq->owner_opcode; volatile struct mlx4_wqe_data_seg *dseg = (volatile struct mlx4_wqe_data_seg *)(ctrl + 1); @@ -725,7 +724,6 @@ struct pv { ctrl->owner_opcode = rte_cpu_to_be_32(owner_opcode); elt->buf = buf; bytes_sent += buf->pkt_len; - elts_head = elts_head_next; ctrl = ctrl_next; elt = elt_next; } @@ -741,7 +739,7 @@ struct pv { rte_wmb(); /* Ring QP doorbell. */ rte_write32(txq->msq.doorbell_qpn, txq->msq.db); - txq->elts_head = elts_head; + txq->elts_head += i; txq->elts_comp += i; return i; } diff --git a/drivers/net/mlx4/mlx4_txq.c b/drivers/net/mlx4/mlx4_txq.c index 4c7b62a..7eb4b04 100644 --- a/drivers/net/mlx4/mlx4_txq.c +++ b/drivers/net/mlx4/mlx4_txq.c @@ -76,17 +76,16 @@ unsigned int elts_head = txq->elts_head; unsigned int elts_tail = txq->elts_tail; struct txq_elt (*elts)[txq->elts_n] = txq->elts; + unsigned int elts_m = txq->elts_n - 1; DEBUG("%p: freeing WRs", (void *)txq); while (elts_tail != elts_head) { - struct txq_elt *elt = &(*elts)[elts_tail]; + struct txq_elt *elt = &(*elts)[elts_tail++ & elts_m]; assert(elt->buf != NULL); rte_pktmbuf_free(elt->buf); elt->buf = NULL; elt->wqe = NULL; - if (++elts_tail == RTE_DIM(*elts)) - elts_tail = 0; } txq->elts_tail = txq->elts_head; } @@ -208,7 +207,7 @@ struct txq_mp2mr_mbuf_check_data { struct mlx4dv_obj mlxdv; struct mlx4dv_qp dv_qp; struct mlx4dv_cq dv_cq; - struct txq_elt (*elts)[desc]; + struct txq_elt (*elts)[rte_align32pow2(desc)]; struct ibv_qp_init_attr qp_init_attr; struct txq *txq; uint8_t *bounce_buf; @@ -252,6 +251,12 @@ struct txq_mp2mr_mbuf_check_data { ERROR("%p: invalid number of Tx descriptors", (void *)dev); return -rte_errno; } + if (desc != RTE_DIM(*elts)) { + desc = RTE_DIM(*elts); + WARN("%p: increased number of descriptors in Tx queue %u" + " to the next power of two (%u)", + (void *)dev, idx, desc); + } /* Allocate and initialize Tx queue. */ mlx4_zmallocv_socket("TXQ", vec, RTE_DIM(vec), socket); if (!txq) {