From patchwork Wed Dec 6 14:48:11 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matan Azrad X-Patchwork-Id: 31951 Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 5D58B1B03D; Wed, 6 Dec 2017 15:49:02 +0100 (CET) Received: from EUR01-HE1-obe.outbound.protection.outlook.com (mail-he1eur01on0059.outbound.protection.outlook.com [104.47.0.59]) by dpdk.org (Postfix) with ESMTP id 0B97A199B0 for ; Wed, 6 Dec 2017 15:48:58 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=+bRkXuSz3QsBKdkxnJLfnWHTRi6GybXq/cRo13HWEhQ=; b=DQNxyvYM0OzrKJJ9giBJ8FdDy4W5Jt1cTSfI/wt0LWojbmAZh98N2s7e4GkdFbRBrt8ELnglyqpF9pk/BKDIc0c2E6XLoin9PT1CKMwnbJWgOt7CjDv9yKj/ZfMZvnrMbW+bJpfemt7Ix4ZT4J2rQd83Vtw96u+hKlpU578Gx6Q= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=matan@mellanox.com; Received: from mellanox.com (37.142.13.130) by HE1PR0502MB3659.eurprd05.prod.outlook.com (2603:10a6:7:85::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.282.5; Wed, 6 Dec 2017 14:48:56 +0000 From: Matan Azrad To: Adrien Mazarguil Cc: dev@dpdk.org Date: Wed, 6 Dec 2017 14:48:11 +0000 Message-Id: <1512571693-15338-7-git-send-email-matan@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1512571693-15338-1-git-send-email-matan@mellanox.com> References: <1511871570-16826-1-git-send-email-matan@mellanox.com> <1512571693-15338-1-git-send-email-matan@mellanox.com> MIME-Version: 1.0 X-Originating-IP: [37.142.13.130] X-ClientProxiedBy: HE1P191CA0003.EURP191.PROD.OUTLOOK.COM (2603:10a6:3:cf::13) To HE1PR0502MB3659.eurprd05.prod.outlook.com (2603:10a6:7:85::17) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: ef588e76-8592-45d2-119e-08d53cb87aa2 X-MS-Office365-Filtering-HT: Tenant X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(5600026)(4604075)(4534020)(4602075)(4627115)(201703031133081)(201702281549075)(48565401081)(2017052603286); SRVR:HE1PR0502MB3659; X-Microsoft-Exchange-Diagnostics: 1; HE1PR0502MB3659; 3:ybZoFDMJszC6ZXN7ON7pNp0UBUYdv4uFNYJmmB+sFu4u0/QNiWS3iI0BXL6kmuwtFfVjcEoeGVeRYwJR1iaw/R53ey5swtwP6TNt7aJAKSnmNtBoqaB9o9QwarpKNkQ7La36k1YNow8pM7QVObt3rgidbvTIKRnf5PIR8dy2wC6YNT4HGLjmDyQ1NjtrRVm39u6vFTM/+XALZ4T2p1I2A0ZjNQKL+7e0eGbLwhvkdcYJ5SNbRdwaqGx/8H0niuG/; 25:SPw9lmy4MCLhT8UVmz+a8kV5GH1uByUX9y8tYowWLRUCBLODL1xgSXmytNjeTowk3zAHmk7eRwt3XhitVY66kK8nW3L/4zbwbtf0wtCSHcf6pff+sXQGvoKJfmh9ZX8w02Kz4subVzvFQwwUHu1UEyVpwMRZXqV+u9exnNJvwPPzFbnnJk2s7KiKyTn6b6E+1XCmPjdAs77MD6/X3Fgxjv2IhbvErsmbognK1FH/GfBrGh6QUMOMZfOE4RPhnNCAOmUGo4jwK5Jj+cat4vto6gt8BSb45TSy6jiA4miSxfcXGHC8in7OGsiWflchPC4dxJ8WgxmXhiOxSjJUnmK2JA==; 31:atE4B3rriEwPlxab5YyclMyBv3AjFZr5LMrllpHppeXvlqMNjnrg7QZA3pEzbXDusluOuO5Og7qGY/I7nWUSfZuYNRtc0rasNjXdQefF8cHuEN9nRGk/WfEEpR45SESk3ZliRh6wtiBoz7dXgB/ojlp8+lNUkI42CbNMUAaLAb4YP0xBFJXDE5TZK926tG5PZhNRHr+c+bZSLRjbIf+fLxfF0Qmq8tLewmHIynH74n8= X-MS-TrafficTypeDiagnostic: HE1PR0502MB3659: X-LD-Processed: a652971c-7d2e-4d9b-a6a4-d149256f461b,ExtAddr X-Microsoft-Exchange-Diagnostics: 1; HE1PR0502MB3659; 20:sHJl5E2jYZRvurQEfU1Lxf+hVSLJE9lapRPvnhBRM5vAutUt6gK9T87ziTAPJ3/q716HKss2nrGbeE0ZVHoJghvHx0whRxUyPkxDlkh0DofNPSpTx1bPgMYVrB+j+OqDukXTw4jO708gmjj0m7pX81EuBloL1yDOJUIF8lf7biwV6gl/x105O+6cZckSMcDxm8cO654cF8jpSKoDzcqZ8WFXPr9tYrCzRoswrP+fBVuBYVAp3dG4k+WBl2qggkXQSqLHe9Eeb+ifXMbtVUi5zRJZR8q22hR8ViZhSA4LCyPIijDqZ6S4k90CzUpvZq8+WurAdJUjdm6VEHStoo7w60PHPK6AL1PnBBreSF2cWE04O4mfZgKJzOCsmCyE/YgpGWTUjwicE7YiOTt4gkBI2eFsluq/iQ0YH3zTn3JegzYhDlQ6fIt+CW67b4WjT1xP6w8iobimewl2kQmaDavDuc+ISmYLjLNSfDUcQEnPzm718Htdy+xV7SKhrlTJ7OHA; 4:7lyaRzPh5VFIt+/ZZjyPUDZc3JeDNW2/oGVlrr+1hucPg8YyZo14mJyDIznTFBfwYhMXjd74utSc4CSFn3pyKjn72IR1/Y0nFcBKHcF2btp0qhzfATCYbsz6lqHe9ODbwVC2lfU1lv7+fC6R2Xsxzm8FkKZJjVOV/B56b4cM4ozlW6TbZ1Fe4NXEBR2f8iVsWFSZ0M5DoRx9IUvnqR/RSx1BwvdV98bdWYDSWUUqsrzb47/G8fWv5JFLmHOU/Ot+hfDodY39oxUMJCB+b2rr2A== X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(6040450)(2401047)(5005006)(8121501046)(3231022)(10201501046)(3002001)(93006095)(93001095)(6055026)(6041248)(20161123562025)(20161123564025)(201703131423075)(201703011903075)(201702281528075)(201703061421075)(20161123555025)(20161123560025)(20161123558100)(6072148)(201708071742011); SRVR:HE1PR0502MB3659; BCL:0; PCL:0; RULEID:(100000803101)(100110400095); SRVR:HE1PR0502MB3659; X-Forefront-PRVS: 05134F8B4F X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(7966004)(366004)(39860400002)(346002)(376002)(189003)(199004)(6916009)(2950100002)(6116002)(4326008)(16526018)(6666003)(3846002)(47776003)(8936002)(76176011)(508600001)(8676002)(81166006)(101416001)(97736004)(33026002)(86362001)(55016002)(4720700003)(21086003)(81156014)(105586002)(7736002)(66066001)(25786009)(2906002)(69596002)(305945005)(5660300001)(106356001)(33646002)(50226002)(51416003)(50466002)(16586007)(7696005)(52116002)(316002)(53936002)(68736007)(36756003)(48376002); DIR:OUT; SFP:1101; SCL:1; SRVR:HE1PR0502MB3659; H:mellanox.com; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; Received-SPF: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; HE1PR0502MB3659; 23:Yl16w23t5+YEGjwqwOJz/kmn2VmUJbsnPT8nG8b?= Jz8au+ngeTuyTTIaMuTPIumCzqt78JmJ/TH1Bxs6megDh/hskeVXBOGjAPuXVXoPeYyKOxGvCYRNyBigtqZ+2JYmOQikiXHWkJZs9iubOGnZriXBnpOAUPMBnE1Hc3Z0pnDgkahbd0NMXl+iu0XfTw91f8T70csFAq3rrzHSWYWK3MvDWaxt0adeMtIvqRm4JeQLhWxVqImjETP3fpKTvRFsR/Oh5GT5HfKiGakeUdGbNjLIYzWbUW1xdi1xjNHf1a6n13KY1P/wqc4QXeL06fBmm15Vm6xJmpuUp4PLO7M1hxMMhHyAzN888v651x7jCP8E/2f01wfGl0GPVCaR0vCfT790rxmjmQjC1lK0JQQ9EpJ70KaQyega+fRueCj1T0525wWmUvwyJN8mQfiGsdA2HMPHee1JTkSQ879PH2RZjYCOJFY2RYs5FKCCCX7dSvxWs6ezVoPB07YA0HXmRQkdZNfSvbhK9KqkXt+36UU9JoE7hlFa8foAjs3I28iFZ7Ca3XJ0QEB8kByJg2xYhsoFQzMUPPYtq+peEC7vpaPWv//AG+FTuh3fbyrMMeiUIDN07MUcPQOmhdtQ0gvBaUFzQ70TbVQEp/3Uk9A2luIEZbTobdaGWbOHsb1qBJq/V6yudYvPDEVuva2FMIZGbVezbXbOyjaBvdDxuzWpdvb1W98cQbnE+PNqZDlpoiXbuZcfHnO/mhMJTljcbXEuwOnY+WvWy2IMKGXH+sdS4BGL0wXQFWVHc6dIH1lHPYwTy/VsSzakKEbTwrxgBbPLAekVD5hXJZfXfJUm6dK8nWA/aY4DD7nlE7rYekLA0+QuCMzY/B92DmXIUOh9QzsxwAqAo+DjNQ8hL8hmAc3tfeM5C++j164fRbsYv5+yjOU32dJdvor/a0dqb1JSwvuE+XybMtsl/IhORL++Y09e7u6ibkSS4i1+Oo3/qWJohSByolSfId9R+5leHgAIhgvfbyXgVVkdkQMUggsIcUYACGreaLjWTtqfI5bg9013B12EFzvLzjdSiQ2Tn0n7MK7UhnKWXpyvrRLBibnwl+LV7SXRTmA9fLWFwW5/Ij5qegNJ/AN72S2+oACCwtAQHwvQWVUMk X-Microsoft-Exchange-Diagnostics: 1; HE1PR0502MB3659; 6:3QFV37HKGQV9BXuGnYbtH4+4xQ7gnNpfEBWs7ZuxGVl5g2ULt5rGpZqFa7yljRgh5piFT3zW8UNTwWJ2xyRLceTIdH6Ejgyhc+7a8Gi3aMGtJfNQlr3/643PN20pcZF8pKL6Qidrk9QoqY0M+JHzwqYXHLKsaPGWr+4UztwH59h8b5MlpJRob/C/fpDqdCWeS/F1chivkb9cILuFc+OmTNuZF6HCGHm1NJY2vPL0/gxEUJZRDiHYDK4GCIOFC17GgmqPVM9v7RBC3mksfeR7himwH0/nPMQVfScujECmez4Jvv/qbT0DmGEYFYfXXOF7foMW3pAbhSe7sg5VO1uEFD1B9r7E+hJv7dGP/2pSxyE=; 5:tJTZwCkBPq57wjjwEPwP9nHErbbkZfozjSop54Gl14PTYST5UuoCqvAJb5aaoeRrxWEQpGrEkBFOrmZQY43EjeTQub4kdUtvbN+pLXp1NuurFRHyRBZcRr4mYs2DNGQADg2bhxT+Cv1UP50nrBAmTyVeU1qRiixACMH4GJBJESs=; 24:0eYw6noRkakytvh6qeU7XTIR4F7jBCBlKKpvt6CiHxczZ02m/RJOkd1TRVeQFHSCe8mUtn9k0BNLGvRFzTN9ZqX2uY1piwdePmtcklAAo+s=; 7:Ei4KTSsjD5GetSHZbOadlMrJlHOBtSHguZnTRkvxCQfTZzxabcEerfBlnvJe8SOOhe5Ph+HN6WWkgj1c1WDHnHUVN2qWt+9A0WJACf1Uz87lr81fgxQA3FelhcALOEI551CRK0aa6381rNeDRwqPfiDfaZbVDMGuOc6xxlFtvef7i4cksB7uIe6bnlF0jEnzxauWSbHREcD6ON53UH9+JxEpxS4zlOmvZ0pUmUj8/aA+7rxpImzzpPtYISKZhxY6 SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Dec 2017 14:48:56.6571 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: ef588e76-8592-45d2-119e-08d53cb87aa2 X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-Transport-CrossTenantHeadersStamped: HE1PR0502MB3659 Subject: [dpdk-dev] [PATCH v2 6/8] net/mlx4: mitigate Tx send entry size calculations X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" The previuse code took a send queue entry size for stamping from the send queue entry pointed by completion queue entry; This 2 reads were done per packet in completion stage. The completion burst packets number is managed by fixed size stored in Tx queue, so we can infer that each valid completion entry actually frees the next fixed number packets. The descriptors ring holds the send queue entry, so we just can infer all the completion burst packet entries size by simple calculation and prevent calculations per packet. Adjust completion functions to free full completion bursts packets by one time and prevent per packet work queue entry reads and calculations. Save only start of completion burst or Tx burst send queue entry pointers in the appropriate descriptor element. Signed-off-by: Matan Azrad Acked-by: Adrien Mazarguil --- drivers/net/mlx4/mlx4_rxtx.c | 105 +++++++++++++++++++------------------------ drivers/net/mlx4/mlx4_rxtx.h | 5 ++- 2 files changed, 50 insertions(+), 60 deletions(-) diff --git a/drivers/net/mlx4/mlx4_rxtx.c b/drivers/net/mlx4/mlx4_rxtx.c index 2467d1d..8b8d95e 100644 --- a/drivers/net/mlx4/mlx4_rxtx.c +++ b/drivers/net/mlx4/mlx4_rxtx.c @@ -258,55 +258,48 @@ struct pv { }; /** - * Stamp a WQE so it won't be reused by the HW. + * Stamp TXBB burst so it won't be reused by the HW. * * Routine is used when freeing WQE used by the chip or when failing * building an WQ entry has failed leaving partial information on the queue. * * @param sq * Pointer to the SQ structure. - * @param wqe - * Pointer of WQE address to stamp. + * @param start + * Pointer to the first TXBB to stamp. + * @param end + * Pointer to the followed end TXBB to stamp. * * @return - * WQE size and updates WQE address to the next WQE. + * Stamping burst size in byte units. */ static uint32_t -mlx4_txq_stamp_freed_wqe(struct mlx4_sq *sq, volatile uint32_t **wqe) +mlx4_txq_stamp_freed_wqe(struct mlx4_sq *sq, volatile uint32_t *start, + volatile uint32_t *end) { uint32_t stamp = sq->stamp; - volatile uint32_t *next_txbb = *wqe; - /* Extract the size from the control segment of the WQE. */ - uint32_t size = RTE_ALIGN((uint32_t) - ((((volatile struct mlx4_wqe_ctrl_seg *) - next_txbb)->fence_size & 0x3f) << 4), - MLX4_TXBB_SIZE); - uint32_t size_cd = size; + int32_t size = (intptr_t)end - (intptr_t)start; - /* Optimize the common case when there is no wrap-around. */ - if ((uintptr_t)next_txbb + size < (uintptr_t)sq->eob) { - /* Stamp the freed descriptor. */ + assert(start != end); + /* Hold SQ ring wrap around. */ + if (size < 0) { + size = (int32_t)sq->size + size; do { - *next_txbb = stamp; - next_txbb += MLX4_SQ_STAMP_DWORDS; - size_cd -= MLX4_TXBB_SIZE; - } while (size_cd); - } else { - /* Stamp the freed descriptor. */ - do { - *next_txbb = stamp; - next_txbb += MLX4_SQ_STAMP_DWORDS; - if ((volatile uint8_t *)next_txbb >= sq->eob) { - next_txbb = (volatile uint32_t *)sq->buf; - /* Flip invalid stamping ownership. */ - stamp ^= RTE_BE32(0x1 << MLX4_SQ_OWNER_BIT); - sq->stamp = stamp; - } - size_cd -= MLX4_TXBB_SIZE; - } while (size_cd); + *start = stamp; + start += MLX4_SQ_STAMP_DWORDS; + } while (start != (volatile uint32_t *)sq->eob); + start = (volatile uint32_t *)sq->buf; + /* Flip invalid stamping ownership. */ + stamp ^= RTE_BE32(0x1 << MLX4_SQ_OWNER_BIT); + sq->stamp = stamp; + if (start == end) + return size; } - *wqe = next_txbb; - return size; + do { + *start = stamp; + start += MLX4_SQ_STAMP_DWORDS; + } while (start != end); + return (uint32_t)size; } /** @@ -327,14 +320,10 @@ struct pv { unsigned int elts_tail = txq->elts_tail; struct mlx4_cq *cq = &txq->mcq; volatile struct mlx4_cqe *cqe; + uint32_t completed; uint32_t cons_index = cq->cons_index; - volatile uint32_t *first_wqe; - volatile uint32_t *next_wqe = (volatile uint32_t *) - ((&(*txq->elts)[elts_tail])->wqe); - volatile uint32_t *last_wqe; - uint16_t mask = (((uintptr_t)sq->eob - (uintptr_t)sq->buf) >> - MLX4_TXBB_SHIFT) - 1; - uint32_t pkts = 0; + volatile uint32_t *first_txbb; + /* * Traverse over all CQ entries reported and handle each WQ entry * reported by them. @@ -360,28 +349,23 @@ struct pv { break; } #endif /* NDEBUG */ - /* Get WQE address buy index from the CQE. */ - last_wqe = (volatile uint32_t *)((uintptr_t)sq->buf + - ((rte_be_to_cpu_16(cqe->wqe_index) & mask) << - MLX4_TXBB_SHIFT)); - do { - /* Free next descriptor. */ - first_wqe = next_wqe; - sq->remain_size += - mlx4_txq_stamp_freed_wqe(sq, &next_wqe); - pkts++; - } while (first_wqe != last_wqe); cons_index++; } while (1); - if (unlikely(pkts == 0)) + completed = (cons_index - cq->cons_index) * txq->elts_comp_cd_init; + if (unlikely(!completed)) return; + /* First stamping address is the end of the last one. */ + first_txbb = (&(*txq->elts)[elts_tail])->eocb; + elts_tail += completed; + if (elts_tail >= elts_n) + elts_tail -= elts_n; + /* The new tail element holds the end address. */ + sq->remain_size += mlx4_txq_stamp_freed_wqe(sq, first_txbb, + (&(*txq->elts)[elts_tail])->eocb); /* Update CQ consumer index. */ cq->cons_index = cons_index; *cq->set_ci_db = rte_cpu_to_be_32(cons_index & MLX4_CQ_DB_CI_MASK); - txq->elts_comp -= pkts; - elts_tail += pkts; - if (elts_tail >= elts_n) - elts_tail -= elts_n; + txq->elts_comp -= completed; txq->elts_tail = elts_tail; } @@ -616,7 +600,7 @@ struct pv { if (max > pkts_n) max = pkts_n; elt = &(*txq->elts)[elts_head]; - /* Each element saves its appropriate work queue. */ + /* First Tx burst element saves the next WQE control segment. */ ctrl = elt->wqe; for (i = 0; (i != max); ++i) { struct rte_mbuf *buf = pkts[i]; @@ -691,6 +675,8 @@ struct pv { * that no ICRC should be calculated. */ if (--txq->elts_comp_cd == 0) { + /* Save the completion burst end address. */ + elt_next->eocb = (volatile uint32_t *)ctrl_next; txq->elts_comp_cd = txq->elts_comp_cd_init; srcrb.flags = RTE_BE32(MLX4_WQE_CTRL_SOLICIT | MLX4_WQE_CTRL_CQ_UPDATE); @@ -740,13 +726,14 @@ struct pv { elt->buf = buf; bytes_sent += buf->pkt_len; elts_head = elts_head_next; - elt_next->wqe = ctrl_next; ctrl = ctrl_next; elt = elt_next; } /* Take a shortcut if nothing must be sent. */ if (unlikely(i == 0)) return 0; + /* Save WQE address of the next Tx burst element. */ + elt->wqe = ctrl; /* Increment send statistics counters. */ txq->stats.opackets += i; txq->stats.obytes += bytes_sent; diff --git a/drivers/net/mlx4/mlx4_rxtx.h b/drivers/net/mlx4/mlx4_rxtx.h index d56e48d..36ae03a 100644 --- a/drivers/net/mlx4/mlx4_rxtx.h +++ b/drivers/net/mlx4/mlx4_rxtx.h @@ -105,7 +105,10 @@ struct mlx4_rss { /** Tx element. */ struct txq_elt { struct rte_mbuf *buf; /**< Buffer. */ - volatile struct mlx4_wqe_ctrl_seg *wqe; /**< SQ WQE. */ + union { + volatile struct mlx4_wqe_ctrl_seg *wqe; /**< SQ WQE. */ + volatile uint32_t *eocb; /**< End of completion burst. */ + }; }; /** Rx queue counters. */