From patchwork Thu Aug 31 06:37:14 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Santosh Shukla X-Patchwork-Id: 28115 Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [IPv6:::1]) by dpdk.org (Postfix) with ESMTP id 53C825A6E; Thu, 31 Aug 2017 08:38:15 +0200 (CEST) Received: from NAM01-SN1-obe.outbound.protection.outlook.com (mail-sn1nam01on0049.outbound.protection.outlook.com [104.47.32.49]) by dpdk.org (Postfix) with ESMTP id EDB82FFA for ; Thu, 31 Aug 2017 08:38:12 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=CAVIUMNETWORKS.onmicrosoft.com; s=selector1-cavium-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=AYcSNbo/aWb/S7Uvihx5dfBRdwukUNZe8SXoYFneTGM=; b=n+CDlkAFOQHQanz77OITi/nHDUKFFAwr9bELZ8fo3qSgTvC5LslIz+DYtMiogwgi3ONWbQmmgIopviRkrRhjUnbQYuqJ8vaUFlshT/MexYVrts0CS8JJu7Srwdt3ATba93qyGNa53nIFGGgnJaAA2p7TpX2fvOa0Sv1tC9bnemU= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Santosh.Shukla@cavium.com; Received: from localhost.localdomain (111.93.218.67) by MWHPR07MB3104.namprd07.prod.outlook.com (10.172.95.10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.13.10; Thu, 31 Aug 2017 06:38:08 +0000 From: Santosh Shukla To: dev@dpdk.org, olivier.matz@6wind.com Cc: jerin.jacob@caviumnetworks.com, john.mcnamara@intel.com, thomas@monjalon.net, hemant.agrawal@nxp.com, Santosh Shukla Date: Thu, 31 Aug 2017 12:07:14 +0530 Message-Id: <20170831063719.19273-6-santosh.shukla@caviumnetworks.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20170831063719.19273-1-santosh.shukla@caviumnetworks.com> References: <20170824132903.32057-1-santosh.shukla@caviumnetworks.com> <20170831063719.19273-1-santosh.shukla@caviumnetworks.com> MIME-Version: 1.0 X-Originating-IP: [111.93.218.67] X-ClientProxiedBy: PN1PR01CA0089.INDPRD01.PROD.OUTLOOK.COM (10.174.144.157) To MWHPR07MB3104.namprd07.prod.outlook.com (10.172.95.10) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 59cf8c8d-ab65-408d-c75d-08d4f03ad962 X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(300000500095)(300135000095)(300000501095)(300135300095)(22001)(300000502095)(300135100095)(2017030254152)(300000503095)(300135400095)(2017052603199)(201703131423075)(201703031133081)(201702281549075)(300000504095)(300135200095)(300000505095)(300135600095)(300000506095)(300135500095); SRVR:MWHPR07MB3104; X-Microsoft-Exchange-Diagnostics: 1; MWHPR07MB3104; 3:NJnGzIgU5PMzEeXZUy2l47dlyYXXSkDUXKNmji1prGQSmxRfMXphQhZStYYqoe6GjpTNmJYQ5Z18rrw0XU2RglVdaPSmyjk5cXxrqTPTpVqcUKuQBRJeQZsWtA3AmyJOY/9wCJ/yugfBGUuV0bJ9/lcAf0HcEvq1OyHu/hRMK0zgYywuifKJ7yssKe5hd0geqQUEy7dbO5tKeBap0Se14c9owhajUBguPEFs5dlDrSMM0rQjA9xgAqi6QATHlI42; 25:HUYDGPGcm0RLIuRlLyQIIs+YpJcSrr666PMGqSyH++u7lRO7uJ/lyh/FxsMGoLQdngs76/kqy7zJ/tvuVQjjJEU2fL4vzYzHg4FFWq3TMfZjkPaeM8KmaJGuWIcHHR7bLdhJgMn6BfoWYjwgV/y2O3whM1OXS1uYbaOaJIfFSKBa6u7L8lcuwpM9/y8rQ6+9fAm1Pqz8eUk4Lb2a9r60zYNgv4g6jecos/vsyIk1ZyIcOeCH5JHo+kQqn/ySCzXe9dE2Jk49uU9NU1tWW4V2nGrFzLUCUvvg/k8o+YSmLWp1iadxwCUrQ5LFGyXZrrOnD8btmCt2RHifVv5hUiq+uQ==; 31:DoXT+9TpwoOshWt3u31i3mY/zsLHEa5CO7vM25wC5f8R5MmjbHsTo7t5IloNYaGTkeLhNeGAUHvLQL1faRWsk3SRrHojiVDkbIvRbktahOYw+50JJ2TEsERFtq9dXwA2pcZbLEeBbkQA/Bk4lcn0H4vwOwde4u5rR4BPmnekrOzRxCrOVvf5oMRlIMxulmpbM3cAfHSvYDAM82/23825T5vaQA3edZizNFUc7n2cVMM= X-MS-TrafficTypeDiagnostic: MWHPR07MB3104: X-Microsoft-Exchange-Diagnostics: 1; MWHPR07MB3104; 20:VRMwlH5e3Ixy/fUhZLGMeMRfmKQ5gqeOZhPOelNRKz1aHKj8v8floAuFgjrStDzp7NnpEmMJ47nznFxHMGY1KxBCkpdgukplkHnt3yXELBRoOJ8J26zN4RIvlpC68vJ3g8eZVzv98DAkFdqxSeUqgR0cpNPmeLLnSlhme2SpSpi4Js9sgUUPwDfXF3ekCd4J3RI1BuuOlYgHFSDtCyToignTw5xW9KKwS4kb318K0PWHVyZAPfnOMwp0MkK8YDcYEwZ/z5lD5gOBQv0JbjN7t8pm19LugyyhXx03cig34RTyM9o9nz3IonxhqmO/wrNdnTF1E9rXEXp9AhLtcR5Yhzr/VjtWKg7PK5pefNP/4aKCrSsJanLAY2KXRFjlnBDcT8s6j9jOHgq5/hmljHGY9lHKpjZUQkOYT1ZCnVrSk4g5pWcKxxwjrJb7zKbcVDRjxD0l9fBMnU9b9UcdS1XVmi4LNA4qZgP1hOx/xU0tUxm1yCN9cW3c3e/eQz4axdDMuE8mWbweqQjcl30m2DS3veXK2kXJFkwu1bZnrKZ8FN6My17tQMMoiZsWWBpjwMXT7GXAa9uJSljwIxbo7h/7SVxnlVD/klXlE2pv33Qu1J0=; 4:WcgPhT87a3qB9z/OnrdtasSxSE6EMxP/Ouv26VRlvC+WihJb3KLcA72nv1W0bvYuEnRUiixXw8kTwn7XCK7K4BQATYwQi4/eZR0rGJr9t9EfEWyjOdCamN9vmgyJ6BXbErZzBVpf1MKU4T1603JlWXP2tiMCvRZWFZZXswijT2ba9XkpuaOsaFnaSOVhaA2bJELG8PzxRBIDP09oyo8QPLjbctNDBWJxuoNjpu3KdHQ0uY0oCEZONkhstWbcU9k2 X-Exchange-Antispam-Report-Test: UriScan:; X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(601004)(2401047)(8121501046)(5005006)(10201501046)(100000703101)(100105400095)(93006095)(3002001)(6041248)(20161123564025)(20161123555025)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123560025)(20161123562025)(20161123558100)(6072148)(201708071742011)(100000704101)(100105200095)(100000705101)(100105500095); SRVR:MWHPR07MB3104; BCL:0; PCL:0; RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095); SRVR:MWHPR07MB3104; X-Forefront-PRVS: 04163EF38A X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(6009001)(6069001)(199003)(189002)(66066001)(6486002)(33646002)(50226002)(47776003)(6506006)(68736007)(36756003)(50986999)(101416001)(4326008)(76176999)(189998001)(25786009)(50466002)(48376002)(7736002)(5003940100001)(6512007)(105586002)(8656003)(478600001)(6666003)(2906002)(110136004)(107886003)(305945005)(53936002)(8676002)(42186005)(5009440100003)(72206003)(42882006)(2950100002)(575784001)(1076002)(97736004)(81166006)(81156014)(8936002)(3846002)(6116002)(5660300001)(106356001); DIR:OUT; SFP:1101; SCL:1; SRVR:MWHPR07MB3104; H:localhost.localdomain; FPR:; SPF:None; PTR:InfoNoRecords; A:1; MX:1; LANG:en; Received-SPF: None (protection.outlook.com: cavium.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; MWHPR07MB3104; 23:hrq7/hAOOH1NMz+ydwauIb7VD0y9Gxz63DSyQQbDu?= gzUWbbTVIMRQJAkeFdFR4kum2ZyLljB4D34R8hAe341rBrZnRejZDjUTAM7YSuRDY7ACdIOev0+Am6QpUr9WbTrns3BTS35zWTDkXFkZtduwlRwLAjdu0JCTAaGWF47vi14PQUG5UGRYwQxQa8c4VMvRD/8ABxSJe9eIX2cPy/z5f5TRxAu9Y+j7IKP3K5zATfbmAS4XiE2D2Nfz+MJI0YynZ42AcL6kjrVwH7NRURitMZOtp8gGU5dctOZIE573mKkoGX/OgkL9uIPoemOKtzDhLs98eg3U3+qbqDYifsKpUhybjEWDQvmAS3obdg7hLo2U6S8nCTgDOfV+TU4JCZbHlfbabqiP/mqyZVfn7qb4cXsq2jcvYbu7XZR6HrdsJWKJaBp+49iKVTaSY1mM98wD5CJRXKmtjz9dQMEmjR6t6u18poT78uAIK1WgdQntR4vzH+Z2LweKDqb+H8kr3EI823+AVktnwPvsLgkTFjeHVMWcPppsCTXQwCozK12rYaef1UDT7Soa1lfeyUVB59DtpzGMIN4xHz2DdkvkLYTa2WyigPZWOrb05l7zAqAfuN7qLmnuilZ93Glv7xiNqYNzWrIbihCca0Z3+ce9DOsT5MU+cyhoCFWGrHBXmUEAkLCQBZzk/6gptDV8n/sqZ9+p612uA7p0+N89pti1fl+xgaBnKhRf9huIgP7EmsdiCrRMniqAsgxEzK9POdPFiDSSywYdT+c1Kqrl58SFxiBdtnXXRpMoQRFWgDaAOR6hPQFWxsU1/ZjuLOVWq3tZ1k5UF7khqLvTvRic0MpoLnSCpV1Hu+AQ8IIofXwDw2NHTUxIRlBnLViGoX1SfrT1jhsBEVYJoGszyLQYKteNiMoFIdRAtHJR73Pn0txjO00AbXq3cZDgQbprPCtaQaDflDNrlIcFXP856K/mKmi/+Hh002yeN1YEOV4P1PZMx6uYNp4W+gv1An6LL+waMflgp/fV8254vaExeNT+Umh81uJRYY27pATCu6NnpI3rjkHgcL/UdtSbhUCKNpzReT44r4JvuxrNFQYJBW18KEHhLPfp/BzsR0iFRGfuNHAC+3Imyqh9l/An3UdkgXAwsT4A4LCBgtf6IvgkSAN11y4yb1sbQ== X-Microsoft-Exchange-Diagnostics: 1; MWHPR07MB3104; 6:zb0GiVVIpm4XcKexcFBz3uydyo41a/sabwkcMgNL1KhmBcAjJ0Cy59ml1FmczvNHu21qDmk6Dwh1AcgSpvAejlOvzCJa6hJ/6msDWc4SmHftblC53EhPrwWNu4K7iNhb7wdoPQbCcb1PYCUXORF++0dcqAyuhaPpybPUVekTmQE0bJ15m2HP2854yMbCAxO1TrSMKW2Kv5RrZxeVYGa1M0qcOD6bI4gpdE/YoI2iU6TzCkk/NKNQzPK5qw+cJXLnWaJfNPESvejGclA8te9otUvXPWflHYHpmtckD21BzlrUC0PZPGM6jILWrzSmtMhV5M1IZy+8XG73Box1LyOxug==; 5:ziPzKyrl4GQNoXghNYjkvXq0tiZpYdL/XMk5mIIbf66IkEqSDJq8tiPlJnEQIZjobQPs1Fwil0zlwyq+BiDRFFgv0ywOdCfwfuqhCA/Ag8mnBNnRP3VIJhsU0tVKi05BcAkSP1iL41FG34wBSSsgmA==; 24:2/4M+N7WRwC8gjsY11pnn/u7wmk1sED7h7i5Bqn+u7XOHh7f4KrH7UoJjLYueAHN6rIs0Xb1y2CQUzN++IG69o7psBt8skgfvXE9AWDqBOA=; 7:Fh698x3rCOgz61px8qNB82hdWV2yXOq/fpoy7d8pYRSCgUEjbA2AJUqwbcQ4N4V1L6wAalmP4N9i/Y1jLsV7UgJ25XLJMqNkR42jnMRSfMkICLapgEpCXzTY44miR/lZ4M86U+P45snZ8Cbp6htS83ugAuFb93PdilbzmT3Oo59I0Psc5Jr4iB404EWtqStWdnqn914idVrq7qSNVoltOju7xMnlBWxa5gO4kPm7Ll4= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: caviumnetworks.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 31 Aug 2017 06:38:08.5575 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 711e4ccf-2e9b-4bcf-a551-4094005b6194 X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR07MB3104 Subject: [dpdk-dev] [PATCH v2 05/10] mempool/octeontx: implement pool free X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Upon pool free request from application, Octeon FPA free does following: - Uses mbox to reset fpapf pool setup. - frees fpavf resources. Signed-off-by: Santosh Shukla Signed-off-by: Jerin Jacob --- v1 --> v2: - Using newly introduced api 'octeontx_fpa_bufpool_gpool()' for getting the pool-id from handle. drivers/mempool/octeontx/octeontx_fpavf.c | 111 ++++++++++++++++++++++++ drivers/mempool/octeontx/octeontx_fpavf.h | 2 + drivers/mempool/octeontx/rte_mempool_octeontx.c | 12 ++- 3 files changed, 124 insertions(+), 1 deletion(-) diff --git a/drivers/mempool/octeontx/octeontx_fpavf.c b/drivers/mempool/octeontx/octeontx_fpavf.c index c0c9d8325..44253b09e 100644 --- a/drivers/mempool/octeontx/octeontx_fpavf.c +++ b/drivers/mempool/octeontx/octeontx_fpavf.c @@ -581,6 +581,117 @@ octeontx_fpa_bufpool_create(unsigned int object_size, unsigned int object_count, return (uintptr_t)NULL; } +/* + * Destroy a buffer pool. + */ +int +octeontx_fpa_bufpool_destroy(uintptr_t handle, int node_id) +{ + void **node, **curr, *head = NULL; + uint64_t sz; + uint64_t cnt, avail; + uint8_t gpool; + uintptr_t pool_bar; + int ret; + + RTE_SET_USED(node_id); + + /* Wait for all outstanding writes to be comitted */ + rte_smp_wmb(); + + if (unlikely(!octeontx_fpa_handle_valid(handle))) + return -EINVAL; + + /* get the pool */ + gpool = octeontx_fpa_bufpool_gpool(handle); + + /* Get pool bar address from handle */ + pool_bar = handle & ~(uint64_t)FPA_GPOOL_MASK; + + /* Check for no outstanding buffers */ + cnt = fpavf_read64((void *)((uintptr_t)pool_bar + + FPA_VF_VHAURA_CNT(gpool))); + if (cnt) { + fpavf_log_dbg("buffer exist in pool cnt %ld\n", cnt); + return -EBUSY; + } + + rte_spinlock_lock(&fpadev.lock); + + avail = fpavf_read64((void *)((uintptr_t)pool_bar + + FPA_VF_VHPOOL_AVAILABLE(gpool))); + + /* Prepare to empty the entire POOL */ + fpavf_write64(avail, (void *)((uintptr_t)pool_bar + + FPA_VF_VHAURA_CNT_LIMIT(gpool))); + fpavf_write64(avail + 1, (void *)((uintptr_t)pool_bar + + FPA_VF_VHAURA_CNT_THRESHOLD(gpool))); + + /* Empty the pool */ + /* Invalidate the POOL */ + octeontx_gpool_free(gpool); + + /* Process all buffers in the pool */ + while (avail--) { + + /* Yank a buffer from the pool */ + node = (void *)(uintptr_t) + fpavf_read64((void *) + (pool_bar + FPA_VF_VHAURA_OP_ALLOC(gpool))); + + if (node == NULL) { + fpavf_log_err("GAURA[%u] missing %" PRIx64 " buf\n", + gpool, avail); + break; + } + + /* Imsert it into an ordered linked list */ + for (curr = &head; curr[0] != NULL; curr = curr[0]) { + if ((uintptr_t)node <= (uintptr_t)curr[0]) + break; + } + node[0] = curr[0]; + curr[0] = node; + } + + /* Verify the linked list to be a perfect series */ + sz = octeontx_fpa_bufpool_block_size(handle) << 7; + for (curr = head; curr != NULL && curr[0] != NULL; + curr = curr[0]) { + if (curr == curr[0] || + ((uintptr_t)curr != ((uintptr_t)curr[0] - sz))) { + fpavf_log_err("POOL# %u buf sequence err (%p vs. %p)\n", + gpool, curr, curr[0]); + } + } + + /* Disable pool operation */ + fpavf_write64(~0ul, (void *)((uintptr_t)pool_bar + + FPA_VF_VHPOOL_START_ADDR(gpool))); + fpavf_write64(~0ul, (void *)((uintptr_t)pool_bar + + FPA_VF_VHPOOL_END_ADDR(gpool))); + + (void)octeontx_fpapf_pool_destroy(gpool); + + /* Deactivate the AURA */ + fpavf_write64(0, (void *)((uintptr_t)pool_bar + + FPA_VF_VHAURA_CNT_LIMIT(gpool))); + fpavf_write64(0, (void *)((uintptr_t)pool_bar + + FPA_VF_VHAURA_CNT_THRESHOLD(gpool))); + + ret = octeontx_fpapf_aura_detach(gpool); + if (ret) { + fpavf_log_err("Failed to dettach gaura %u. error code=%d\n", + gpool, ret); + } + + /* Free VF */ + (void)octeontx_fpavf_free(gpool); + + rte_spinlock_unlock(&fpadev.lock); + return 0; +} + static void octeontx_fpavf_setup(void) { diff --git a/drivers/mempool/octeontx/octeontx_fpavf.h b/drivers/mempool/octeontx/octeontx_fpavf.h index 23a458363..28440e810 100644 --- a/drivers/mempool/octeontx/octeontx_fpavf.h +++ b/drivers/mempool/octeontx/octeontx_fpavf.h @@ -136,6 +136,8 @@ octeontx_fpa_bufpool_create(unsigned int object_size, unsigned int object_count, unsigned int buf_offset, char **va_start, int node); int +octeontx_fpa_bufpool_destroy(uintptr_t handle, int node); +int octeontx_fpa_bufpool_block_size(uintptr_t handle); static __rte_always_inline uint8_t diff --git a/drivers/mempool/octeontx/rte_mempool_octeontx.c b/drivers/mempool/octeontx/rte_mempool_octeontx.c index 73648aa7f..6754a78c0 100644 --- a/drivers/mempool/octeontx/rte_mempool_octeontx.c +++ b/drivers/mempool/octeontx/rte_mempool_octeontx.c @@ -74,10 +74,20 @@ octeontx_fpavf_alloc(struct rte_mempool *mp) return rc; } +static void +octeontx_fpavf_free(struct rte_mempool *mp) +{ + uintptr_t pool; + + pool = (uintptr_t)mp->pool_id; + + octeontx_fpa_bufpool_destroy(pool, mp->socket_id); +} + static struct rte_mempool_ops octeontx_fpavf_ops = { .name = "octeontx_fpavf", .alloc = octeontx_fpavf_alloc, - .free = NULL, + .free = octeontx_fpavf_free, .enqueue = NULL, .dequeue = NULL, .get_count = NULL,