From patchwork Sun Oct 8 12:40:06 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Santosh Shukla X-Patchwork-Id: 29888 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id B6ED71B1AB; Sun, 8 Oct 2017 14:41:14 +0200 (CEST) Received: from NAM03-CO1-obe.outbound.protection.outlook.com (mail-co1nam03on0072.outbound.protection.outlook.com [104.47.40.72]) by dpdk.org (Postfix) with ESMTP id 06AAE1B1AB for ; Sun, 8 Oct 2017 14:41:12 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=CAVIUMNETWORKS.onmicrosoft.com; s=selector1-cavium-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=X8xO9+qcF+alRokSF5+LdUZ+a9XGeaFh6MMIFdUq3hY=; b=dUnJQfRBJmt0sHALShHoMlYj2Im7CCJdFSWDCcz7KCmApxEzaBw9aztgtiATcIuypvla5BJI716NhFXYNcfPvCNHB34PnmYDg6FgVuCq93sipjahtdQX6isM/+wYM2lFKAI9lx0ylRVZuqRnThOKhkLWOzrlJQzA83F3OlMbO/4= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Santosh.Shukla@cavium.com; Received: from localhost.localdomain (14.140.2.178) by MWHPR07MB3103.namprd07.prod.outlook.com (10.172.95.9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.77.7; Sun, 8 Oct 2017 12:41:07 +0000 From: Santosh Shukla To: olivier.matz@6wind.com, dev@dpdk.org Cc: thomas@monjalon.net, jerin.jacob@caviumnetworks.com, hemant.agrawal@nxp.com, Santosh Shukla Date: Sun, 8 Oct 2017 18:10:06 +0530 Message-Id: <20171008124011.1577-6-santosh.shukla@caviumnetworks.com> X-Mailer: git-send-email 2.14.1 In-Reply-To: <20171008124011.1577-1-santosh.shukla@caviumnetworks.com> References: <20170831063719.19273-1-santosh.shukla@caviumnetworks.com> <20171008124011.1577-1-santosh.shukla@caviumnetworks.com> MIME-Version: 1.0 X-Originating-IP: [14.140.2.178] X-ClientProxiedBy: BMXPR01CA0045.INDPRD01.PROD.OUTLOOK.COM (10.174.214.31) To MWHPR07MB3103.namprd07.prod.outlook.com (10.172.95.9) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 4521a80f-3092-4f50-7eed-08d50e49da63 X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001)(2017030254152)(2017052603199)(201703131423075)(201703031133081)(201702281549075); SRVR:MWHPR07MB3103; X-Microsoft-Exchange-Diagnostics: 1; MWHPR07MB3103; 3:ajBbodRjLVLlVsR7OfvTJvVG4RnCvVYH5OVv3n8iGyop77iDxpBxqneXfNEMO4V6tdOuSw+Bcw2EsAY8U2fR8WyWEbd+eUiSJ3a6jPfiy7C5kWUS+avPcWwUcJXGD0QQAm4lfTsccnQb4u3Yo25OMHzccyxr+m8OcQSfKdCmNm32kzSNtaAlJBgyVxU1iBfStQ7I98zef2ryTUuKGUImyrQYRqpNbOFc1Cg552zs2TGGF6rrj9MV8rFUMwjqngr8; 25:60ceevDHO+qI4FDOSbQqk8pBQoie/DgZyj/3hA1pmrsPLeawNuZ6MyHxf2xzq55pPgJqDj5Q1sBBv6TUYtAgCcBYSNzmdxQRVz2po6DF5zJ58jBnjWUDhXVZYzMQeGPkrL7eseezEDbWi9bYPnpyxxktkxtMW8IsDENjBSdwO/96ncvhPYSJqUsTXYtOUK1J5alp5YCZpJFTAykKW1Cgb28XxBGqRtcxVWyHOl/EmfJ4sX7r3RiepVc/S7f54U4he/w2OwsqkXGaC6W92jiS6Xg69mKAxEFB0lMR2VaN3jE+RQdNAuQenj79YfBo3YBRXbIopYUiHzR1NA5ypF4dpA==; 31:veePacSeg8TvMcQpVnnWJ7t66DsyBdaDyDgMVPNU68lyFSWMCp3saS4G23lKXk1GastG13Ik0/yg0keM+rlRSuV3ZYF6fVtFpb5vpz1+LvRNQvMLyaDvfs//Y/3MFzRU3yaaTNqUiUnS0VRZtzJHFFkYneG9UZhE67ZKi9iCoFZK3iVi2v7yL1EouDSNyQ6e6Ob7Kw0Yyl/GCM5WGcsfeiqZukL53YjO8LdJp42FqfU= X-MS-TrafficTypeDiagnostic: MWHPR07MB3103: X-Microsoft-Exchange-Diagnostics: 1; MWHPR07MB3103; 20:brDA3EW3+XkqK2Fb3B6f5f86c3jEN/tKrci3OMb0kL/s24DPqNsQz6TwaV8wyEV65DSNUZ/OdSANudi9x/Bos10sMty9Zde1FedKzg6V4n5ZleYYLoEtkEEXcuLP8ml7FdVjNztOcsqKuIE2of/tEhifBnRDz6XmiVAbVhXH7wLpsyW2+BU/B9UH+h+rWQLKhE4UZn/DA2iBpvvDryGIgu5k00yfb1X/MWe8RAo0wFf6RIuAKfo7vYZxNPsES8LMgIXjh/gMwV69YmTGK7jLxjrtkdlZzjqi8D2CIfGx7YS9Y40HZ1p5hDHkxKKbmKDS58GIjuldhgJ6sy8grbo1nIqDkCfyoSyhY/dZ6kQZu4b5p4CXSAlTz99a6mYk8tWQFK9NZ+NPdWI8ijNusq04QP882DVVCWXOehq/BdAj4s0kPcPybTGWEHMYVy3s+8J5KxmUxIiJU7AAsRbUQUkT3xhvfmNlb7JWlLJLJZpsgU26FAiotuObjdlHatc//Ih5winO9eto1xlqHFJwLdC7VsfHMBfu78sH5BTgXG8kPfiefXXlgC3IBC83djw/hTWAAjpv1iCGnNo0eGVqN4wUqfCuzbDs65GGe5vXYKrg5qc=; 4:kGlFE19Ed4o4WDYtQBUxI2uKXsZi+Q6t4fnbm0RcuqbjAoQor+PlS/Z0zIUAbvK89aUOxDrxWIoXGVNQIq9YVi+lOEyGB/aOy24/V6uGmRqpEvTsogv1g1mN/hGr3vejnIrlQ9hygWh4aPJlO21mTW57KYjS5U3+AskfHPT42USApJLKA+8vR4jIWuyuEbZx4SBZA9Usb69ZvoX7yAx0+WSNpd5+NmP7tBzSFG78Dddb65czmTdif5S3tEZQXOvI X-Exchange-Antispam-Report-Test: UriScan:; X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(2401047)(5005006)(8121501046)(3002001)(100000703101)(100105400095)(93006095)(10201501046)(6041248)(20161123562025)(20161123560025)(20161123555025)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123558100)(20161123564025)(6072148)(201708071742011)(100000704101)(100105200095)(100000705101)(100105500095); SRVR:MWHPR07MB3103; BCL:0; PCL:0; RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095); SRVR:MWHPR07MB3103; X-Forefront-PRVS: 0454444834 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(6009001)(6069001)(376002)(346002)(189002)(199003)(3846002)(16526018)(1076002)(6512007)(2950100002)(33646002)(478600001)(97736004)(107886003)(50466002)(25786009)(106356001)(8936002)(48376002)(36756003)(4326008)(5660300001)(50226002)(105586002)(305945005)(5009440100003)(16586007)(189998001)(72206003)(66066001)(76176999)(81156014)(6486002)(5003940100001)(8656003)(68736007)(47776003)(316002)(53936002)(7736002)(81166006)(50986999)(2906002)(42882006)(575784001)(6116002)(8676002)(101416001)(6506006)(110426004)(4533004); DIR:OUT; SFP:1101; SCL:1; SRVR:MWHPR07MB3103; H:localhost.localdomain; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; Received-SPF: None (protection.outlook.com: cavium.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; MWHPR07MB3103; 23:gq4MorqlflMcV+u9u5tZXOonPY5hMxxQrrxXljhaC?= usN8Pb4onl52kmkPt9GfBmow+StTNA5Q50lYX/3aX+JfiVvEP1bmvKVZhqnl6MAixIZ93D+M/kJ0colIDcX2HPjXjxk7W3or76bqUon4bwZ0jasv7Ckb0qgfiHJpodVQZFTbYtALxDs6iDPCCy3mJZED1eD/uV46WAnwgn/M7sGxdiqpAQ+h6IbKXCGmSalTmrWivzbo3LdkSJ4KyeHrwq2bqdcqLwV53rS2gGQodeSimvjwjTsK4eGs6qZnqJa7lQNiCuh4lIH8gOqsJwGz+MdUZVWi7JTTxq2+y/pujG8v4za7CxszWFIYokLwXcb1LkG1KuLiVVr3/RqAu4bT6SDi4opHS/YdcZNIrgYUGJm/qqJ+m0y7HG+a14vtPwLgOw6d2HPZ+ioufmRNEP8B0sntbOlqTj+N39n6epL/3HbYmmfJXFoIECBSNO7Ok1vC1HGg32YBgmGwSOy9bL/eGBVCdoEf3yOjdJNjZPOkNyV6nImD5os4Zedu3FMNkapnLmD13j07RZtUItmJz+8koFJLKhxv+u1YQpz/p8y3svA+grYWPH6Nh2zpGct8FCzciYN9OmvPfP2FFjtKK82IVL7ko5PdZBD3h5iXaqz3aG3UFszrjp3MuqldRR6D2m/GPd5IXDILOh4j9jMa6+dKlYvI6lAbj5iqH9Lfx+Ur0rMt/w/XwfE6e+ku2yiNO2pUnfbOIjdS9GS1eu2Irdo3N1k7a0GX3u/eIGtl6ct5BP01M66HX2UzbaDc8U+D1zYTM8bQhi8fsUMhQaqCiD0ZrezQpiYqL+KexeTf9y/Z80qzs/ryvuicV6tG09RVYL865Itv+cXRtj+RtGlWEnxZ8kf2Mjq+o9MtDvcqE+JtRU4E0nY8VxMbwlCjeTnyD9XRcmPv1Y43biRZ+1gGj/aU8TRLDHT93YN8yRdB3CZy6sDofGxs4+hMmT4xEFzh+MDa7jObRgLAJdjIX5FmvFr5Dmtwyl0UJth6/Z0beAD1j6Scv5bUnMpmewUA84hzwsMUWA7DaqjegTQ0fTjxE66LZ01GOjdRumrdJi3JpOxW/fj1QME1uZBSYSR2OyzBGC+Dw0bzq3EIfvCIyqV8s0QHLL1dQfPtCAI2pzoUHPG2EhsLIAlsWPLRX+Uw73AFVsnSRo6mXAK6NC1+tb7u8ZJhy6dm3M3/ZKTG5jhz5Z9u5XX3w== X-Microsoft-Exchange-Diagnostics: 1; MWHPR07MB3103; 6:BsDDI4CH/qhHeNH3cEdkeYre46Za9SXW49IoqmXX7rlIwy/YKBfUBe8yFRMWJGj0bCzp17muOwbwnSIZtsRQlHpEVIMrTV9zNi9+7K/4LVvJ/Dh7pUg5WrxGakPbckDdsV9SYTCXJbMZYoYcJ7ZauipP3Y2Vagtztq3UTczmvQQLwMg5B+IdGw850Cm9FWLSHEp/YvfYsNKg4aO7Y9aG5urvyH/f+tYm+auuK9imJnnSlh5jCn4pL2/ZanOEaoCo3ZzC24j4nLBqH486zrvlT7Xy0G/JOVZnY6IjnlEXdd6F9PJzEbvUasJ8u/mxAmt6tFMEKHTEoFZirZaj5uIaBw==; 5:llu21XyY+OdRq2XMo+7OANPk9QZmWAkFV4r0P+ngY3jxw9M8ZfCUBgpq+sIn+iStNa+MVdmSrdWqDQFfrUKmsrojDvf7ivIvRea5uB7UI1vR5g2HEnyIGWxJi6IzW3v8JGrtOkBmpM/MW184OfeUfw==; 24:IfzgnOEWS9/u2rCl6KLIA++bzJ1sK0KxNikX+/wWbzr4tHaGQqQKY5Yy3XQiNFZ0XGnaPvzepddI5CAUoFdagP2jT/Lch1Cl6b698NHRBDQ=; 7:GvVKbEnchhC+OZ1N/EFO5kXkCGMFJoMvhL1xVPBc4tiy4p6PyAZUk6UjEUZi2XuBPGVfycStKGKoftozCPf42isTAB8OQ1kk7odi5bA/M5H1oPqGO13OUfd071rji2nUJDXgd7OdPrSc3lOxvDXu/grivXbAv3j5hMFLJalMbznh+xo3E+u5IPZ4rhdEKs+AsHGzQUQxhMoX5tp/HTPgSh6PbgBdq78yIeXdlQTvqUk= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: caviumnetworks.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Oct 2017 12:41:07.7460 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 711e4ccf-2e9b-4bcf-a551-4094005b6194 X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR07MB3103 Subject: [dpdk-dev] [PATCH v3 05/10] mempool/octeontx: add support for free X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Upon pool free request from application, Octeon FPA free does following: - Uses mbox to reset fpapf pool setup. - frees fpavf resources. Signed-off-by: Santosh Shukla Signed-off-by: Jerin Jacob --- drivers/mempool/octeontx/octeontx_fpavf.c | 111 ++++++++++++++++++++++++ drivers/mempool/octeontx/octeontx_fpavf.h | 2 + drivers/mempool/octeontx/rte_mempool_octeontx.c | 12 ++- 3 files changed, 124 insertions(+), 1 deletion(-) diff --git a/drivers/mempool/octeontx/octeontx_fpavf.c b/drivers/mempool/octeontx/octeontx_fpavf.c index c0c9d8325..44253b09e 100644 --- a/drivers/mempool/octeontx/octeontx_fpavf.c +++ b/drivers/mempool/octeontx/octeontx_fpavf.c @@ -581,6 +581,117 @@ octeontx_fpa_bufpool_create(unsigned int object_size, unsigned int object_count, return (uintptr_t)NULL; } +/* + * Destroy a buffer pool. + */ +int +octeontx_fpa_bufpool_destroy(uintptr_t handle, int node_id) +{ + void **node, **curr, *head = NULL; + uint64_t sz; + uint64_t cnt, avail; + uint8_t gpool; + uintptr_t pool_bar; + int ret; + + RTE_SET_USED(node_id); + + /* Wait for all outstanding writes to be comitted */ + rte_smp_wmb(); + + if (unlikely(!octeontx_fpa_handle_valid(handle))) + return -EINVAL; + + /* get the pool */ + gpool = octeontx_fpa_bufpool_gpool(handle); + + /* Get pool bar address from handle */ + pool_bar = handle & ~(uint64_t)FPA_GPOOL_MASK; + + /* Check for no outstanding buffers */ + cnt = fpavf_read64((void *)((uintptr_t)pool_bar + + FPA_VF_VHAURA_CNT(gpool))); + if (cnt) { + fpavf_log_dbg("buffer exist in pool cnt %ld\n", cnt); + return -EBUSY; + } + + rte_spinlock_lock(&fpadev.lock); + + avail = fpavf_read64((void *)((uintptr_t)pool_bar + + FPA_VF_VHPOOL_AVAILABLE(gpool))); + + /* Prepare to empty the entire POOL */ + fpavf_write64(avail, (void *)((uintptr_t)pool_bar + + FPA_VF_VHAURA_CNT_LIMIT(gpool))); + fpavf_write64(avail + 1, (void *)((uintptr_t)pool_bar + + FPA_VF_VHAURA_CNT_THRESHOLD(gpool))); + + /* Empty the pool */ + /* Invalidate the POOL */ + octeontx_gpool_free(gpool); + + /* Process all buffers in the pool */ + while (avail--) { + + /* Yank a buffer from the pool */ + node = (void *)(uintptr_t) + fpavf_read64((void *) + (pool_bar + FPA_VF_VHAURA_OP_ALLOC(gpool))); + + if (node == NULL) { + fpavf_log_err("GAURA[%u] missing %" PRIx64 " buf\n", + gpool, avail); + break; + } + + /* Imsert it into an ordered linked list */ + for (curr = &head; curr[0] != NULL; curr = curr[0]) { + if ((uintptr_t)node <= (uintptr_t)curr[0]) + break; + } + node[0] = curr[0]; + curr[0] = node; + } + + /* Verify the linked list to be a perfect series */ + sz = octeontx_fpa_bufpool_block_size(handle) << 7; + for (curr = head; curr != NULL && curr[0] != NULL; + curr = curr[0]) { + if (curr == curr[0] || + ((uintptr_t)curr != ((uintptr_t)curr[0] - sz))) { + fpavf_log_err("POOL# %u buf sequence err (%p vs. %p)\n", + gpool, curr, curr[0]); + } + } + + /* Disable pool operation */ + fpavf_write64(~0ul, (void *)((uintptr_t)pool_bar + + FPA_VF_VHPOOL_START_ADDR(gpool))); + fpavf_write64(~0ul, (void *)((uintptr_t)pool_bar + + FPA_VF_VHPOOL_END_ADDR(gpool))); + + (void)octeontx_fpapf_pool_destroy(gpool); + + /* Deactivate the AURA */ + fpavf_write64(0, (void *)((uintptr_t)pool_bar + + FPA_VF_VHAURA_CNT_LIMIT(gpool))); + fpavf_write64(0, (void *)((uintptr_t)pool_bar + + FPA_VF_VHAURA_CNT_THRESHOLD(gpool))); + + ret = octeontx_fpapf_aura_detach(gpool); + if (ret) { + fpavf_log_err("Failed to dettach gaura %u. error code=%d\n", + gpool, ret); + } + + /* Free VF */ + (void)octeontx_fpavf_free(gpool); + + rte_spinlock_unlock(&fpadev.lock); + return 0; +} + static void octeontx_fpavf_setup(void) { diff --git a/drivers/mempool/octeontx/octeontx_fpavf.h b/drivers/mempool/octeontx/octeontx_fpavf.h index 23a458363..28440e810 100644 --- a/drivers/mempool/octeontx/octeontx_fpavf.h +++ b/drivers/mempool/octeontx/octeontx_fpavf.h @@ -136,6 +136,8 @@ octeontx_fpa_bufpool_create(unsigned int object_size, unsigned int object_count, unsigned int buf_offset, char **va_start, int node); int +octeontx_fpa_bufpool_destroy(uintptr_t handle, int node); +int octeontx_fpa_bufpool_block_size(uintptr_t handle); static __rte_always_inline uint8_t diff --git a/drivers/mempool/octeontx/rte_mempool_octeontx.c b/drivers/mempool/octeontx/rte_mempool_octeontx.c index d930a81f9..6ac4b7dc0 100644 --- a/drivers/mempool/octeontx/rte_mempool_octeontx.c +++ b/drivers/mempool/octeontx/rte_mempool_octeontx.c @@ -74,10 +74,20 @@ octeontx_fpavf_alloc(struct rte_mempool *mp) return rc; } +static void +octeontx_fpavf_free(struct rte_mempool *mp) +{ + uintptr_t pool; + + pool = (uintptr_t)mp->pool_id; + + octeontx_fpa_bufpool_destroy(pool, mp->socket_id); +} + static struct rte_mempool_ops octeontx_fpavf_ops = { .name = "octeontx_fpavf", .alloc = octeontx_fpavf_alloc, - .free = NULL, + .free = octeontx_fpavf_free, .enqueue = NULL, .dequeue = NULL, .get_count = NULL,