From patchwork Fri Apr 27 16:25:55 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shreyansh Jain X-Patchwork-Id: 39124 Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id DCEE5AACD; Fri, 27 Apr 2018 18:10:13 +0200 (CEST) Received: from EUR01-VE1-obe.outbound.protection.outlook.com (mail-ve1eur01on0068.outbound.protection.outlook.com [104.47.1.68]) by dpdk.org (Postfix) with ESMTP id CE2EDAAD7 for ; Fri, 27 Apr 2018 18:10:10 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nxp.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=+2nOF/uyKi/nEzID3P8SNuoFJWj1ST/A/217G58B5dY=; b=URrvZxtEH5b38d7Biy6bo5unaUyPoWm8ft3lqMtgF/A7MhdU5XEMdgLUV6qCLlo1Hz/adtSrmWr/8S42yC6yqgY+HQO3POoD4bDRNNDD921OPIfea/DRe/xeoss9Epx6SWNKDy57G3zbMGAL8Cq0Wfhm2y02jLsVOUe7r2cdLhg= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=shreyansh.jain@nxp.com; Received: from Tophie.ap.freescale.net (14.142.187.166) by DB6PR0402MB2774.eurprd04.prod.outlook.com (2603:10a6:4:96::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.696.15; Fri, 27 Apr 2018 16:10:06 +0000 From: Shreyansh Jain To: thomas@monjalon.net, dev@dpdk.org Cc: hemant.agrawal@nxp.com, akhil.goyal@nxp.com, anatoly.burakov@intel.com, Shreyansh Jain Date: Fri, 27 Apr 2018 21:55:55 +0530 Message-Id: <20180427162556.22781-3-shreyansh.jain@nxp.com> X-Mailer: git-send-email 2.14.1 In-Reply-To: <20180427162556.22781-1-shreyansh.jain@nxp.com> References: <20180427162556.22781-1-shreyansh.jain@nxp.com> MIME-Version: 1.0 X-Originating-IP: [14.142.187.166] X-ClientProxiedBy: OSAPR01CA0035.jpnprd01.prod.outlook.com (2603:1096:603:14::23) To DB6PR0402MB2774.eurprd04.prod.outlook.com (2603:10a6:4:96::11) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-HT: Tenant X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(4652020)(48565401081)(5600026)(4534165)(7168020)(4627221)(201703031133081)(201702281549075)(2017052603328)(7153060)(7193020); SRVR:DB6PR0402MB2774; X-Microsoft-Exchange-Diagnostics: 1; DB6PR0402MB2774; 3:kPYF2zcFbPXvPpKVUsio7RCvc6SU80jpDEj+voK+GYo9C9mEDb0vaCwGfWDrvKGhctEkx0rC9JrResCWsM7L1bERZ9RN5lLxqMlfMReTQP9e1RTzq+17PrIcmdUaBanKv6dpVoNHqsiPaInVKLCW04qZbH8uOdOlZkACm32fjnntRM4H+4HVmG71CoYZMgMIEoiOfWC6TLrsmrTfXtcF4zlfI1bnTiG2+m7fpOGkGdIig7fy8N199ZZThBP+zR8y; 25:UmhQ61tPZ8ea408Ms6yjU6H2hKbwN4wW/MQvkGxee5oWYOVp9P+lf5b8qppsBlCc12lFfQl9UUlKX1ufbciJyga2mTHhApyV4netTbAFsqZGtCHDSXNWteyRSeaRqs00kk9QR/yi8PDIZRMH4Me60+BS2gV01O4rprPJfw5uPWQAftr9zX1VsandiIcmYEu+8k1dLPFe9ySr8yAgqqi19/LpEI3KljemZV/dSH4mQXNI/pXNqbdvy6o0sk854L2fco4nbtXPyZq/T1BAhPw2sAqBpuKB9OO3BGkF3YUoTv2LSF4G9FBNjO+OZLPrz/ZeUM9+TP13ho812ukOKCGXbw==; 31:2bVs0/xmG4VIvnNNtktwHzVEMRNHuMWGI/pKUbkDVC3Mmm7BGj8QedllsXAwj6xBOD4Z0BbIG/uDpBcJ62nDs8NVuAv2XauE784/8TeH/MSCtBRH1IyRi+jxR4dANalAnbRvZi1GYpJ1WeM7ZdE4MU+RsZ1xuhK5ZErcYc3GL/lDygjKK2E+i39PEUv3yYxPPL0MCuCaaGBffXqkhyYlXUIryOpRL2UdXmQOU7A07r4= X-MS-TrafficTypeDiagnostic: DB6PR0402MB2774: X-Microsoft-Exchange-Diagnostics: 1; DB6PR0402MB2774; 20:YGhLFZmTiaE8fiw1JH/EbEU5PKTwFaYmzORUZTcuuOK7S30YPZXWZ5BUkAh7ns9odDGAAnDpkaZ3qylrsjk+JMGfZL91kOVXfunECKNaWombCAJPP1TwshW6ltnEfc50oVKJAJorZd5SGwVWAq3/CW5VVJbM4U9kjg0gEgwUI22/hCLibHkBbbuvtUFDDYBRuNnM0SM4J6EJMn8NihpZGe61VA49QF6Rj2ejTXQIJOmPusJ2uBFDLlDbhJG498PcCOinNaW3lVnb9rnKTT6ZY5EHOmYbBNPy7uNS3jTyxcZYAThmYBhknyx9wh0I2K0tq60a9l7ebv9XXDU+qvdXrVi79LVX6V09EsGhbtjPdkCcSvO52L6oZAULdPgDx1P/60s8m/rP8upTMX1uN/EvGGLahy8w4d17xVyIYA5HbFEtXdGEeUepU+PdBt0O6kvJ03Nihyvhw//EL88XQdhLBzmQNWeYFgJj9p4TsyVQ+HbecvUzgKy9FOf2leCJtewa; 4:tyTSfH44Xsv9QUrs7R0fNdHYZuDF7As9LwPDMm8paEJfOhIW3QLDXCpmHtWz0mxl0+AsWlK7Rhkpg72NMa0Ola0JRwnk1DVC4Dh+3BFOCIF7I72hcTFf13j+afJXYV3hFmky9J0LpsKQwpUO4mIf3+bDiUwczz9AnbTlkJ+YgNjudqLXNnbxaEnHyb3wD7AHSeefPm0EM/a55sSZwPKZfx2JSOIiwMGIi6YnvA8Dv8EUFGb6jOyhA4Rf5tMKJT/rhzvbIiDVwWzvSCuNErVpfLVjmSUAKu5Mb1VSsUIya+XFX9dIIUeNbBb8sTHURXBwXxpT/p1CsfAkeBAlP0UxvIPOnetHGqiaofsoQk01VIQ= X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(185117386973197)(275809806118684); X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(5005006)(8121501046)(3002001)(10201501046)(3231232)(944501410)(52105095)(93006095)(93001095)(6055026)(6041310)(20161123562045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123564045)(20161123560045)(20161123558120)(6072148)(201708071742011); SRVR:DB6PR0402MB2774; BCL:0; PCL:0; RULEID:; SRVR:DB6PR0402MB2774; X-Forefront-PRVS: 0655F9F006 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(39860400002)(346002)(366004)(39380400002)(376002)(396003)(189003)(199004)(36756003)(68736007)(478600001)(8676002)(3846002)(5009440100003)(52116002)(8936002)(51416003)(6116002)(16586007)(81166006)(1076002)(105586002)(2906002)(5660300001)(81156014)(106356001)(50226002)(48376002)(50466002)(305945005)(25786009)(6512007)(6486002)(7736002)(316002)(6666003)(2616005)(16526019)(186003)(11346002)(76176011)(55236004)(44832011)(26005)(53936002)(486006)(386003)(476003)(956004)(97736004)(66066001)(59450400001)(86362001)(6506007)(446003)(4326008)(47776003)(110426005); DIR:OUT; SFP:1101; SCL:1; SRVR:DB6PR0402MB2774; H:Tophie.ap.freescale.net; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; Received-SPF: None (protection.outlook.com: nxp.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; DB6PR0402MB2774; 23:bFI+Ynym5ZjGN8QtEvS5DiQDX4doU8qlhS/9INZ?= tbKRLJsJhKqRaPpULXqkM2DrjPnyTuO9Sm39JjcKZVfHbz2gEdBMvT97qmFnV165P7VIj6J+ut3txKmSzKtg+hFsS8t3o2+jJFSqapJWQiyIhvHD/uXLcVlidR4//mMh/eTNO34R2CK9R8Mo1YE/EvcaAR9xGGkPpF3N7clskqwCjKMBIKhV4SeA82cxKiOpzrKBJZkjSR+Y/LdZrKQL7goZF3SeavvDCF/7Q0cfE8ellOano15nKLmCGW0t20FghhSU4sU0sfeZN8Vd7WZV5ziWUYymY8pPIsfgAwwiDYrgrJEpjLE/PyuG/SLoauCUlaudZoOHZwtGSV4DQccsvjg44On0kCM4emsBbyEUBjP5Y3xmcw14c30rmjPP59H+0TMFef8CMj4koKO5Ad2B8ikgEvPtXEy9V9RujQKSWhY5xkznpZHbftjM9W3nJsRZV73jDWjyETf7H2OGaMsirNx4UNOi2TZZIJKzuVmVGGV3HNX7pJ6lqveuDTp8v995O36KGD3HBanwiPSENjf1I2OB0K3B1R4WkKOeca8xdYW3CrW72u6URXtSA3NukTgt5p0gAwebgq0lqsgxFA6+xQtloEgoWlMcZVTCegHcE5D2ZAlSnePzyu+8mBl3AK+7ILzDIg5mYVPvRaTsK7QIMBC05FUb0TYlsy5nNsTjoS1dWhik1tnnIrXmqbB738XD781UhemqXLKBX536xvjQ1cTtJAd7FEodrWxpy+D//PsOfaDgW1b9KmsZWUIikCjhXlIupDZtsaX966UOXAtGIoUh7h2XiskDm7Uad/xPh9QX/Ar2vTvOo4UzWs59g4rmcWrmTi2TuhYvlZqF7qUqjyo5QeZSRh7CNyHSK7ljw+CH24i44LEhsp0g91NQSWN/aj8FAz8nMdnIkIODjgi2jG/QC4NnZjccq9gZeT5QkcTuNCRMUIIAIT1gSXbtKlLBgD96FcbKQFT/1FIrFNFVn9ge+2Os5c5n6i6/4xqoI/O4z81NVDOO6XizM6ZeDwAh2npMAgHE0nhP2cQaC5vaY+4TYUj4TRSNqszgSofja7b6EdhPUbnLCb8fxwdB+L/3h3g1kiMJPTBAFZPN/iBaVbF1YfGpzUTmYjkIelXUFA1xMJlzA/6HT9RPl+s5UsYdpseVi3wfs0S5b2HKTwykbE88kBfpYrfTFRtQ2Pau7IgufB23uzb7/GVGg+KVSMFXQAYqvYkTtPiSY7SgjVPnoK/cA9H8OiQzBDX4aqPjeHq+LI10MBzZTPcJPpX+Eat9HGk4= X-Microsoft-Antispam-Message-Info: JU1MqRFtqzlzOOe54hufEKvb1ZNpzk8JsQIMPzW5sF9QieSlKy8Rru0c7VM9B5QOaizHCYbUe+dyCWq0cYVQAVjPV/nYnnS7lWg4MINUrcApKmx0uRlKRaNkf+06FaPdlPR5BUZq9RkZLkf500p21+C2S5shwPJTgnf3xAF1TGNaKRKlENRX1kkgXAARlBHt X-Microsoft-Exchange-Diagnostics: 1; DB6PR0402MB2774; 6:xzryX4HbvD23/l9SMS1w7FxZO1HM+d3Ye0dnAyl1Lb7HaD+eM78d8RCzw6Xas4aWIY+DgJKFTHlnDyiUUyecXAI37WLea0PQaX7td11i0ATDVgKNOwOi6EMapHD2SVgdou1xepk/kyAv/orNX2oItq8N0GrITmEH48yQrFENyBO28KMrZGOpFOo7ro7ZPMsMSiSWJ7n2UOsvXJFahloA54sKzvLYotzX7JzgNhuEUvsocrEWR2Xhb632wKMUVEqbMoAKfnjVqiYxxu+L48YQo4EoqJGdsGGf2pWrjrCSWyIGCkLh98C4xgQdUrSEsYeS+337d2mG8PpjO1buLxwJa1xh1RsEqTp5L0w6h8ATHdXAFpDALIIQcnfWlapD4EUJOT2It367kiudf9+sGE1FOqK8XLd+MI4sXLRgB/pgpz5YGhRVE5eaJnJHIQ3YgQIuF1QsOAS0t9oVg+iCaWjhvQ==; 5:/aDKRd741oMD+WSNqAUm2FEJTRCsTj9ud/wDlFuJgGOD8c++VAWiV9oCMCERkTCI0vhmW1+fnx7H4eNUr3FY2doPLQNJFDzbOZRnY2eXlZPvKppk9Envk7ga74WOh5RcFki6PyaIr894Puqx7l6Dlz0uXci2Gl7Nz80124bjJsg=; 24:Eu9S4G7rJ95O2uPb8O5kE/hrBoDpEMo8Qsl4OSbCjwor/YS1TMf1GIVJhHZgG2wlc3WiCdLFlN0xzc6ht7IvfGQhB6fn7um7ChyADeh/ctU= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; DB6PR0402MB2774; 7:Q0yAkzW7iR+aQpeoERl1na0VGOyrL7q9gZLPsJNebVvtGXNbmTSxVEwHJEgeSUkQSxCGu1r1dWaeeoXuP1VsWoF/RoMTvjO5++UoBX109SsYVlWBKs0Dpu5Q4WOIafjfQtKlLhY3mqgk/74U9sSNue2Lkfm2O/FzH4uiLh6kgTCpKawu3QYUKvf5Xkg7BBLXgNlMEwai5O0vWDpCy8SuhIkrTtXEmZ9YSqZiLxqPSl0/+rHv7xYK8IOsCVR5CNkM X-MS-Office365-Filtering-Correlation-Id: 13f936bf-88cc-44df-4479-08d5ac59597c X-OriginatorOrg: nxp.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Apr 2018 16:10:06.2113 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 13f936bf-88cc-44df-4479-08d5ac59597c X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 686ea1d3-bc2b-4c6f-a92c-d99c5c301635 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB6PR0402MB2774 Subject: [dpdk-dev] [PATCH 2/3] bus/fslmc: optimize physical to virtual address searching X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" With Hotplugging memory support, the order of memseg has been changed from physically contiguous to virtual contiguous. FSLMC bus and dpaa2 drivers depend on PA to VA address conversion when in Physical addressing mode. This patch creates a list of blocks requested to be pinned to the DPAA2 mempool. For searching physical addresses, it is expected that it would belong to this list (from hardware pool) and hence it is less expensive than memseg walks. Though, this has marginal impact on performance vis-a-vis legacy mode with physically contiguous memsegs. Signed-off-by: Shreyansh Jain --- An optimized algorithm is being worked upon based on some recent patches in hotplugging. That would improve/recover the performance. Until that time, this patch is to be treated a stop-gap solution. --- drivers/bus/fslmc/portal/dpaa2_hw_pvt.h | 23 +++++++++++++++++ drivers/mempool/dpaa2/dpaa2_hw_mempool.c | 43 ++++++++++++++++++++++++++++++++ 2 files changed, 66 insertions(+) diff --git a/drivers/bus/fslmc/portal/dpaa2_hw_pvt.h b/drivers/bus/fslmc/portal/dpaa2_hw_pvt.h index c76393d45..da6e639dc 100644 --- a/drivers/bus/fslmc/portal/dpaa2_hw_pvt.h +++ b/drivers/bus/fslmc/portal/dpaa2_hw_pvt.h @@ -254,15 +254,38 @@ enum qbman_fd_format { */ #define DPAA2_EQ_RESP_ALWAYS 1 +/* Various structures representing contiguous memory maps */ +struct dpaa2_memseg { + TAILQ_ENTRY(dpaa2_memseg) next; + char *vaddr; + rte_iova_t iova; + size_t len; +}; + +TAILQ_HEAD(dpaa2_memseg_list, dpaa2_memseg); +extern struct dpaa2_memseg_list dpaa2_memsegs; + #ifdef RTE_LIBRTE_DPAA2_USE_PHYS_IOVA extern uint8_t dpaa2_virt_mode; static void *dpaa2_mem_ptov(phys_addr_t paddr) __attribute__((unused)); /* todo - this is costly, need to write a fast coversion routine */ static void *dpaa2_mem_ptov(phys_addr_t paddr) { + struct dpaa2_memseg *ms; + if (dpaa2_virt_mode) return (void *)(size_t)paddr; + /* Check if the address is already part of the memseg list internally + * maintained by the dpaa2 driver. + */ + TAILQ_FOREACH(ms, &dpaa2_memsegs, next) { + if (paddr >= ms->iova && paddr < + ms->iova + ms->len) + return RTE_PTR_ADD(ms->vaddr, (paddr - ms->iova)); + } + + /* If not, Fallback to full memseg list searching */ return rte_mem_iova2virt(paddr); } diff --git a/drivers/mempool/dpaa2/dpaa2_hw_mempool.c b/drivers/mempool/dpaa2/dpaa2_hw_mempool.c index ce7a4c577..4c44c33cc 100644 --- a/drivers/mempool/dpaa2/dpaa2_hw_mempool.c +++ b/drivers/mempool/dpaa2/dpaa2_hw_mempool.c @@ -32,6 +32,13 @@ struct dpaa2_bp_info rte_dpaa2_bpid_info[MAX_BPID]; static struct dpaa2_bp_list *h_bp_list; +/* List of all the memseg information locally maintained in dpaa2 driver. This + * is to optimize the PA_to_VA searches until a better mechanism (algo) is + * available. + */ +struct dpaa2_memseg_list dpaa2_memsegs + = TAILQ_HEAD_INITIALIZER(dpaa2_memsegs); + /* Dynamic logging identified for mempool */ int dpaa2_logtype_mempool; @@ -358,6 +365,41 @@ rte_hw_mbuf_get_count(const struct rte_mempool *mp) return num_of_bufs; } +static int +dpaa2_populate(struct rte_mempool *mp, unsigned int max_objs, + void *vaddr, rte_iova_t paddr, size_t len, + rte_mempool_populate_obj_cb_t *obj_cb, void *obj_cb_arg) +{ + struct dpaa2_memseg *ms; + + /* For each memory chunk pinned to the Mempool, a linked list of the + * represeted memsegs is created for searching when PA to VA + * conversion is required. + */ + ms = rte_zmalloc(NULL, sizeof(struct dpaa2_memseg), 0); + if (!ms) { + DPAA2_MEMPOOL_ERR("Unable to allocate internal memory."); + DPAA2_MEMPOOL_WARN("Fast Physical to Virtual Addr translation would not be available."); + /* If the element is not added, it would only lead to failure + * in searching for the element and the logic would Fallback + * to traditional DPDK memseg traversal code. So, this is not + * a blocking error - but, error would be printed on screen. + */ + return 0; + } + + ms->vaddr = vaddr; + ms->iova = paddr; + ms->len = len; + /* Head insertions are generally faster than tail insertions as the + * buffers pinned are picked from rear end. + */ + TAILQ_INSERT_HEAD(&dpaa2_memsegs, ms, next); + + return rte_mempool_op_populate_default(mp, max_objs, vaddr, paddr, len, + obj_cb, obj_cb_arg); +} + struct rte_mempool_ops dpaa2_mpool_ops = { .name = DPAA2_MEMPOOL_OPS_NAME, .alloc = rte_hw_mbuf_create_pool, @@ -365,6 +407,7 @@ struct rte_mempool_ops dpaa2_mpool_ops = { .enqueue = rte_hw_mbuf_free_bulk, .dequeue = rte_dpaa2_mbuf_alloc_bulk, .get_count = rte_hw_mbuf_get_count, + .populate = dpaa2_populate, }; MEMPOOL_REGISTER_OPS(dpaa2_mpool_ops);