From patchwork Thu Dec 27 06:23:19 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hemant Agrawal X-Patchwork-Id: 49307 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 2A50A5F51; Thu, 27 Dec 2018 07:23:37 +0100 (CET) Received: from EUR04-HE1-obe.outbound.protection.outlook.com (mail-eopbgr70047.outbound.protection.outlook.com [40.107.7.47]) by dpdk.org (Postfix) with ESMTP id 20D0A5B34 for ; Thu, 27 Dec 2018 07:23:21 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nxp.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=V0zDPId1tA37ylTk4KGJXHJqeDf5b5eTB8M7Fdo8Tok=; b=xiQPGApVCWgEuzCvibtYk7j3ASMyrIop4r6n8fZrxSAbYsaLXD4EHu62XIIUm9+pt0sdw/1OZDckmRqxwIxJtVXXmjgMJRY8oTTi9Ew2J2TNj6oNvL7kcQ6M60ENZ3daLNiLbvrlwFiSY8bVYnhroTtOkI4EpWJyKdnwkQPX4do= Received: from VI1PR0401MB2541.eurprd04.prod.outlook.com (10.168.65.19) by VI1PR0401MB2382.eurprd04.prod.outlook.com (10.169.134.11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1471.20; Thu, 27 Dec 2018 06:23:19 +0000 Received: from VI1PR0401MB2541.eurprd04.prod.outlook.com ([fe80::555e:164:f8f0:dbb4]) by VI1PR0401MB2541.eurprd04.prod.outlook.com ([fe80::555e:164:f8f0:dbb4%2]) with mapi id 15.20.1471.019; Thu, 27 Dec 2018 06:23:19 +0000 From: Hemant Agrawal To: "dev@dpdk.org" CC: "ferruh.yigit@intel.com" , Shreyansh Jain Thread-Topic: [PATCH 19/20] bus/fslmc: add support for secondary processes Thread-Index: AQHUnaypIrgbNA0uEk2pPjfeef0FxA== Date: Thu, 27 Dec 2018 06:23:19 +0000 Message-ID: <20181227062233.30781-20-hemant.agrawal@nxp.com> References: <20181227062233.30781-1-hemant.agrawal@nxp.com> In-Reply-To: <20181227062233.30781-1-hemant.agrawal@nxp.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [14.143.30.134] x-mailer: git-send-email 2.17.1 x-clientproxiedby: BM1PR01CA0120.INDPRD01.PROD.OUTLOOK.COM (2603:1096:b00:40::14) To VI1PR0401MB2541.eurprd04.prod.outlook.com (2603:10a6:800:56::19) authentication-results: spf=none (sender IP is ) smtp.mailfrom=hemant.agrawal@nxp.com; x-ms-exchange-messagesentrepresentingtype: 1 x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; VI1PR0401MB2382; 6:V5Khm4CF07gTb8NVuSHmcmUCWNE51hAPlktYpStL/2QcqsHs34QLmnLD4iYdXeebOeD6tj1Lz6zXcX86NlZeWtaNUWiJUznp3yUG2/pmscikeQ2kK2OhHC3iTjuRBLhvRisJx6PcfsPBCnEIfwzdcW3vexrVHiU1rb7sdhsNnbYYj4rdPOy1sQD4nPaMIm8lGg8gWxRwWYkobcuOOVuiz36HDm/5mnAMJK9uy185muypb1owWUDWs+Ny3efpSs79HCfQzMfg1SwDER2ZuBFW0/8SXC9uOTsU9h4FPD/2qiBUCD7QHChbLbc5qP2cV4ILpOjarhLlz7mQAcCvu7sR3dpm6BSp4BsYFt5a7iKP3YNO5MSNTpjStczzjhxFiQUbNiQYEmm6qW5QGTK4l0CxzJBL9vmFVOLO8yy1Lh1sBosK9Vf8LAjkOK5+yvF1Z0sEtjREVRhYkdKfwxjeTGpuHg==; 5:Cp7M46sYKWCQI6FbyrsuQkxDD82nlIKamWJnDhgHHOO9uY+PHBpgveFjN4ujzmxnoCyVWtofa23T0aPfcad2PferCCD9fZLoOdD4v0yVMgDVbsqpdJ+CzaYXWiVp61qkEgTVf2rRU8v+wsQQo6was9LWX6hJWvBh6tHNxX7991Y=; 7:39wxUqeSvcfen+SIZavDwRgM+++Ehz3bXeScVwqRU3Zp+rmX8u3CaknGYHO4kGEO9mM75488uqDzkr1tQinJj3yewXTn/44JYiq0bLtGqjPSnwqjMVv7DdSE2hynMFpOot7M1LmnC5xIuXvlRrfvgA== x-ms-office365-filtering-correlation-id: 1255d6ce-6de0-4277-83ec-08d66bc3cb71 x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(7168020)(4627221)(201703031133081)(201702281549075)(8990200)(5600074)(711020)(4618075)(2017052603328)(7153060)(7193020); SRVR:VI1PR0401MB2382; x-ms-traffictypediagnostic: VI1PR0401MB2382: x-microsoft-antispam-prvs: x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(8211001083)(3230021)(908002)(999002)(5005026)(6040522)(2401047)(8121501046)(10201501046)(3002001)(93006095)(93001095)(3231475)(944501520)(52105112)(6055026)(6041310)(20161123560045)(20161123562045)(20161123564045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123558120)(201708071742011)(7699051)(76991095); SRVR:VI1PR0401MB2382; BCL:0; PCL:0; RULEID:; SRVR:VI1PR0401MB2382; x-forefront-prvs: 0899B47777 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(136003)(396003)(366004)(376002)(39860400002)(346002)(199004)(189003)(5660300001)(1076003)(26005)(76176011)(186003)(71200400001)(256004)(25786009)(105586002)(4326008)(6436002)(305945005)(11346002)(106356001)(14444005)(446003)(44832011)(476003)(7736002)(55236004)(102836004)(78486014)(486006)(386003)(68736007)(52116002)(86362001)(6506007)(2616005)(71190400001)(66066001)(316002)(54906003)(6116002)(3846002)(2906002)(14454004)(36756003)(99286004)(50226002)(8936002)(1730700003)(81156014)(81166006)(8676002)(6512007)(53936002)(478600001)(97736004)(6486002)(6916009)(5640700003)(2351001)(2501003); DIR:OUT; SFP:1101; SCL:1; SRVR:VI1PR0401MB2382; H:VI1PR0401MB2541.eurprd04.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: nxp.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: TQzfRMRdrsTD7tA1C2Xk/CT5xguDyyClcmVp4JqGzqkeC+/vbbI+qEf/rkyRA9D2XAabb0/VGNrheZxRZB49PHGJ7hLAhpPF6Em15ZulzBVB/O19I1SHGoYiBkJV1+kWXqkF+GDV+1JwbxN76zgsbVggQyRJmxNav5PPVc7bvVOdOzS6sbaUGfZqX+fu4BKMux1bmAlU8tjulDaf3AJVh+kOrlE/nir3TjFGSyBH+rNfjC54aL752/4lcLG74J/ocD4sN6MZyYaYtWt8rl/fSjzptfB1I/vPNnfEltCE8GRO8WClJAuEMjQ/1ObBOHYa spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM MIME-Version: 1.0 X-OriginatorOrg: nxp.com X-MS-Exchange-CrossTenant-Network-Message-Id: 1255d6ce-6de0-4277-83ec-08d66bc3cb71 X-MS-Exchange-CrossTenant-originalarrivaltime: 27 Dec 2018 06:23:19.8637 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 686ea1d3-bc2b-4c6f-a92c-d99c5c301635 X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0401MB2382 Subject: [dpdk-dev] [PATCH 19/20] bus/fslmc: add support for secondary processes X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Shreyansh Jain Previously FSLMC bus only supported blacklisting of DPNI (eth), DPSECI (crypto) devices. With this patch, devices like DPIO, DPMCP, and other DP* can also be blacklisted/whitelisted. This is a required condition for secondary processes where the secondary needs to be passed a mutually exclusive list of resources as compared the primary and all other secondaries. This patch also moves the DPIO memory from malloc to hugepage so that in future in case the DPIO list can be shared, it can be accessed in secondaries. Once this patch is done, multi-process cases can be executed by whitelisting/blacklisting devices in each instance. Signed-off-by: Shreyansh Jain --- drivers/bus/fslmc/fslmc_vfio.c | 51 ++++++++++++++++++++---- drivers/bus/fslmc/portal/dpaa2_hw_dpio.c | 24 +++++++++-- 2 files changed, 63 insertions(+), 12 deletions(-) diff --git a/drivers/bus/fslmc/fslmc_vfio.c b/drivers/bus/fslmc/fslmc_vfio.c index 98768a46c..1aae56fa9 100644 --- a/drivers/bus/fslmc/fslmc_vfio.c +++ b/drivers/bus/fslmc/fslmc_vfio.c @@ -1,7 +1,7 @@ /* SPDX-License-Identifier: BSD-3-Clause * * Copyright (c) 2015-2016 Freescale Semiconductor, Inc. All rights reserved. - * Copyright 2016 NXP + * Copyright 2016-2018 NXP * */ @@ -610,6 +610,15 @@ fslmc_process_mcp(struct rte_dpaa2_device *dev) /* check the MC version compatibility */ dpmng.regs = (void *)v_addr; + + /* In case of secondary processes, MC version check is no longer + * required. + */ + if (rte_eal_process_type() == RTE_PROC_SECONDARY) { + rte_mcp_ptr_list[0] = (void *)v_addr; + return 0; + } + if (mc_get_version(&dpmng, CMD_PRI_LOW, &mc_ver_info)) { DPAA2_BUS_ERR("Unable to obtain MC version"); ret = -1; @@ -653,6 +662,15 @@ fslmc_vfio_process_group(void) /* Search the MCP as that should be initialized first. */ TAILQ_FOREACH_SAFE(dev, &rte_fslmc_bus.device_list, next, dev_temp) { if (dev->dev_type == DPAA2_MPORTAL) { + if (dev->device.devargs && + dev->device.devargs->policy == RTE_DEV_BLACKLISTED) { + DPAA2_BUS_LOG(DEBUG, "%s Blacklisted, skipping", + dev->device.name); + TAILQ_REMOVE(&rte_fslmc_bus.device_list, + dev, next); + continue; + } + ret = fslmc_process_mcp(dev); if (ret) { DPAA2_BUS_ERR("Unable to map MC Portal"); @@ -677,6 +695,13 @@ fslmc_vfio_process_group(void) } TAILQ_FOREACH_SAFE(dev, &rte_fslmc_bus.device_list, next, dev_temp) { + if (dev->device.devargs && + dev->device.devargs->policy == RTE_DEV_BLACKLISTED) { + DPAA2_BUS_LOG(DEBUG, "%s Blacklisted, skipping", + dev->device.name); + TAILQ_REMOVE(&rte_fslmc_bus.device_list, dev, next); + continue; + } switch (dev->dev_type) { case DPAA2_ETH: case DPAA2_CRYPTO: @@ -689,10 +714,17 @@ fslmc_vfio_process_group(void) } break; case DPAA2_CON: - case DPAA2_IO: case DPAA2_CI: case DPAA2_BPOOL: case DPAA2_MUX: + /* IN case of secondary processes, all control objects + * like dpbp, dpcon, dpci are not initialized/required + * - all of these are assumed to be initialized and made + * available by primary. + */ + if (rte_eal_process_type() == RTE_PROC_SECONDARY) + continue; + /* Call the object creation routine and remove the * device entry from device list */ @@ -703,12 +735,15 @@ fslmc_vfio_process_group(void) return -1; } - /* This device is not required to be in the DPDK - * exposed device list. - */ - TAILQ_REMOVE(&rte_fslmc_bus.device_list, dev, next); - free(dev); - dev = NULL; + break; + case DPAA2_IO: + ret = fslmc_process_iodevices(dev); + if (ret) { + DPAA2_BUS_DEBUG("Dev (%s) init failed", + dev->device.name); + return -1; + } + break; case DPAA2_UNKNOWN: default: diff --git a/drivers/bus/fslmc/portal/dpaa2_hw_dpio.c b/drivers/bus/fslmc/portal/dpaa2_hw_dpio.c index cd28441f3..f377f24ae 100644 --- a/drivers/bus/fslmc/portal/dpaa2_hw_dpio.c +++ b/drivers/bus/fslmc/portal/dpaa2_hw_dpio.c @@ -418,9 +418,8 @@ dpaa2_create_dpio_device(int vdev_fd, goto err; } - dpio_dev->dpio = malloc(sizeof(struct fsl_mc_io)); - memset(dpio_dev->dpio, 0, sizeof(struct fsl_mc_io)); - + dpio_dev->dpio = rte_zmalloc(NULL, sizeof(struct fsl_mc_io), + RTE_CACHE_LINE_SIZE); if (!dpio_dev->dpio) { DPAA2_BUS_ERR("Memory allocation failure"); goto err; @@ -535,9 +534,26 @@ dpaa2_create_dpio_device(int vdev_fd, if (dpio_dev->dpio) { dpio_disable(dpio_dev->dpio, CMD_PRI_LOW, dpio_dev->token); dpio_close(dpio_dev->dpio, CMD_PRI_LOW, dpio_dev->token); - free(dpio_dev->dpio); + rte_free(dpio_dev->dpio); } + rte_free(dpio_dev); + + /* For each element in the list, cleanup */ + TAILQ_FOREACH(dpio_dev, &dpio_dev_list, next) { + if (dpio_dev->dpio) { + dpio_disable(dpio_dev->dpio, CMD_PRI_LOW, + dpio_dev->token); + dpio_close(dpio_dev->dpio, CMD_PRI_LOW, + dpio_dev->token); + rte_free(dpio_dev->dpio); + } + rte_free(dpio_dev); + } + + /* Preventing re-use of the list with old entries */ + TAILQ_INIT(&dpio_dev_list); + return -1; }