From patchwork Fri Jan 11 12:25:08 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shreyansh Jain X-Patchwork-Id: 49718 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 5C7751BC0F; Fri, 11 Jan 2019 13:25:11 +0100 (CET) Received: from EUR03-VE1-obe.outbound.protection.outlook.com (mail-eopbgr50060.outbound.protection.outlook.com [40.107.5.60]) by dpdk.org (Postfix) with ESMTP id 329801BC8C for ; Fri, 11 Jan 2019 13:25:10 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nxp.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=yuWP96M7QRgup3uLJDMWRXTpho9WDLzl5pbt7eDGMnU=; b=sVfgER2mzznXo6mc24XWFS729Towjg1SDGUqcJP89T0ZqaDXw3GCdECSzZAvws5G4Z95+n351oc6P7arObEbfs1Vu0dhCNO14+hhLuzVlEP+J5MJOMYWeI2zT0/7zji8M3KYsvV7ViJ5VhOqwtMTRoeAp31dGxFVI6aWGR/hglY= Received: from VI1PR04MB4688.eurprd04.prod.outlook.com (20.177.56.80) by VI1PR04MB5455.eurprd04.prod.outlook.com (20.178.121.149) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1516.14; Fri, 11 Jan 2019 12:25:08 +0000 Received: from VI1PR04MB4688.eurprd04.prod.outlook.com ([fe80::b1eb:7e7e:7b90:7b4]) by VI1PR04MB4688.eurprd04.prod.outlook.com ([fe80::b1eb:7e7e:7b90:7b4%4]) with mapi id 15.20.1516.016; Fri, 11 Jan 2019 12:25:08 +0000 From: Shreyansh Jain To: "dev@dpdk.org" CC: "ferruh.yigit@intel.com" , Shreyansh Jain Thread-Topic: [PATCH v3 19/19] bus/fslmc: add support for secondary processes Thread-Index: AQHUqaiwSEsjrXOHl0qrh5KmN1iRrA== Date: Fri, 11 Jan 2019 12:25:08 +0000 Message-ID: <20190111122305.7133-20-shreyansh.jain@nxp.com> References: <20190111115712.6482-1-shreyansh.jain@nxp.com> <20190111122305.7133-1-shreyansh.jain@nxp.com> In-Reply-To: <20190111122305.7133-1-shreyansh.jain@nxp.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [14.143.254.130] x-mailer: git-send-email 2.17.1 x-clientproxiedby: TY2PR01CA0017.jpnprd01.prod.outlook.com (2603:1096:404:a::29) To VI1PR04MB4688.eurprd04.prod.outlook.com (2603:10a6:803:71::16) authentication-results: spf=none (sender IP is ) smtp.mailfrom=shreyansh.jain@nxp.com; x-ms-exchange-messagesentrepresentingtype: 1 x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; VI1PR04MB5455; 6:sdXofFlR5iXHfEwz6B767jyJyC8d9Ybi47KThSH9fpkilOXtcr6BC8rN1ctBR3XFVgivyoKJF/Di5HxDNDB7OV0N93gcUF9kjVRVqtZJtrQjmz1XNqqa/0JBf7pCyb7+iCB4bMuKPownszLXGmSHEVKSv7TC1epZ7bI5OExDstEZZo1bYRXjbrdyG7lrQKS3/LQ4el+DHivaQbh8Tm+vN6ej9f04fI9dzbgIecUrETx2X+FcphduTaZcC7p1YexwuHMiDu/iLdoXs931s9blwI7Pv/3G+t6uuuLzADGO5+PIDhaVfnZqoa3VE2VgdVb60qrdFw/MQksXSrq8ntIPDAYKmVhyQFtdaL4986Ll0+4NKFo4N1elNsrHObsZtOkpVWsNQIQRMemWKE2UddjQEtga5Fc0HRE6BD2oCRAhxZKO/R7ZEoVKWiZWcvC+Feo7EdyiarQ3249N3eJYH1drxw==; 5:NJOPqfZXnEriszh+WQXOmyNRD4/+3f1wOu8Odgja+7LTTTmGieDkpCKtPd62oaa9WcbHdkixTIXpSt9ApaQkwRhwGbXD+OKkbrlH5sClxaprr2N/gMyBYdFoPotaYSJA2qCITCpQmBeB6qUmgTPYeCFaGkYJiFVkCH8OW+GEfxIoL7+rYvrs2Ah2Jmq5+f9vhv6HWOxLg6pUwEmjDHEoyQ==; 7:NBu0o2FnZuGE53Bm5wAm8aEs5e4whSTU5rjsmtfT6/6DSaWFW+sYzcw4W3rToppYtv8kIhPSLOuSDU3VGOzMWlv37TdT4+B3KEXWPFuOT73hJuE6VjPitiYxkbxNZECkmvg4bvz/6xyMLrfYw4QwIg== x-ms-office365-filtering-correlation-id: 2e1b958c-1afe-49cb-c59b-08d677bfd314 x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(7168020)(4627221)(201703031133081)(201702281549075)(8990200)(5600109)(711020)(4618075)(2017052603328)(7153060)(7193020); SRVR:VI1PR04MB5455; x-ms-traffictypediagnostic: VI1PR04MB5455: x-microsoft-antispam-prvs: x-forefront-prvs: 09144DB0F7 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(136003)(366004)(376002)(39860400002)(396003)(346002)(189003)(199004)(66066001)(186003)(86362001)(71200400001)(2351001)(71190400001)(55236004)(386003)(36756003)(6506007)(26005)(99286004)(105586002)(6116002)(14444005)(106356001)(102836004)(2906002)(54906003)(5660300001)(3846002)(256004)(8936002)(97736004)(78486014)(50226002)(2501003)(6486002)(305945005)(6436002)(14454004)(7736002)(8676002)(316002)(1730700003)(81166006)(81156014)(5640700003)(4326008)(2616005)(446003)(6916009)(478600001)(1076003)(52116002)(6512007)(44832011)(486006)(76176011)(53936002)(11346002)(68736007)(476003)(25786009); DIR:OUT; SFP:1101; SCL:1; SRVR:VI1PR04MB5455; H:VI1PR04MB4688.eurprd04.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; received-spf: None (protection.outlook.com: nxp.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: eE4LZOEiPjmfZlWLoLkO15LE01f5NAgwhpUEsfWHWkwTuhPsKxVEyqxqAdf39LtWSdBA7HEteEkafQ4UxKY7vJz41sGOBFGOWZfmUBjU/Io722jG9mxEr8f9MOeg7brYpsV2iFD184oIFvtBxVk6L3F4wWqivWQ+HdcSnReQTlIYAJxT15E5+48S/tP1Ji5iaRv+IU2TYbUAqb9diX9yMEyoP2+dor+TaZJH9ROUmtO5EN007qfOoEg/+0oP9vxyG5OBMtGnJhkqv5F3Ed84NSZgRmeH4dOiDZSFfO0LgS47HC0val2wS8XYxS9EPoAWUrV3QIiB2ZFIvkFyS23qdKdR2pPnqULitWxGBdfZmdO+FPr6juo6DVe+LYldBnEnLmRsfXa0EzmKBPhDnDqUhwuYVeBvghHSIAH+rIGxGX8= spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM MIME-Version: 1.0 X-OriginatorOrg: nxp.com X-MS-Exchange-CrossTenant-Network-Message-Id: 2e1b958c-1afe-49cb-c59b-08d677bfd314 X-MS-Exchange-CrossTenant-originalarrivaltime: 11 Jan 2019 12:25:05.9720 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 686ea1d3-bc2b-4c6f-a92c-d99c5c301635 X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB5455 Subject: [dpdk-dev] [PATCH v3 19/19] bus/fslmc: add support for secondary processes X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Previously FSLMC bus only supported blacklisting of DPNI (eth), DPSECI (crypto) devices. With this patch, devices like DPIO, DPMCP, and other DP* can also be blacklisted/whitelisted. This is a required condition for secondary processes where the secondary needs to be passed a mutually exclusive list of resources as compared the primary and all other secondaries. This patch also moves the DPIO memory from malloc to hugepage so that in future in case the DPIO list can be shared, it can be accessed in secondaries. Once this patch is done, multi-process cases can be executed by whitelisting/blacklisting devices in each instance. Signed-off-by: Shreyansh Jain --- drivers/bus/fslmc/fslmc_vfio.c | 51 ++++++++++++++++++++---- drivers/bus/fslmc/portal/dpaa2_hw_dpio.c | 24 +++++++++-- 2 files changed, 63 insertions(+), 12 deletions(-) diff --git a/drivers/bus/fslmc/fslmc_vfio.c b/drivers/bus/fslmc/fslmc_vfio.c index 98768a46c..1aae56fa9 100644 --- a/drivers/bus/fslmc/fslmc_vfio.c +++ b/drivers/bus/fslmc/fslmc_vfio.c @@ -1,7 +1,7 @@ /* SPDX-License-Identifier: BSD-3-Clause * * Copyright (c) 2015-2016 Freescale Semiconductor, Inc. All rights reserved. - * Copyright 2016 NXP + * Copyright 2016-2018 NXP * */ @@ -610,6 +610,15 @@ fslmc_process_mcp(struct rte_dpaa2_device *dev) /* check the MC version compatibility */ dpmng.regs = (void *)v_addr; + + /* In case of secondary processes, MC version check is no longer + * required. + */ + if (rte_eal_process_type() == RTE_PROC_SECONDARY) { + rte_mcp_ptr_list[0] = (void *)v_addr; + return 0; + } + if (mc_get_version(&dpmng, CMD_PRI_LOW, &mc_ver_info)) { DPAA2_BUS_ERR("Unable to obtain MC version"); ret = -1; @@ -653,6 +662,15 @@ fslmc_vfio_process_group(void) /* Search the MCP as that should be initialized first. */ TAILQ_FOREACH_SAFE(dev, &rte_fslmc_bus.device_list, next, dev_temp) { if (dev->dev_type == DPAA2_MPORTAL) { + if (dev->device.devargs && + dev->device.devargs->policy == RTE_DEV_BLACKLISTED) { + DPAA2_BUS_LOG(DEBUG, "%s Blacklisted, skipping", + dev->device.name); + TAILQ_REMOVE(&rte_fslmc_bus.device_list, + dev, next); + continue; + } + ret = fslmc_process_mcp(dev); if (ret) { DPAA2_BUS_ERR("Unable to map MC Portal"); @@ -677,6 +695,13 @@ fslmc_vfio_process_group(void) } TAILQ_FOREACH_SAFE(dev, &rte_fslmc_bus.device_list, next, dev_temp) { + if (dev->device.devargs && + dev->device.devargs->policy == RTE_DEV_BLACKLISTED) { + DPAA2_BUS_LOG(DEBUG, "%s Blacklisted, skipping", + dev->device.name); + TAILQ_REMOVE(&rte_fslmc_bus.device_list, dev, next); + continue; + } switch (dev->dev_type) { case DPAA2_ETH: case DPAA2_CRYPTO: @@ -689,10 +714,17 @@ fslmc_vfio_process_group(void) } break; case DPAA2_CON: - case DPAA2_IO: case DPAA2_CI: case DPAA2_BPOOL: case DPAA2_MUX: + /* IN case of secondary processes, all control objects + * like dpbp, dpcon, dpci are not initialized/required + * - all of these are assumed to be initialized and made + * available by primary. + */ + if (rte_eal_process_type() == RTE_PROC_SECONDARY) + continue; + /* Call the object creation routine and remove the * device entry from device list */ @@ -703,12 +735,15 @@ fslmc_vfio_process_group(void) return -1; } - /* This device is not required to be in the DPDK - * exposed device list. - */ - TAILQ_REMOVE(&rte_fslmc_bus.device_list, dev, next); - free(dev); - dev = NULL; + break; + case DPAA2_IO: + ret = fslmc_process_iodevices(dev); + if (ret) { + DPAA2_BUS_DEBUG("Dev (%s) init failed", + dev->device.name); + return -1; + } + break; case DPAA2_UNKNOWN: default: diff --git a/drivers/bus/fslmc/portal/dpaa2_hw_dpio.c b/drivers/bus/fslmc/portal/dpaa2_hw_dpio.c index cd28441f3..f377f24ae 100644 --- a/drivers/bus/fslmc/portal/dpaa2_hw_dpio.c +++ b/drivers/bus/fslmc/portal/dpaa2_hw_dpio.c @@ -418,9 +418,8 @@ dpaa2_create_dpio_device(int vdev_fd, goto err; } - dpio_dev->dpio = malloc(sizeof(struct fsl_mc_io)); - memset(dpio_dev->dpio, 0, sizeof(struct fsl_mc_io)); - + dpio_dev->dpio = rte_zmalloc(NULL, sizeof(struct fsl_mc_io), + RTE_CACHE_LINE_SIZE); if (!dpio_dev->dpio) { DPAA2_BUS_ERR("Memory allocation failure"); goto err; @@ -535,9 +534,26 @@ dpaa2_create_dpio_device(int vdev_fd, if (dpio_dev->dpio) { dpio_disable(dpio_dev->dpio, CMD_PRI_LOW, dpio_dev->token); dpio_close(dpio_dev->dpio, CMD_PRI_LOW, dpio_dev->token); - free(dpio_dev->dpio); + rte_free(dpio_dev->dpio); } + rte_free(dpio_dev); + + /* For each element in the list, cleanup */ + TAILQ_FOREACH(dpio_dev, &dpio_dev_list, next) { + if (dpio_dev->dpio) { + dpio_disable(dpio_dev->dpio, CMD_PRI_LOW, + dpio_dev->token); + dpio_close(dpio_dev->dpio, CMD_PRI_LOW, + dpio_dev->token); + rte_free(dpio_dev->dpio); + } + rte_free(dpio_dev); + } + + /* Preventing re-use of the list with old entries */ + TAILQ_INIT(&dpio_dev_list); + return -1; }