From patchwork Fri Aug 18 12:51:27 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shreyansh Jain X-Patchwork-Id: 27646 Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [IPv6:::1]) by dpdk.org (Postfix) with ESMTP id 1C6B09155; Fri, 18 Aug 2017 14:42:00 +0200 (CEST) Received: from NAM01-BY2-obe.outbound.protection.outlook.com (mail-by2nam01on0052.outbound.protection.outlook.com [104.47.34.52]) by dpdk.org (Postfix) with ESMTP id 5D78D914B for ; Fri, 18 Aug 2017 14:41:58 +0200 (CEST) Received: from CY1PR03CA0037.namprd03.prod.outlook.com (10.174.128.47) by CO2PR03MB2261.namprd03.prod.outlook.com (10.166.92.142) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1362.18; Fri, 18 Aug 2017 12:41:56 +0000 Received: from BL2FFO11FD015.protection.gbl (2a01:111:f400:7c09::102) by CY1PR03CA0037.outlook.office365.com (2603:10b6:600::47) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1362.18 via Frontend Transport; Fri, 18 Aug 2017 12:41:56 +0000 Authentication-Results: spf=fail (sender IP is 192.88.168.50) smtp.mailfrom=nxp.com; nxp.com; dkim=none (message not signed) header.d=none;nxp.com; dmarc=fail action=none header.from=nxp.com; Received-SPF: Fail (protection.outlook.com: domain of nxp.com does not designate 192.88.168.50 as permitted sender) receiver=protection.outlook.com; client-ip=192.88.168.50; helo=tx30smr01.am.freescale.net; Received: from tx30smr01.am.freescale.net (192.88.168.50) by BL2FFO11FD015.mail.protection.outlook.com (10.173.160.223) with Microsoft SMTP Server (version=TLS1_0, cipher=TLS_RSA_WITH_AES_256_CBC_SHA) id 15.1.1341.15 via Frontend Transport; Fri, 18 Aug 2017 12:41:55 +0000 Received: from Tophie.ap.freescale.net ([10.232.14.39]) by tx30smr01.am.freescale.net (8.14.3/8.14.0) with ESMTP id v7ICfeCM028231; Fri, 18 Aug 2017 05:41:53 -0700 From: Shreyansh Jain To: CC: , , Shreyansh Jain Date: Fri, 18 Aug 2017 18:21:27 +0530 Message-ID: <20170818125127.22346-7-shreyansh.jain@nxp.com> X-Mailer: git-send-email 2.9.3 In-Reply-To: <20170818125127.22346-1-shreyansh.jain@nxp.com> References: <20170818125127.22346-1-shreyansh.jain@nxp.com> X-EOPAttributedMessage: 0 X-Matching-Connectors: 131475337163796815; (91ab9b29-cfa4-454e-5278-08d120cd25b8); () X-Forefront-Antispam-Report: CIP:192.88.168.50; IPV:NLI; CTRY:US; EFV:NLI; SFV:NSPM; SFS:(10009020)(6009001)(336005)(39860400002)(39380400002)(2980300002)(1109001)(1110001)(339900001)(199003)(189002)(2906002)(47776003)(48376002)(50466002)(86362001)(33646002)(105606002)(76176999)(2351001)(50986999)(106466001)(498600001)(626005)(4326008)(8656003)(97736004)(575784001)(6916009)(2950100002)(6666003)(104016004)(356003)(5003940100001)(50226002)(68736007)(189998001)(36756003)(5660300001)(1076002)(81156014)(81166006)(54906002)(77096006)(8936002)(85426001)(53936002)(8676002)(305945005)(110136004); DIR:OUT; SFP:1101; SCL:1; SRVR:CO2PR03MB2261; H:tx30smr01.am.freescale.net; FPR:; SPF:Fail; PTR:InfoDomainNonexistent; A:1; MX:1; LANG:en; X-Microsoft-Exchange-Diagnostics: 1; BL2FFO11FD015; 1:NN38mg3judzD3JBGWgxU7KObS/B5WNzSrSSDfYZV/ZoJ9Nbsr4QG71OUqeJVP+ZDEuGgQ4uW2ZoI8qtLpgRPiXNDBNjCv7nO58InQdu171g3bIiBFmwHfuhGlyTZp9BZ MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: ed231d67-46d7-4720-0fed-08d4e63682d4 X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(300000500095)(300135000095)(300000501095)(300135300095)(22001)(300000502095)(300135100095)(300000503095)(300135400095)(2017052603031)(201703131430075)(201703131517081)(300000504095)(300135200095)(300000505095)(300135600095)(300000506095)(300135500095); SRVR:CO2PR03MB2261; X-Microsoft-Exchange-Diagnostics: 1; CO2PR03MB2261; 3:DZ2Pd/oDP1OPZPPW7CG4ABfiaqkRFQnexhIhFUryZlSfHDIzBfRwlH/wO6/nqQIfaAM13E2jpyu8E6uhVnyO0g70QwUPmv7begHJ26mcSD7q0WJW+/1Yad5KsjSVUMHEDnK19Wr46ySZD4A8L0l34qLPrx//SRWm3m+P/JIWAPczt0zgyaGviZ6huvH+lVy4bW7fhLNnIS9AYCFp/hCDNEM4aHb1qhq8e9Y1WlG4Ochw5QfWGwdm8u5oBqoFkOUbVmGyRYKuA30uQH1znZURg1QK5kGF5jxekLlBvQlU7htgmchQNYB4KJf2yJ2L7eZj/elG7eu0WST4gpY+aQEzwAAtVCYlcK6TLuMyEJT/hAk=; 25:SpP3B+2mRSOk9q9g0ikLy7mpx/DtaGiXwdgOd2asShSmhNWNuBSsClfxnJPNVkIki2Z7e3sI9GVyGNpjvjHkix1ptmLjrjyr5WFx59PoRJ+WJd+0ZGdba7HVprk3EQL0njCWETZx5UizjavkXCZaxfhbMBZamDjjiLQuSmx/PA2ltPowAyVAfLtIQnPTEk4bpEUXSv9u/tOu48FV44pXlgmH09T8MJWxVSurpKJM+KR8DE+o7zlAc6Y0aYCp04u9SX7ovU9KKKNN/IZLL6m2RqC3l4sPlOcqpGzgV4cznIUbEK5bA7ydbP6ls7Rcz/fwSgmFGhD4gYFD7vMnpr/nZA== X-MS-TrafficTypeDiagnostic: CO2PR03MB2261: X-Microsoft-Exchange-Diagnostics: 1; CO2PR03MB2261; 31:CvBQPNhuMsDwU2mwe1YP3exCnW+FKp+LwnOBgVA2eLTTOXUVXwrZBeqlNbSDmzxen/fJYbRkUXQreZrLNRzRLd5heMmdsoOQB3hWsxX/xOt2mxs0hPlLPofNAaKviNDKqTCmD30z8VAhgmsG6MXoA/kdJvBmgGVOqjIowxPWdg4ktUsBBTYvrHGZxC4MbEiByGymHsxyok0BuhJ1rTtxUzBTMclDVWwMEGinbLYAUSo=; 4:rAeM4tp1EySmUapF1kcqVhZOoHKQfxHxErHPpZyS9/FcY3KowEPWzFWKMN6O5azoIixJKR60+ximOXTQh0YOZSwj7HN2B9O4ouf7vzX6Ce9LtCknckJeuH6id3ao4Be7Ul8RYPdqpu/GC9fn0xrxNF/Budcn4YPel13cw0HaCip0gi/icXaEhEKMsqxPLhux0UDUn24hJBGC0jhoX8UBZQQu7kehzsDld1o8lBSaahyHB+Byc3CgRYHfCwJV5NCsdAbgEsWoB9Aa5w9fbTtrmD1r9VocTwn3JjEDp6yDVSlAyX112/4QPdjIYehyMMLrSZ/UFjeeOfinI6rcHf0egnt8OCEnhtaNF+d/yqxJBBJRVwXsl9mrA5AW0NB/g0l6 X-Exchange-Antispam-Report-Test: UriScan:(185117386973197)(275809806118684)(21532816269658); X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6095135)(601004)(2401047)(8121501046)(5005006)(13016025)(13018025)(3002001)(93006095)(93001095)(100000703101)(100105400095)(10201501046)(6055026)(6096035)(20161123565025)(201703131430075)(201703131433075)(201703131448075)(201703161259150)(201703151042153)(20161123563025)(20161123561025)(20161123556025)(20161123559100)(201708071742011)(100000704101)(100105200095)(100000705101)(100105500095); SRVR:CO2PR03MB2261; BCL:0; PCL:0; RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(400006)(100000804101)(100110200095)(100000805101)(100110500095); SRVR:CO2PR03MB2261; X-Forefront-PRVS: 040359335D X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; CO2PR03MB2261; 23:jeTjg5Cm4gqC6MjEuoFzzk5/PGxqQiHeqVhJtuWFm?= aDiK65zd5qsdsBvjFgz6DkMngO8Xj5VoOLT4Dd+9y/yFC5nFD0UQ0awBJE3MZCp8jAgF2XV1bDAsCv5ojhRWjAy43yzoOQ1h9+KbHCAN+u7qjvC7cCSpTSDJEfQLLlKFMxdf5yvZl8XbsM7v8B0tw02LpUIcVXiWAkvBp77hU0pVt/k9oJNAyCeymUMnD3l54e5/6CqDmx88DHmVEckpLsu3SSn6DFbKVpKOSV1YwppRrXnjLysmyPQkxXYIYJMq84Ap55+R/q+QsI53U4mzecBTLV/WsfL521SqRqJtZsVAt+pNqoZ8UIeCKRiwmrz3uOp8w+MN7NFr5lfBEblLPQPNIImi5i8Ao1lGeT1qG2cW/7YwXYS1w5hnJJOpO4/0MS7ziIch0KvSgCBj4aqnMoLIVrFg+hI0fW4WHWn7rLKBDFekvq9d3u2/5B12pupjTllsmOEJCX/+GnksrEBOeuFmAGiiGLn9kc6YNbYI402k8/fttOxHrNcrTR1isl2MqAbN0KTv7pP7qxNNbpEYJnkov64ggxC8oKrVO0O/sphHSulkDDTfFqIg4pqnIheQDfqXy65NfbdcIJd/OtmXRn+k7NoBFdccsFy5IBF4SO5zphS3hzJswB5QE8AaXvEC3fYzRBq99IIHHHqgjD7skAyelf6b6VWAHB64JsVVfsu2698bt6AARm2ZpYoU2jc+iYFy95PlOd26wjFic9Ws4x3fxRtZDOFIigslj5eG7hrvBBNifXWiCJMie1RlIutAa94ZZHcuJoXrgeMHa8FvCUS7m0SGOPLupp2LwDPJP9anxVdkylJiO7cLURYFsBjwEWwm0Xoyyc2h2w7ri8Bu7mvsIymddTKKvzKhY+gBIf/w92oUOKx/xNmROaXbZNmQDRp7kMbC76FRGUxCFaF3JATOOM0SuLfdOfn8kPoqrLT25KKQcA9EbwYG3vmW41ZOYGhzQ4Z2UhrHmgSAiGppHnrzxxAC4XjwNFlBgNycaAJ3A5S183OQIK160gm6ylufAHirNLYJuANxHrGCPa8UuMQnj86ilxwKvpNYuiM9q+mcM9J4x9wJbfChBvWvO4XESHOcSBKVa3S4xMdEgpBof7+hksZaPs+T425RtwP5QUHUMnfwHahXdtpuDoKO3x7qKo= X-Microsoft-Exchange-Diagnostics: 1; CO2PR03MB2261; 6:hISI0e6Om7q5XktQA3msh/2E6rklliOPkbaRq33d9m0IYXuxfqd2f5gVXdAaxsorqmebPT4pH3LUujrqddKYLpR4tF/+QO16p97PnnQRe73/Xba4GyB/xsmwNrhwUXa+O2bDMCOlAFhVBXBBvgSKKrAc1nN2BXQLT3A14Rb/75g3aJvP8VGAmgRtsYywlejuj4D6w7onOENcfqP6ZjW75hLM0YL6p3HIhdSzuWz8rDA1bpbGldSDph0zcU7q5Wv7HtN+IykM30BWHCJFP1t8e6FAMoEx4uxjy5g+8P98j+LcrbsopjIpfPLD9xZrUNGGvlbaxUihrdea9Eam9MWILg==; 5:obQsoAZ8QdQsGMb7WlVT0o02HMn8mGOtOCMpp2UC7T1WBPtkOrn1Zi9vm6WII9HyONGYzJroeKN3jOgg86OUxmEag3hCwiDeDX6PAeV61Vj79uHWG3A15926/YT8hmqoXELG//e2VegkSWHbew7iCg==; 24:lzOd4KjOYrLs2G+fu5ye+EjVYwnDYJyO1w6wj/JrmJG7Ts/ytCEz5W542lPNPMQ3PhSs5TGtjNq+x1/rPWn+TttAQkk4CNms9Exb7VFDn/k=; 7:YnDxj2FYgW69EI3t8HghbTstIbAaPljoHuNAV6GvYfOqWU6UXmxUQkUWPrcP44kr+XpZZR/cZ7lcNBLaKGV2deLFW5QpSYwf71/r1bqUz/yzC/+GAdxIkIeqxdP1XoqoSWZVYQc7sfV3j2ahmSaZ0N2HWMCsiASJnpd4BWkbauYylGgfRHZ1a1KLn3atOtKRXeGp50+sFt3sEpQbgdrvxn0Fj39t3HfN/IXtnfDD3ew= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-MS-Exchange-CrossTenant-OriginalArrivalTime: 18 Aug 2017 12:41:55.9896 (UTC) X-MS-Exchange-CrossTenant-Id: 5afe0b00-7697-4969-b663-5eab37d5f47e X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=5afe0b00-7697-4969-b663-5eab37d5f47e; Ip=[192.88.168.50]; Helo=[tx30smr01.am.freescale.net] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO2PR03MB2261 Subject: [dpdk-dev] [PATCH 6/6] bus/fslmc: refactor scan and probe functions X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Scan now searches for devices through sysfs interface and adds them to a list for later initialization. During probe, each device is initialized accroding to its property. Signed-off-by: Shreyansh Jain --- drivers/bus/fslmc/fslmc_bus.c | 185 ++++++++++++- drivers/bus/fslmc/fslmc_vfio.c | 456 ++++++++++++++----------------- drivers/bus/fslmc/fslmc_vfio.h | 4 +- drivers/bus/fslmc/portal/dpaa2_hw_dpio.c | 1 - drivers/bus/fslmc/rte_fslmc.h | 3 + 5 files changed, 390 insertions(+), 259 deletions(-) diff --git a/drivers/bus/fslmc/fslmc_bus.c b/drivers/bus/fslmc/fslmc_bus.c index f71598d..0feeba0 100644 --- a/drivers/bus/fslmc/fslmc_bus.c +++ b/drivers/bus/fslmc/fslmc_bus.c @@ -42,32 +42,178 @@ #include #include -#include "rte_fslmc.h" -#include "fslmc_vfio.h" +#include +#include #define FSLMC_BUS_LOG(level, fmt, args...) \ RTE_LOG(level, EAL, "%s(): " fmt "\n", __func__, ##args) +#define VFIO_IOMMU_GROUP_PATH "/sys/kernel/iommu_groups" + struct rte_fslmc_bus rte_fslmc_bus; +static void +cleanup_fslmc_device_list(void) +{ + struct rte_dpaa2_device *dev; + struct rte_dpaa2_device *t_dev; + + TAILQ_FOREACH_SAFE(dev, &rte_fslmc_bus.device_list, next, t_dev) { + TAILQ_REMOVE(&rte_fslmc_bus.device_list, dev, next); + free(dev); + dev = NULL; + } +} + +static int +scan_one_fslmc_device(char *dev_name) +{ + char *dup_dev_name, *t_ptr; + struct rte_dpaa2_device *dev; + + if (!dev_name) + return -1; + + /* Ignore the Container name itself */ + if (!strncmp("dprc", dev_name, 4)) + return 0; + + /* Creating a temporary copy to perform cut-parse over string */ + dup_dev_name = strdup(dev_name); + if (!dup_dev_name) { + FSLMC_BUS_LOG(ERR, "Out of memory."); + return -ENOMEM; + } + + /* For all other devices, we allocate rte_dpaa2_device. + * For those devices where there is no driver, probe would release + * the memory associated with the rte_dpaa2_device after necessary + * initialization. + */ + dev = calloc(1, sizeof(struct rte_dpaa2_device)); + if (!dev) { + FSLMC_BUS_LOG(ERR, "Out of memory."); + free(dup_dev_name); + return -ENOMEM; + } + + /* Parse the device name and ID */ + t_ptr = strtok(dup_dev_name, "."); + if (!t_ptr) { + FSLMC_BUS_LOG(ERR, "Incorrect device string observed."); + goto cleanup; + } + if (!strncmp("dpni", t_ptr, 4)) + dev->dev_type = DPAA2_ETH; + else if (!strncmp("dpseci", t_ptr, 6)) + dev->dev_type = DPAA2_CRYPTO; + else if (!strncmp("dpcon", t_ptr, 5)) + dev->dev_type = DPAA2_CON; + else if (!strncmp("dpbp", t_ptr, 4)) + dev->dev_type = DPAA2_BPOOL; + else if (!strncmp("dpio", t_ptr, 4)) + dev->dev_type = DPAA2_IO; + else if (!strncmp("dpci", t_ptr, 5)) + dev->dev_type = DPAA2_CI; + else if (!strncmp("dpmcp", t_ptr, 5)) + dev->dev_type = DPAA2_MPORTAL; + else + dev->dev_type = DPAA2_UNKNOWN; + + t_ptr = strtok(NULL, "."); + if (!t_ptr) { + FSLMC_BUS_LOG(ERR, "Incorrect device string observed (%s).", + t_ptr); + goto cleanup; + } + + sscanf(t_ptr, "%hu", &dev->object_id); + dev->device.name = strdup(dev_name); + if (!dev->device.name) { + FSLMC_BUS_LOG(ERR, "Out of memory."); + goto cleanup; + } + + /* Add device in the fslmc device list */ + TAILQ_INSERT_TAIL(&rte_fslmc_bus.device_list, dev, next); + if (dev->dev_type == DPAA2_ETH) + RTE_LOG(INFO, EAL, "%s: Eth Device scanned.\n", + dev->device.name); + if (dev->dev_type == DPAA2_CRYPTO) + RTE_LOG(INFO, EAL, "%s: Crypto Device scanned.\n", + dev->device.name); + RTE_LOG(DEBUG, EAL, "Add device (%s) with object ID: %d, Type = %d\n", + dev->device.name, dev->object_id, dev->dev_type); + + /* Don't need the duplicated device filesystem entry anymore */ + if (dup_dev_name) + free(dup_dev_name); + + return 0; +cleanup: + if (dup_dev_name) + free(dup_dev_name); + if (dev) + free(dev); + return -1; +} + static int rte_fslmc_scan(void) { int ret; + int device_count = 0; + char fslmc_dirpath[PATH_MAX]; + DIR *dir; + struct dirent *entry; + static int process_once; + int groupid; - ret = fslmc_vfio_setup_group(); - if (ret) { - FSLMC_BUS_LOG(ERR, "fslmc: Unable to setup VFIO"); - return ret; + if (process_once) { + FSLMC_BUS_LOG(DEBUG, + "Fslmc bus already scanned. Not rescanning"); + return 0; } + process_once = 1; - ret = fslmc_vfio_process_group(); - if (ret) { - FSLMC_BUS_LOG(ERR, "fslmc: Unable to setup devices"); - return -1; + ret = fslmc_get_container_group(&groupid); + if (ret != 0) + goto scan_fail; + + /* Scan devices on the group */ + sprintf(fslmc_dirpath, "%s/%d/devices", VFIO_IOMMU_GROUP_PATH, + groupid); + dir = opendir(fslmc_dirpath); + if (!dir) { + FSLMC_BUS_LOG(ERR, "Unable to open VFIO group dir."); + goto scan_fail; + } + + while ((entry = readdir(dir)) != NULL) { + if (entry->d_name[0] == '.' || entry->d_type != DT_LNK) + continue; + + ret = scan_one_fslmc_device(entry->d_name); + if (ret != 0) { + /* Error in parsing directory - exit gracefully */ + goto scan_fail_cleanup; + } + device_count += 1; } RTE_LOG(INFO, EAL, "fslmc: Bus scan completed\n"); + + closedir(dir); + return 0; + +scan_fail_cleanup: + closedir(dir); + + /* Remove all devices in the list */ + cleanup_fslmc_device_list(); +scan_fail: + RTE_LOG(DEBUG, EAL, "Error in scanning FSLMC Bus. Skipping.\n"); + /* Irrespective of failure, scan only return success */ return 0; } @@ -88,6 +234,18 @@ rte_fslmc_probe(void) struct rte_dpaa2_device *dev; struct rte_dpaa2_driver *drv; + ret = fslmc_vfio_setup_group(); + if (ret) { + FSLMC_BUS_LOG(ERR, "Unable to setup VFIO"); + goto probe_fail; + } + + ret = fslmc_vfio_process_group(); + if (ret) { + FSLMC_BUS_LOG(ERR, "Unable to setup devices"); + goto probe_fail; + } + TAILQ_FOREACH(dev, &rte_fslmc_bus.device_list, next) { TAILQ_FOREACH(drv, &rte_fslmc_bus.driver_list, next) { ret = rte_fslmc_match(drv, dev); @@ -103,7 +261,12 @@ rte_fslmc_probe(void) break; } } - return ret; + + return 0; + +probe_fail: + RTE_LOG(DEBUG, EAL, "Error in probing FSLMC Bus. Skipping.\n"); + return 0; } static struct rte_device * diff --git a/drivers/bus/fslmc/fslmc_vfio.c b/drivers/bus/fslmc/fslmc_vfio.c index f0d69a9..a27814e 100644 --- a/drivers/bus/fslmc/fslmc_vfio.c +++ b/drivers/bus/fslmc/fslmc_vfio.c @@ -68,13 +68,15 @@ /** Pathname of FSL-MC devices directory. */ #define SYSFS_FSL_MC_DEVICES "/sys/bus/fsl-mc/devices" +#define FSLMC_CONTAINER_MAX_LEN 8 /**< Of the format dprc.XX */ + /* Number of VFIO containers & groups with in */ static struct fslmc_vfio_group vfio_group; static struct fslmc_vfio_container vfio_container; static int container_device_fd; +static char *g_container; static uint32_t *msi_intr_vaddr; void *(*rte_mcp_ptr_list); -static uint32_t mcp_id; static int is_dma_done; static struct rte_dpaa2_object_list dpaa2_obj_list = @@ -89,10 +91,57 @@ rte_fslmc_object_register(struct rte_dpaa2_object *object) TAILQ_INSERT_TAIL(&dpaa2_obj_list, object, next); } -static int vfio_connect_container(void) +int +fslmc_get_container_group(int *groupid) +{ + int ret; + char *container; + + if (!g_container) { + container = getenv("DPRC"); + if (container == NULL) { + RTE_LOG(WARNING, EAL, + "Environment variable DPRC not set.\n"); + return -1; + } + + if (strlen(container) >= FSLMC_CONTAINER_MAX_LEN) { + FSLMC_VFIO_LOG(ERR, "Invalid container name: %s\n", + container); + return -1; + } + + g_container = strdup(container); + if (!g_container) { + FSLMC_VFIO_LOG(ERR, "Out of memory."); + return -ENOMEM; + } + } + + /* get group number */ + ret = vfio_get_group_no(SYSFS_FSL_MC_DEVICES, g_container, groupid); + if (ret <= 0) { + FSLMC_VFIO_LOG(ERR, "Unable to find %s IOMMU group", + g_container); + return -1; + } + + FSLMC_VFIO_LOG(DEBUG, "Container: %s has VFIO iommu group id = %d", + g_container, *groupid); + + return 0; +} + +static int +vfio_connect_container(void) { int fd, ret; + if (vfio_container.used) { + FSLMC_VFIO_LOG(DEBUG, "No container available."); + return -1; + } + /* Try connecting to vfio container if already created */ if (!ioctl(vfio_group.fd, VFIO_GROUP_SET_CONTAINER, &vfio_container.fd)) { @@ -284,48 +333,6 @@ static int64_t vfio_map_mcp_obj(struct fslmc_vfio_group *group, char *mcp_obj) return v_addr; } -static inline int -dpaa2_compare_dpaa2_dev(const struct rte_dpaa2_device *dev, - const struct rte_dpaa2_device *dev2) -{ - /*not the same family device */ - if (dev->dev_type != DPAA2_ETH || - dev->dev_type != DPAA2_CRYPTO) - return -1; - - if (dev->object_id == dev2->object_id) - return 0; - else - return 1; -} - -static void -fslmc_bus_add_device(struct rte_dpaa2_device *dev) -{ - struct rte_fslmc_device_list *dev_l; - - dev_l = &rte_fslmc_bus.device_list; - - /* device is valid, add in list (sorted) */ - if (TAILQ_EMPTY(dev_l)) { - TAILQ_INSERT_TAIL(dev_l, dev, next); - } else { - struct rte_dpaa2_device *dev2; - int ret; - - TAILQ_FOREACH(dev2, dev_l, next) { - ret = dpaa2_compare_dpaa2_dev(dev, dev2); - if (ret <= 0) - continue; - - TAILQ_INSERT_BEFORE(dev2, dev, next); - return; - } - - TAILQ_INSERT_TAIL(dev_l, dev, next); - } -} - #define IRQ_SET_BUF_LEN (sizeof(struct vfio_irq_set) + sizeof(int)) int rte_dpaa2_intr_enable(struct rte_intr_handle *intr_handle, @@ -360,283 +367,242 @@ int rte_dpaa2_intr_enable(struct rte_intr_handle *intr_handle, return 0; } -/* Following function shall fetch total available list of MC devices - * from VFIO container & populate private list of devices and other - * data structures +/* + * fslmc_process_iodevices for processing only IO (ETH, CRYPTO, and possibly + * EVENT) devices. */ -int fslmc_vfio_process_group(void) +static int +fslmc_process_iodevices(struct rte_dpaa2_device *dev) { - struct fslmc_vfio_device *vdev; + int dev_fd; struct vfio_device_info device_info = { .argsz = sizeof(device_info) }; - char *temp_obj, *object_type, *mcp_obj, *dev_name; - int32_t object_id, i, dev_fd, ret; - DIR *d; - struct dirent *dir; - char path[PATH_MAX]; - int64_t v_addr; - int ndev_count; - struct fslmc_vfio_group *group = &vfio_group; - static int process_once; + struct rte_dpaa2_object *object = NULL; - /* if already done once */ - if (process_once) { - FSLMC_VFIO_LOG(DEBUG, - "Already scanned once - re-scan not supported"); - return 0; + dev_fd = ioctl(vfio_group.fd, VFIO_GROUP_GET_DEVICE_FD, + dev->device.name); + if (dev_fd <= 0) { + FSLMC_VFIO_LOG(ERR, "Unable to obtain device FD for device:%s", + dev->device.name); + return -1; } - process_once = 0; - - sprintf(path, "/sys/kernel/iommu_groups/%d/devices", group->groupid); - d = opendir(path); - if (!d) { - FSLMC_VFIO_LOG(ERR, "Unable to open directory %s", path); + if (ioctl(dev_fd, VFIO_DEVICE_GET_INFO, &device_info)) { + FSLMC_VFIO_LOG(ERR, "DPAA2 VFIO_DEVICE_GET_INFO fail"); return -1; } - /*Counting the number of devices in a group and getting the mcp ID*/ - ndev_count = 0; - mcp_obj = NULL; - while ((dir = readdir(d)) != NULL) { - if (dir->d_type == DT_LNK) { - ndev_count++; - if (!strncmp("dpmcp", dir->d_name, 5)) { - if (mcp_obj) - free(mcp_obj); - mcp_obj = malloc(sizeof(dir->d_name)); - if (!mcp_obj) { - FSLMC_VFIO_LOG(ERR, - "mcp obj:alloc failed"); - closedir(d); - return -ENOMEM; - } - strcpy(mcp_obj, dir->d_name); - temp_obj = strtok(dir->d_name, "."); - temp_obj = strtok(NULL, "."); - sscanf(temp_obj, "%d", &mcp_id); - } + switch (dev->dev_type) { + case DPAA2_CON: + case DPAA2_IO: + case DPAA2_CI: + case DPAA2_BPOOL: + TAILQ_FOREACH(object, &dpaa2_obj_list, next) { + if (dev->dev_type == object->dev_type) + object->create(dev_fd, &device_info, + dev->object_id); + else + continue; } + break; + default: + break; } - closedir(d); - d = NULL; - if (!mcp_obj) { - FSLMC_VFIO_LOG(ERR, "DPAA2 MCP Object not Found"); - return -ENODEV; - } - RTE_LOG(INFO, EAL, "fslmc: DPRC contains = %d devices\n", ndev_count); - - /* Allocate the memory depends upon number of objects in a group*/ - group->vfio_device = (struct fslmc_vfio_device *)malloc(ndev_count * - sizeof(struct fslmc_vfio_device)); - if (!(group->vfio_device)) { - FSLMC_VFIO_LOG(ERR, "vfio device: Unable to allocate memory\n"); - free(mcp_obj); - return -ENOMEM; - } - /* Allocate memory for MC Portal list */ + FSLMC_VFIO_LOG(DEBUG, "Device (%s) abstracted from VFIO", + dev->device.name); + return 0; +} + +static int +fslmc_process_mcp(struct rte_dpaa2_device *dev) +{ + int64_t v_addr; + char *dev_name; + rte_mcp_ptr_list = malloc(sizeof(void *) * 1); if (!rte_mcp_ptr_list) { - FSLMC_VFIO_LOG(ERR, "portal list: Unable to allocate memory!"); - free(mcp_obj); - goto FAILURE; + FSLMC_VFIO_LOG(ERR, "Out of memory"); + return -ENOMEM; } - v_addr = vfio_map_mcp_obj(group, mcp_obj); - free(mcp_obj); - if (v_addr == (int64_t)MAP_FAILED) { - FSLMC_VFIO_LOG(ERR, "Error mapping region (errno = %d)", errno); - goto FAILURE; + dev_name = strdup(dev->device.name); + if (!dev_name) { + FSLMC_VFIO_LOG(ERR, "Out of memory."); + free(rte_mcp_ptr_list); + rte_mcp_ptr_list = NULL; + return -ENOMEM; } - rte_mcp_ptr_list[0] = (void *)v_addr; - - d = opendir(path); - if (!d) { - FSLMC_VFIO_LOG(ERR, "Unable to open %s Directory", path); - goto FAILURE; + v_addr = vfio_map_mcp_obj(&vfio_group, dev_name); + if (v_addr == (int64_t)MAP_FAILED) { + FSLMC_VFIO_LOG(ERR, "Error mapping region (errno = %d)", + errno); + free(rte_mcp_ptr_list); + rte_mcp_ptr_list = NULL; + return -1; } - i = 0; - /* Parsing each object and initiating them*/ - while ((dir = readdir(d)) != NULL) { - if (dir->d_type != DT_LNK) - continue; - if (!strncmp("dprc", dir->d_name, 4) || - !strncmp("dpmcp", dir->d_name, 5)) - continue; - dev_name = malloc(sizeof(dir->d_name)); - if (!dev_name) { - FSLMC_VFIO_LOG(ERR, "name: Unable to allocate memory"); - goto FAILURE; - } - strcpy(dev_name, dir->d_name); - object_type = strtok(dir->d_name, "."); - temp_obj = strtok(NULL, "."); - sscanf(temp_obj, "%d", &object_id); - - /* getting the device fd*/ - dev_fd = ioctl(group->fd, VFIO_GROUP_GET_DEVICE_FD, dev_name); - if (dev_fd < 0) { - FSLMC_VFIO_LOG(ERR, - "GET_DEVICE_FD error fd: %s, Group: %d", - dev_name, group->fd); - free(dev_name); - goto FAILURE; - } + rte_mcp_ptr_list[0] = (void *)v_addr; - free(dev_name); - vdev = &group->vfio_device[group->object_index++]; - vdev->fd = dev_fd; - vdev->index = i; - i++; - /* Get Device inofrmation */ - if (ioctl(vdev->fd, VFIO_DEVICE_GET_INFO, &device_info)) { - FSLMC_VFIO_LOG(ERR, "DPAA2 VFIO_DEVICE_GET_INFO fail"); - goto FAILURE; - } - if (!strcmp(object_type, "dpni") || - !strcmp(object_type, "dpseci")) { - struct rte_dpaa2_device *dev; + return 0; +} - dev = malloc(sizeof(struct rte_dpaa2_device)); - if (dev == NULL) +int +fslmc_vfio_process_group(void) +{ + int ret; + int found_mportal = 0; + struct rte_dpaa2_device *dev, *dev_temp; + + /* Search the MCP as that should be initialized first. */ + TAILQ_FOREACH_SAFE(dev, &rte_fslmc_bus.device_list, next, dev_temp) { + if (dev->dev_type == DPAA2_MPORTAL) { + ret = fslmc_process_mcp(dev); + if (ret) { + FSLMC_VFIO_LOG(DEBUG, "Unable to map Portal."); return -1; - - memset(dev, 0, sizeof(*dev)); - /* store hw_id of dpni/dpseci device */ - dev->object_id = object_id; - dev->dev_type = (strcmp(object_type, "dpseci")) ? - DPAA2_ETH : DPAA2_CRYPTO; - - sprintf(dev->name, "%s.%d", object_type, object_id); - dev->device.name = dev->name; - - fslmc_bus_add_device(dev); - FSLMC_VFIO_LOG(DEBUG, "DPAA2: Added %s", dev->name); - } else { - /* Parse all other objects */ - struct rte_dpaa2_object *object; - - TAILQ_FOREACH(object, &dpaa2_obj_list, next) { - if (!strcmp(object_type, object->name)) - object->create(dev_fd, &device_info, - object_id); - else - continue; } + if (!found_mportal) + found_mportal = 1; + + TAILQ_REMOVE(&rte_fslmc_bus.device_list, dev, next); + free(dev); + dev = NULL; + /* Ideally there is only a single dpmcp, but in case + * multiple exists, looping on remaining devices. + */ } } - closedir(d); - ret = dpaa2_affine_qbman_swp(); - if (ret) - FSLMC_VFIO_LOG(DEBUG, "Error in affining qbman swp %d", ret); + /* Cannot continue if there is not even a single mportal */ + if (!found_mportal) { + FSLMC_VFIO_LOG(DEBUG, + "No MC Portal device found. Not continuing."); + return -1; + } + + TAILQ_FOREACH_SAFE(dev, &rte_fslmc_bus.device_list, next, dev_temp) { + if (!dev) + break; - return 0; + switch (dev->dev_type) { + case DPAA2_ETH: + case DPAA2_CRYPTO: + ret = fslmc_process_iodevices(dev); + if (ret) { + FSLMC_VFIO_LOG(DEBUG, + "Dev (%s) init failed.", + dev->device.name); + return ret; + } + break; + case DPAA2_CON: + case DPAA2_IO: + case DPAA2_CI: + case DPAA2_BPOOL: + /* Call the object creation routine and remove the + * device entry from device list + */ + ret = fslmc_process_iodevices(dev); + if (ret) { + FSLMC_VFIO_LOG(DEBUG, + "Dev (%s) init failed.", + dev->device.name); + return -1; + } -FAILURE: - if (d) - closedir(d); - if (rte_mcp_ptr_list) { - free(rte_mcp_ptr_list); - rte_mcp_ptr_list = NULL; + /* This device is not required to be in the DPDK + * exposed device list. + */ + TAILQ_REMOVE(&rte_fslmc_bus.device_list, dev, next); + free(dev); + dev = NULL; + break; + case DPAA2_UNKNOWN: + default: + /* Unknown - ignore */ + FSLMC_VFIO_LOG(DEBUG, "Found unknown device (%s).", + dev->device.name); + TAILQ_REMOVE(&rte_fslmc_bus.device_list, dev, next); + free(dev); + dev = NULL; + } } - free(group->vfio_device); - group->vfio_device = NULL; - return -1; + return 0; } -int fslmc_vfio_setup_group(void) +int +fslmc_vfio_setup_group(void) { - struct fslmc_vfio_group *group = NULL; int groupid; int ret; - char *container; struct vfio_group_status status = { .argsz = sizeof(status) }; /* if already done once */ if (container_device_fd) return 0; - container = getenv("DPRC"); - - if (container == NULL) { - FSLMC_VFIO_LOG(ERR, "VFIO container not set in env DPRC"); - return -EOPNOTSUPP; - } - - /* get group number */ - ret = vfio_get_group_no(SYSFS_FSL_MC_DEVICES, container, &groupid); - if (ret == 0) { - RTE_LOG(WARNING, EAL, "%s not managed by VFIO, skipping\n", - container); - return -EOPNOTSUPP; - } - - /* if negative, something failed */ - if (ret < 0) + ret = fslmc_get_container_group(&groupid); + if (ret) { + FSLMC_VFIO_LOG(ERR, "Unable to obtain VFIO groupid."); return ret; + } - FSLMC_VFIO_LOG(DEBUG, "VFIO iommu group id = %d", groupid); - - /* Check if group already exists */ - group = &vfio_group; - if (group->groupid == groupid) { - FSLMC_VFIO_LOG(ERR, "groupid already exists %d", - groupid); + /* In case this group was already opened, continue without any + * processing. + */ + if (vfio_group.groupid == groupid) { + FSLMC_VFIO_LOG(ERR, "groupid already exists %d", groupid); return 0; } - /* get the actual group fd */ + /* Get the actual group fd */ ret = vfio_get_group_fd(groupid); if (ret < 0) return ret; - group->fd = ret; + vfio_group.fd = ret; - /* - * at this point, we know that this group is viable (meaning, - * all devices are either bound to VFIO or not bound to anything) - */ - - ret = ioctl(group->fd, VFIO_GROUP_GET_STATUS, &status); + /* Check group viability */ + ret = ioctl(vfio_group.fd, VFIO_GROUP_GET_STATUS, &status); if (ret) { - FSLMC_VFIO_LOG(ERR, " VFIO error getting group status"); - close(group->fd); + FSLMC_VFIO_LOG(ERR, "VFIO error getting group status"); + close(vfio_group.fd); return ret; } if (!(status.flags & VFIO_GROUP_FLAGS_VIABLE)) { FSLMC_VFIO_LOG(ERR, "VFIO group not viable"); - close(group->fd); + close(vfio_group.fd); return -EPERM; } /* Since Group is VIABLE, Store the groupid */ - group->groupid = groupid; + vfio_group.groupid = groupid; /* check if group does not have a container yet */ if (!(status.flags & VFIO_GROUP_FLAGS_CONTAINER_SET)) { /* Now connect this IOMMU group to given container */ ret = vfio_connect_container(); if (ret) { - FSLMC_VFIO_LOG(ERR, "VFIO error connecting container" - " with groupid %d", groupid); - close(group->fd); + FSLMC_VFIO_LOG(ERR, + "Error connecting container with groupid %d", + groupid); + close(vfio_group.fd); return ret; } } /* Get Device information */ - ret = ioctl(group->fd, VFIO_GROUP_GET_DEVICE_FD, container); + ret = ioctl(vfio_group.fd, VFIO_GROUP_GET_DEVICE_FD, g_container); if (ret < 0) { - FSLMC_VFIO_LOG(ERR, "VFIO error getting device %s fd from" - " group %d", container, group->groupid); + FSLMC_VFIO_LOG(ERR, "Error getting device %s fd from group %d", + g_container, vfio_group.groupid); + close(vfio_group.fd); return ret; } container_device_fd = ret; FSLMC_VFIO_LOG(DEBUG, "VFIO Container FD is [0x%X]", - container_device_fd); + container_device_fd); return 0; } diff --git a/drivers/bus/fslmc/fslmc_vfio.h b/drivers/bus/fslmc/fslmc_vfio.h index 60a5743..edb86d0 100644 --- a/drivers/bus/fslmc/fslmc_vfio.h +++ b/drivers/bus/fslmc/fslmc_vfio.h @@ -36,7 +36,6 @@ #include "eal_vfio.h" -#define DPAA2_VENDOR_ID 0x1957 #define DPAA2_MC_DPNI_DEVID 7 #define DPAA2_MC_DPSECI_DEVID 3 #define DPAA2_MC_DPCON_DEVID 5 @@ -44,7 +43,6 @@ #define DPAA2_MC_DPBP_DEVID 10 #define DPAA2_MC_DPCI_DEVID 11 - typedef struct fslmc_vfio_device { int fd; /* fslmc root container device ?? */ int index; /*index of child object */ @@ -71,6 +69,8 @@ int rte_dpaa2_intr_enable(struct rte_intr_handle *intr_handle, int fslmc_vfio_setup_group(void); int fslmc_vfio_process_group(void); +char *fslmc_get_container(void); +int fslmc_get_container_group(int *gropuid); int rte_fslmc_vfio_dmamap(void); #endif /* _FSLMC_VFIO_H_ */ diff --git a/drivers/bus/fslmc/portal/dpaa2_hw_dpio.c b/drivers/bus/fslmc/portal/dpaa2_hw_dpio.c index da41849..7315555 100644 --- a/drivers/bus/fslmc/portal/dpaa2_hw_dpio.c +++ b/drivers/bus/fslmc/portal/dpaa2_hw_dpio.c @@ -59,7 +59,6 @@ #include #include -#include #include #include "dpaa2_hw_pvt.h" #include "dpaa2_hw_dpio.h" diff --git a/drivers/bus/fslmc/rte_fslmc.h b/drivers/bus/fslmc/rte_fslmc.h index 2f684d9..4c32db6 100644 --- a/drivers/bus/fslmc/rte_fslmc.h +++ b/drivers/bus/fslmc/rte_fslmc.h @@ -56,6 +56,9 @@ extern "C" { #include #include #include +#include + +#include #define FSLMC_OBJECT_MAX_LEN 32 /**< Length of each device on bus */