From patchwork Sat May 27 10:25:12 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shreyansh Jain X-Patchwork-Id: 24769 Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [IPv6:::1]) by dpdk.org (Postfix) with ESMTP id 4FF1C7CCA; Sat, 27 May 2017 12:18:56 +0200 (CEST) Received: from NAM03-BY2-obe.outbound.protection.outlook.com (mail-by2nam03on0046.outbound.protection.outlook.com [104.47.42.46]) by dpdk.org (Postfix) with ESMTP id 35B057CB5 for ; Sat, 27 May 2017 12:18:34 +0200 (CEST) Received: from BLUPR0301CA0020.namprd03.prod.outlook.com (10.162.113.158) by BY1PR0301MB0903.namprd03.prod.outlook.com (10.160.195.142) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1101.14; Sat, 27 May 2017 10:18:32 +0000 Received: from BL2FFO11FD040.protection.gbl (2a01:111:f400:7c09::190) by BLUPR0301CA0020.outlook.office365.com (2a01:111:e400:5259::30) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1124.9 via Frontend Transport; Sat, 27 May 2017 10:18:25 +0000 Authentication-Results: spf=fail (sender IP is 192.88.158.2) smtp.mailfrom=nxp.com; nxp.com; dkim=none (message not signed) header.d=none;nxp.com; dmarc=fail action=none header.from=nxp.com; Received-SPF: Fail (protection.outlook.com: domain of nxp.com does not designate 192.88.158.2 as permitted sender) receiver=protection.outlook.com; client-ip=192.88.158.2; helo=az84smr01.freescale.net; Received: from az84smr01.freescale.net (192.88.158.2) by BL2FFO11FD040.mail.protection.outlook.com (10.173.161.136) with Microsoft SMTP Server (version=TLS1_0, cipher=TLS_RSA_WITH_AES_256_CBC_SHA) id 15.1.1075.5 via Frontend Transport; Sat, 27 May 2017 10:18:24 +0000 Received: from Tophie.ap.freescale.net ([10.232.14.39]) by az84smr01.freescale.net (8.14.3/8.14.0) with ESMTP id v4RAHYu2029968; Sat, 27 May 2017 03:18:22 -0700 From: Shreyansh Jain To: CC: , Date: Sat, 27 May 2017 15:55:12 +0530 Message-ID: <1495880735-1651-17-git-send-email-shreyansh.jain@nxp.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1495880735-1651-1-git-send-email-shreyansh.jain@nxp.com> References: <1495880735-1651-1-git-send-email-shreyansh.jain@nxp.com> X-EOPAttributedMessage: 0 X-Matching-Connectors: 131403539048634307; (91ab9b29-cfa4-454e-5278-08d120cd25b8); () X-Forefront-Antispam-Report: CIP:192.88.158.2; IPV:NLI; CTRY:US; EFV:NLI; SFV:NSPM; SFS:(10009020)(6009001)(336005)(39840400002)(39860400002)(39380400002)(39400400002)(39450400003)(39410400002)(39850400002)(2980300002)(1110001)(1109001)(339900001)(189002)(199003)(9170700003)(38730400002)(47776003)(85426001)(110136004)(104016004)(53936002)(4326008)(50226002)(356003)(76176999)(6916009)(54906002)(53946003)(50466002)(48376002)(2950100002)(6666003)(50986999)(68736007)(2906002)(106466001)(77096006)(105606002)(86362001)(575784001)(498600001)(8656002)(33646002)(81166006)(5003940100001)(8936002)(36756003)(189998001)(2351001)(8676002)(5660300001)(305945005)(2004002); DIR:OUT; SFP:1101; SCL:1; SRVR:BY1PR0301MB0903; H:az84smr01.freescale.net; FPR:; SPF:Fail; MLV:ovrnspm; PTR:InfoDomainNonexistent; MX:1; A:1; LANG:en; X-Microsoft-Exchange-Diagnostics: 1; BL2FFO11FD040; 1:GZ7xNP2lwsk6veu8XjEP0erKYmXzq1N+0V3gAPNta7toF+t7SwfIxlgeMEr5drzr4wrprWgfU2MO118weeeFL4StyoGq6CDTdwrrY/Al6zW9x3bRH4jQHsKnAjQQCksyvzLBpybhhMdoSStlJOd9icz3Fc1HQqAY17Xn22y2QRF5o31SDC+53lSh24GOX0KJ3y4fz10fRx8OO3Vck5xbGQRGWY5wiRWdb/udvBKJ8A7XIziDJyuZ++U7iqUgT8jWQNxWCxEkjAc1yEHtxEjaDJ/eEHMNwbFHuiPgrkWsHz8svJC6AywLnmo481mPCnsLUAMYzpc4qRSmbInEg2qpkWHQqAXqySRNklHwvOGfZEG2aedifQsZa+9c8S3UuDSKLb/99yEhYRLxJ7VNlJlA6EHm4ynI+e3v7xh2EDBjE4trE8BMbaVE4n3Z5wq/BfH+taGdVNv8cp12n2fxBzSuBXs4hQDWOyeYV1YM8g/BNj78KEWOaHmW250zCVgBudMQjF3WRbA79AfnRgoa068+hxPt0Gl83sMmGEgrY5xtDARq87y1mh4p/LkulYqAwyt7GBJJEIo9JQ3tjv5KDSRfgY2/oYTAwIDIafdeq8Jiv9SOBCZvUmtEhlpPJNdqjEdWOPsCosG1MFTW8NbzckmkAeP+ltW4Ugg99btrUonx0wFPskGDXFBsUfmNBBaHohDoq1neuWyYwKTPacv5PB8KCQ== MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BY1PR0301MB0903: X-MS-Office365-Filtering-Correlation-Id: 27cb15cc-3192-4bf7-17c3-08d4a4e9b5b0 X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001)(201703131430075)(201703131517081); SRVR:BY1PR0301MB0903; X-Microsoft-Exchange-Diagnostics: 1; BY1PR0301MB0903; 3:KtFwZkgVXq8AtlHUptBar/cG8O1CW/A3JYfYfyCvES1N9Dp2k1IToD+PhX2WrandJqzGWP25GG0bpK0koFh6/a/SufhpHyh5xj2MXkGZuMBR7LFLUcFjP+Mx/snBg4e3UT4f3CKt/Kkn6mVGz71cWhk1dhQfn1Rso47mfRdMSQHoh8Fehl2PPO8mEbxGcCGvgVBkg9UA7zIjuBPSsiDUHxIpKLGzFBmo4kifh14e+ky9qCQ2A4xZGpE/F3q2I2X0hmitHuOK26Gi1r5tQHNh3DYQ4fNOUBsSWdTC9PYiRBXk3IIQH069AKPTCABm6W40Rv94wR2GzNl9QekfXALcfmt8JFhVFLDpQHHy2MiUNP1QfPFtOQi4lBurbelT6DjjxmtHk+/DRdoztvWzjIhwCXRYLvDIgAgNH/dEwtL7xE5rNwN4PfwYl7vghj7xIdAC; 25:VXpdG9wn3/7W6cc8Yy9yKa7sZ0y/Nn0XaLuLI+TJOGEyaiG/JeBaLacXXfKNByHkCAwp3aRVOiYMzrNGM92Sa0NxY3x9iWmdGQew0tey1su3v39aRm9ikcIfjzo5ddax+6Cyp/0hW/wAJcAyhQF3dzTDTwA2K8FjpFMeDtxmD3gHLQ3wS1rXuEtyUX8VVINFngrgF0J6x4olbjauJgxpIKcuI6w0ZnRzhE04WqvZhqHlWNbV5udKDr4K9pxvCGz3FNGIyf/ZddUxn59W+GRJRUe7qPORlHFAxeIalyJE/mF6erccszVRcMi2NQD/a3/UpdoOCVYIkWyDWg0H1u1X323uMjbKYU4TDFFZJoScwf0zURLK4Yd8GhO55/3XKrywwWW0wI7temF/N3j6jeL5D2Wpw4UpnKSmtGu8AcavEfk6gU5xJMYLaASEHpJCAO2Zv+o3tWEvRKQd1MDZOipt20+xTIvWN7SAawxZPIrsEkE= X-Microsoft-Exchange-Diagnostics: 1; BY1PR0301MB0903; 31:7+jMHcmsFFuFzwCc6fXlBfWnylSjpzRYwnR8QtYtirX9O3XliaFbY5GWK1QBox1rJ3pdaHTaYl6TPry7WuaqkKaADFrTsX8A7UvwgsAKsFvZ3Fj9hdwD1/+4nr77u2llNNKThutWE+daCxq1U0l5DakKwmstRerUo/i9xByEIV5IwyHBlsIW929rKRLuscXOCySyPPL3WNhoTu5gM+MhVDCbMjd7mIMQdBAWDztYvi53Y9ZFk8YWBlL1IUlHhD7JMs0x34CN79SWzMKpbHFQVA== X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(185117386973197)(101931422205132)(275809806118684); X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(6095135)(601004)(2401047)(13016025)(5005006)(13018025)(8121501046)(3002001)(10201501046)(93006095)(93001095)(6055026)(6096035)(20161123561025)(20161123559100)(20161123563025)(20161123565025)(20161123556025)(201703131430075)(201703131433075)(201703131448075)(201703161259150)(201703151042153); SRVR:BY1PR0301MB0903; BCL:0; PCL:0; RULEID:(400006); SRVR:BY1PR0301MB0903; X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; BY1PR0301MB0903; 4:9BaNARrFFYbWP2CXvIVUhu74197sElPaV0sT92vk?= H+Wa/ftLnG5cNo5A41t3YZNZVfknSraLogr23Wf709jXJLkDGgsAs7MOFjKIC5+RkVs62Dw8eLp5irbB1didYZNrAQ16xY7AinWYiKdOlEWxFOowpZPl54Eu8DCj16rNCqowffHQhFOt06GQCBNcsv7Mk9oMyQZfvfTHyAQ9VzvsGjWougPJpjvThydgE0FvBp8uROwS5VkAL6YlLxuvH1R9/3/ID7G/QbhGUJSyswPfmK26KYmB99EF2un5CM83RdhpmtZmyaDSSoDvCjXbP8ea9fdO+BRWSnhX4pNTeQnRSzZufsXnIZZ4QkEutt8DCkwGefaWUebCM6PxR0PSsA8HFOgzaZtmjTIRu3wNRKto9q81z7wPabDqg5yX5mgu33SVsdzQ4Ed5GwDf9my2iuFf9+NoH621lq3Sg0SZHhtdOmA4wZnMuP4Juntr2nv2zm704iBQk0HL6MqGXXo91MyGrEM6/BuI7/NqC0/8+VvHbuScdkpziHfzb4fwQTV7/gDvs5uQPKi3t3rkxMcmu1Er50zMimlkeWvG+W8GsD+QsLYAp4udgS6IC9KGPMdgs+qBrp/ze9PuTwL3MZIlBtVxGCYLHJlvQfmlqLG2gQH+fEWGGF+Ac5iaevNGaKmvat/mdsQJngkev5/b5VcH2hMygg+Oxx4G5fG8AZG8QHmKyLXPWNluNLssCpevgbcwi/KsnGljLWEEHDeeD2PocLZh0ynIcgem7F6iUTHKuEVhhx3YLLnW74UQKyzspws6TSIL40EFYeAxYJzyfAcl87/gt/vi7mhEibTN4QM70JW0RQBfeOSkoLo211cGqxy1QrJSTrvyo2K+QdByJgjE/d04T5otZhkULYPmsUTLnuhhvW6GLNqzlKLxwWG8zXtmryCgUni15t7sRSV6TMA6GM+1vEGkp8QFd82FEC8HBPQ1Gw== X-Forefront-PRVS: 0320B28BE1 X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; BY1PR0301MB0903; 23:GfRXSQOmMMGdLJuDGGAh22YjvdaDHFBAzDj46NQ?= JVbZVX12wLxjWVMy/JqlpZDsbOWKf8djqF0iBNEgLzu8xyftppozlR9b6uTU9qJ8ubVPtX8CH/0l1qvWgD71Uf+hm1Q1WzrWy5JKxvFxt5wgbZeHbhteePrOmuc1j5F4fkxM5kkxEUGJznQgX/IvM+2WXEuZyAAIlFeh7FWoaIQ1P+9yy0SjobdGSdJDh7EypI7yuNpIqocJHYHUva2rCX7zDpJRo+tNGA9CUVG2G2W/Mb7aSxPESrexzBMLi1+IeL+rVi9JQfClq9xzXyNredho0BSOgZrx7t5aZDsR27oXrg01e47K7pxZH8zePx20EcjwbM8Z153eulL4SBnlhRGazl2Kp5CI0ksN9AetcTv1we4gmVNblLdrUYTl8T6TUnF+hrxMUHUSNujU9/32HsEDdtOYAzzBViE5z7i/tEReVPuyzRVWqRyGBR2MKyj7AG2b+41LmxR+tq4meifWglzgniwIwLRws0H1xIkhYYcIwlwTRB7mX9Nou5I8kOoXH/zyCsNi/ZsBqRI2pE4jb0VuJ8Rb6n051O1jeMYksXFVzicHXSoVwzSw+emzaY1O19QtKf6l31LBQvv5MGuzcTjf6BiQ2InsmjS2w+1wfGT+5ZYXtDLm1jYPmdoGiWBkLyiXcUm7hkvVJZesfYodLFBN+F1DS/ansq0rxiP+WxN/g8qSPEL1mcEv4htmYjtqLfR6JQkfGqwx5nal96UVRF2cIIbInWaK6Ka3qB4HPdM05DeAL07x8EUk2V20UphBG9/Y2dp6bj2ovxh8QelSRoV8V2XK0yirKb5D59ffLxMKrZ6PlG7S2o0G/CrIhzL714dnnW6EGqCnjn4f6eJRBFgjumg5ww4W/LOs3rNLoT9fRzABnxn4LmOCAIwNuE14rt9OTrxzC9Zz3sZYQ39arDdDXn31q1z6nCN5Q2zCmnRseK2z3Ml6EPdTQ6LY0Olm1W6AL5yzhUG3fR24NNqSOjn8vEYN2c9hJw3GibDQSVl4xZma5+WfIWGzkm7XgmBRzMpn86JaT6SFvwdST7U4ncv2BekgOwhT34R3b293zBEiXPpPiaKhQSHEVttypPSKzTGM1NFFYSClX3FxdGR0g92Fk46xKuUhwQTqdEG1Mka9brWTtORtldYafVEC0KKs0PXZ2iKD/8uIFm+nEjyHSG8VD7Uy0Qev3FeMnntph2lY28EVBfsjhRB/PU89ByRcGp4/ryG3gkyCSLZZU/0mDM7Vm9djmESuND5FazSsnVLO3PxclMifGelB2cXEMsl/COkrof6WckfcDUP9KyCpFqmoM X-Microsoft-Exchange-Diagnostics: 1; BY1PR0301MB0903; 6:50olLQVM3Kmlu5BOTv80DG8XzrcvglAkZtI0979RoO+NjTcSDIPVR+y1AnmmMdJRFhAXvk6oHCb+ORxq/duhCYa3IwXXSRwMgZFEwJS4EoY4xEaKi8OO5t6zyjCffGWEfIOB2nuecomGpgm9Z+eo2aC+as+o2BwtaFgwOrmDm8IIspYHIXQ0yzoE/Zx7odJJFhXsDjHG8fPG4nIRy+JoOnbHrssOYhvUzSswtxX7ngXssZkQz6lWJcnXTFLs1NVpXwM6LBGNR3qGRNvuMjLhiVIcjr4q9cNWyApJoNkUXn+KULD0IfrScBwjYNOXK8ZHequ9mwLQtxFwqhiE7iqRUSBwLJ0I8F06ewEAeLGIAMQABkTE8es6jJr23JW08qXQslZpOt3elqIrQ9VxpK3MuOCs4PLzTy4cbbW+KOCeI9r2lVSEUYGN4ggm1LlKe0poF59cmde+DDXk9Q+cqumlt/3QzP0+8qt0A72HnsGYrS//5YJKu0wSKXbqHCh0xZyumJCNU2ajY0ZapSQilAbIug==; 5:/YH83E5nIgf8Jr/BV+aAOI+IfzZjqF3pgOqZnE1sFGh7FTk8AorUmmP3wRiqnM+42UFYa4ctKQi/1xHe5J0AZtlTrChKpOj9L6o1/CZWJqM3rHSfG5NxIU3pK47Njes/12mwQr/YaB5ce8oiPgBFmoM5QLxC8M0sC2/c+7SJ6zVA7JTS8WI+LPFfn4/EwmDK; 24:ejK43JT/VoTtJV55uT3rD9B/kXspAsIdTWaa+O/snbag+QgLrq5yxiUIriCmTHnoJH/1CeVhEHIwFVyqzBh+ZvNqffYP9Q7hg5IyG1py2uY= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; BY1PR0301MB0903; 7:5iAup46Tkl62EOQdoPNgMxkQh8ecRf3HjzhLCJg7JwVocMBol+cQBpvW6h4eNoLd7pseseqwvePDSiBR+5FMUk4fqmdmgjzXErRiumY8/4BLugW5GeyFmyQe3rskekZJmNYM8v7xTYhuMUfBAiTDkjCzYimxR3H9Z2PwOiyq4bwC1aBPV4bhL8Ewueb+o7qF3nYB5HhAyiu4sGf7SnrC/kLtGFAa1j0lWGo4s8sY6sYyKJ++YP+bkPWcyYfLhy03Y7+JXfGSd155IEFOoAENLEuNK6vKotZIvJ1MaISoLpagA4KQfiXCw5iocD/gu/4jz3IOIacoxatHybVhW3+78g== X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 May 2017 10:18:24.6450 (UTC) X-MS-Exchange-CrossTenant-Id: 5afe0b00-7697-4969-b663-5eab37d5f47e X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=5afe0b00-7697-4969-b663-5eab37d5f47e; Ip=[192.88.158.2]; Helo=[az84smr01.freescale.net] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY1PR0301MB0903 Subject: [dpdk-dev] [RFC Patch 16/39] bus/dpaa: add BMan hardware interfaces X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Signed-off-by: Geoff Thorpe Signed-off-by: Roy Pledge Signed-off-by: Hemant Agrawal Signed-off-by: Shreyansh Jain --- drivers/bus/dpaa/Makefile | 1 + drivers/bus/dpaa/base/qbman/bman.c | 394 +++++++++++++++++++++ drivers/bus/dpaa/base/qbman/bman.h | 550 ++++++++++++++++++++++++++++++ drivers/bus/dpaa/base/qbman/bman_driver.c | 12 + drivers/bus/dpaa/base/qbman/dpaa_alloc.c | 16 + 5 files changed, 973 insertions(+) create mode 100644 drivers/bus/dpaa/base/qbman/bman.c create mode 100644 drivers/bus/dpaa/base/qbman/bman.h diff --git a/drivers/bus/dpaa/Makefile b/drivers/bus/dpaa/Makefile index f11af6d..29a065a 100644 --- a/drivers/bus/dpaa/Makefile +++ b/drivers/bus/dpaa/Makefile @@ -71,6 +71,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_DPAA_BUS) += \ base/fman/of.c \ base/fman/netcfg_layer.c \ base/qbman/process.c \ + base/qbman/bman.c \ base/qbman/bman_driver.c \ base/qbman/qman.c \ base/qbman/qman_driver.c \ diff --git a/drivers/bus/dpaa/base/qbman/bman.c b/drivers/bus/dpaa/base/qbman/bman.c new file mode 100644 index 0000000..a0bea62 --- /dev/null +++ b/drivers/bus/dpaa/base/qbman/bman.c @@ -0,0 +1,394 @@ +/*- + * This file is provided under a dual BSD/GPLv2 license. When using or + * redistributing this file, you may do so under either license. + * + * BSD LICENSE + * + * Copyright 2008-2016 Freescale Semiconductor Inc. + * Copyright 2017 NXP. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions are met: + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * * Neither the name of the above-listed copyright holders nor the + * names of any contributors may be used to endorse or promote products + * derived from this software without specific prior written permission. + * + * GPL LICENSE SUMMARY + * + * ALTERNATIVELY, this software may be distributed under the terms of the + * GNU General Public License ("GPL") as published by the Free Software + * Foundation, either version 2 of that License or (at your option) any + * later version. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" + * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDERS OR CONTRIBUTORS BE + * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR + * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF + * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS + * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN + * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) + * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE + * POSSIBILITY OF SUCH DAMAGE. + */ + +#include "bman.h" +#include + +/* Compilation constants */ +#define RCR_THRESH 2 /* reread h/w CI when running out of space */ +#define IRQNAME "BMan portal %d" +#define MAX_IRQNAME 16 /* big enough for "BMan portal %d" */ + +struct bman_portal { + struct bm_portal p; + /* 2-element array. pools[0] is mask, pools[1] is snapshot. */ + struct bman_depletion *pools; + int thresh_set; + unsigned long irq_sources; + u32 slowpoll; /* only used when interrupts are off */ + /* When the cpu-affine portal is activated, this is non-NULL */ + const struct bm_portal_config *config; + char irqname[MAX_IRQNAME]; +}; + +static cpumask_t affine_mask; +static DEFINE_SPINLOCK(affine_mask_lock); +static DEFINE_PER_CPU(struct bman_portal, bman_affine_portal); + +static inline struct bman_portal *get_affine_portal(void) +{ + return &get_cpu_var(bman_affine_portal); +} + +/* + * This object type refers to a pool, it isn't *the* pool. There may be + * more than one such object per BMan buffer pool, eg. if different users of + * the pool are operating via different portals. + */ +struct bman_pool { + struct bman_pool_params params; + /* Used for hash-table admin when using depletion notifications. */ + struct bman_portal *portal; + struct bman_pool *next; +#ifdef RTE_LIBRTE_DPAA_CHECKING + atomic_t in_use; +#endif +}; + +static inline +struct bman_portal *bman_create_portal(struct bman_portal *portal, + const struct bm_portal_config *c) +{ + struct bm_portal *p; + const struct bman_depletion *pools = &c->mask; + int ret; + u8 bpid = 0; + + p = &portal->p; + /* + * prep the low-level portal struct with the mapped addresses from the + * config, everything that follows depends on it and "config" is more + * for (de)reference... + */ + p->addr.ce = c->addr_virt[DPAA_PORTAL_CE]; + p->addr.ci = c->addr_virt[DPAA_PORTAL_CI]; + if (bm_rcr_init(p, bm_rcr_pvb, bm_rcr_cce)) { + pr_err("Bman RCR initialisation failed\n"); + return NULL; + } + if (bm_mc_init(p)) { + pr_err("Bman MC initialisation failed\n"); + goto fail_mc; + } + portal->pools = kmalloc(2 * sizeof(*pools), GFP_KERNEL); + if (!portal->pools) + goto fail_pools; + portal->pools[0] = *pools; + bman_depletion_init(portal->pools + 1); + while (bpid < bman_pool_max) { + /* + * Default to all BPIDs disabled, we enable as required at + * run-time. + */ + bm_isr_bscn_mask(p, bpid, 0); + bpid++; + } + portal->slowpoll = 0; + /* Write-to-clear any stale interrupt status bits */ + bm_isr_disable_write(p, 0xffffffff); + portal->irq_sources = 0; + bm_isr_enable_write(p, portal->irq_sources); + bm_isr_status_clear(p, 0xffffffff); + snprintf(portal->irqname, MAX_IRQNAME, IRQNAME, c->cpu); + if (request_irq(c->irq, NULL, 0, portal->irqname, + portal)) { + pr_err("request_irq() failed\n"); + goto fail_irq; + } + + /* Need RCR to be empty before continuing */ + ret = bm_rcr_get_fill(p); + if (ret) { + pr_err("Bman RCR unclean\n"); + goto fail_rcr_empty; + } + /* Success */ + portal->config = c; + + bm_isr_disable_write(p, 0); + bm_isr_uninhibit(p); + return portal; +fail_rcr_empty: + free_irq(c->irq, portal); +fail_irq: + kfree(portal->pools); +fail_pools: + bm_mc_finish(p); +fail_mc: + bm_rcr_finish(p); + return NULL; +} + +struct bman_portal * +bman_create_affine_portal(const struct bm_portal_config *c) +{ + struct bman_portal *portal = get_affine_portal(); + + /*This function is called from the context which is already affine to + *CPU or in other words this in non-migratable to other CPUs. + */ + portal = bman_create_portal(portal, c); + if (portal) { + spin_lock(&affine_mask_lock); + CPU_SET(c->cpu, &affine_mask); + spin_unlock(&affine_mask_lock); + } + return portal; +} + +static inline +void bman_destroy_portal(struct bman_portal *bm) +{ + const struct bm_portal_config *pcfg; + + pcfg = bm->config; + bm_rcr_cce_update(&bm->p); + bm_rcr_cce_update(&bm->p); + + free_irq(pcfg->irq, bm); + + kfree(bm->pools); + bm_mc_finish(&bm->p); + bm_rcr_finish(&bm->p); + bm->config = NULL; +} + +const struct +bm_portal_config *bman_destroy_affine_portal(void) +{ + struct bman_portal *bm = get_affine_portal(); + const struct bm_portal_config *pcfg; + + pcfg = bm->config; + bman_destroy_portal(bm); + spin_lock(&affine_mask_lock); + CPU_CLR(pcfg->cpu, &affine_mask); + spin_unlock(&affine_mask_lock); + return pcfg; +} + +int +bman_get_portal_index(void) +{ + struct bman_portal *p = get_affine_portal(); + return p->config->index; +} + +static const u32 zero_thresholds[4] = {0, 0, 0, 0}; + +struct bman_pool *bman_new_pool(const struct bman_pool_params *params) +{ + struct bman_pool *pool = NULL; + u32 bpid; + + if (params->flags & BMAN_POOL_FLAG_DYNAMIC_BPID) { + int ret = bman_alloc_bpid(&bpid); + + if (ret) + return NULL; + } else { + if (params->bpid >= bman_pool_max) + return NULL; + bpid = params->bpid; + } + if (params->flags & BMAN_POOL_FLAG_THRESH) { + int ret = bm_pool_set(bpid, params->thresholds); + + if (ret) + goto err; + } + + pool = kmalloc(sizeof(*pool), GFP_KERNEL); + if (!pool) + goto err; + pool->params = *params; +#ifdef RTE_LIBRTE_DPAA_CHECKING + atomic_set(&pool->in_use, 1); +#endif + if (params->flags & BMAN_POOL_FLAG_DYNAMIC_BPID) + pool->params.bpid = bpid; + + return pool; +err: + if (params->flags & BMAN_POOL_FLAG_THRESH) + bm_pool_set(bpid, zero_thresholds); + + if (params->flags & BMAN_POOL_FLAG_DYNAMIC_BPID) + bman_release_bpid(bpid); + kfree(pool); + + return NULL; +} + +void bman_free_pool(struct bman_pool *pool) +{ + if (pool->params.flags & BMAN_POOL_FLAG_THRESH) + bm_pool_set(pool->params.bpid, zero_thresholds); + if (pool->params.flags & BMAN_POOL_FLAG_DYNAMIC_BPID) + bman_release_bpid(pool->params.bpid); + kfree(pool); +} + +const struct bman_pool_params *bman_get_params(const struct bman_pool *pool) +{ + return &pool->params; +} + +static void update_rcr_ci(struct bman_portal *p, int avail) +{ + if (avail) + bm_rcr_cce_prefetch(&p->p); + else + bm_rcr_cce_update(&p->p); +} + +#define BMAN_BUF_MASK 0x0000fffffffffffful +int bman_release(struct bman_pool *pool, const struct bm_buffer *bufs, u8 num, + u32 flags __maybe_unused) +{ + struct bman_portal *p; + struct bm_rcr_entry *r; + u32 i = num - 1; + u8 avail; + +#ifdef RTE_LIBRTE_DPAA_CHECKING + if (!num || (num > 8)) + return -EINVAL; + if (pool->params.flags & BMAN_POOL_FLAG_NO_RELEASE) + return -EINVAL; +#endif + + p = get_affine_portal(); + avail = bm_rcr_get_avail(&p->p); + if (avail < 2) + update_rcr_ci(p, avail); + r = bm_rcr_start(&p->p); + if (unlikely(!r)) + return -EBUSY; + + /* + * we can copy all but the first entry, as this can trigger badness + * with the valid-bit + */ + r->bufs[0].opaque = + cpu_to_be64(((u64)pool->params.bpid << 48) | + (bufs[0].opaque & BMAN_BUF_MASK)); + if (i) { + for (i = 1; i < num; i++) + r->bufs[i].opaque = + cpu_to_be64(bufs[i].opaque & BMAN_BUF_MASK); + } + + bm_rcr_pvb_commit(&p->p, BM_RCR_VERB_CMD_BPID_SINGLE | + (num & BM_RCR_VERB_BUFCOUNT_MASK)); + + return 0; +} + +int bman_acquire(struct bman_pool *pool, struct bm_buffer *bufs, u8 num, + u32 flags __maybe_unused) +{ + struct bman_portal *p = get_affine_portal(); + struct bm_mc_command *mcc; + struct bm_mc_result *mcr; + int ret, i; + +#ifdef RTE_LIBRTE_DPAA_CHECKING + if (!num || (num > 8)) + return -EINVAL; + if (pool->params.flags & BMAN_POOL_FLAG_ONLY_RELEASE) + return -EINVAL; +#endif + + mcc = bm_mc_start(&p->p); + mcc->acquire.bpid = pool->params.bpid; + bm_mc_commit(&p->p, BM_MCC_VERB_CMD_ACQUIRE | + (num & BM_MCC_VERB_ACQUIRE_BUFCOUNT)); + while (!(mcr = bm_mc_result(&p->p))) + cpu_relax(); + ret = mcr->verb & BM_MCR_VERB_ACQUIRE_BUFCOUNT; + if (bufs) { + for (i = 0; i < num; i++) + bufs[i].opaque = + be64_to_cpu(mcr->acquire.bufs[i].opaque); + } + if (ret != num) + ret = -ENOMEM; + return ret; +} + +int bman_query_pools(struct bm_pool_state *state) +{ + struct bman_portal *p = get_affine_portal(); + struct bm_mc_result *mcr; + + bm_mc_start(&p->p); + bm_mc_commit(&p->p, BM_MCC_VERB_CMD_QUERY); + while (!(mcr = bm_mc_result(&p->p))) + cpu_relax(); + DPAA_ASSERT((mcr->verb & BM_MCR_VERB_CMD_MASK) == + BM_MCR_VERB_CMD_QUERY); + *state = mcr->query; + state->as.state.state[0] = be32_to_cpu(state->as.state.state[0]); + state->as.state.state[1] = be32_to_cpu(state->as.state.state[1]); + state->ds.state.state[0] = be32_to_cpu(state->ds.state.state[0]); + state->ds.state.state[1] = be32_to_cpu(state->ds.state.state[1]); + return 0; +} + +u32 bman_query_free_buffers(struct bman_pool *pool) +{ + return bm_pool_free_buffers(pool->params.bpid); +} + +int bman_update_pool_thresholds(struct bman_pool *pool, const u32 *thresholds) +{ + u32 bpid; + + bpid = bman_get_params(pool)->bpid; + + return bm_pool_set(bpid, thresholds); +} + +int bman_shutdown_pool(u32 bpid) +{ + struct bman_portal *p = get_affine_portal(); + return bm_shutdown_pool(&p->p, bpid); +} diff --git a/drivers/bus/dpaa/base/qbman/bman.h b/drivers/bus/dpaa/base/qbman/bman.h new file mode 100644 index 0000000..dcca303 --- /dev/null +++ b/drivers/bus/dpaa/base/qbman/bman.h @@ -0,0 +1,550 @@ +/*- + * This file is provided under a dual BSD/GPLv2 license. When using or + * redistributing this file, you may do so under either license. + * + * BSD LICENSE + * + * Copyright 2010-2016 Freescale Semiconductor Inc. + * Copyright 2017 NXP. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions are met: + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * * Neither the name of the above-listed copyright holders nor the + * names of any contributors may be used to endorse or promote products + * derived from this software without specific prior written permission. + * + * GPL LICENSE SUMMARY + * + * ALTERNATIVELY, this software may be distributed under the terms of the + * GNU General Public License ("GPL") as published by the Free Software + * Foundation, either version 2 of that License or (at your option) any + * later version. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" + * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDERS OR CONTRIBUTORS BE + * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR + * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF + * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS + * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN + * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) + * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE + * POSSIBILITY OF SUCH DAMAGE. + */ + +#ifndef __BMAN_H +#define __BMAN_H + +#include "bman_priv.h" + +/* Cache-inhibited register offsets */ +#define BM_REG_RCR_PI_CINH 0x3000 +#define BM_REG_RCR_CI_CINH 0x3100 +#define BM_REG_RCR_ITR 0x3200 +#define BM_REG_CFG 0x3300 +#define BM_REG_SCN(n) (0x3400 + ((n) << 6)) +#define BM_REG_ISR 0x3e00 +#define BM_REG_IIR 0x3ec0 + +/* Cache-enabled register offsets */ +#define BM_CL_CR 0x0000 +#define BM_CL_RR0 0x0100 +#define BM_CL_RR1 0x0140 +#define BM_CL_RCR 0x1000 +#define BM_CL_RCR_PI_CENA 0x3000 +#define BM_CL_RCR_CI_CENA 0x3100 + +/* BTW, the drivers (and h/w programming model) already obtain the required + * synchronisation for portal accesses via lwsync(), hwsync(), and + * data-dependencies. Use of barrier()s or other order-preserving primitives + * simply degrade performance. Hence the use of the __raw_*() interfaces, which + * simply ensure that the compiler treats the portal registers as volatile (ie. + * non-coherent). + */ + +/* Cache-inhibited register access. */ +#define __bm_in(bm, o) be32_to_cpu(__raw_readl((bm)->ci + (o))) +#define __bm_out(bm, o, val) __raw_writel(cpu_to_be32(val), \ + (bm)->ci + (o)) +#define bm_in(reg) __bm_in(&portal->addr, BM_REG_##reg) +#define bm_out(reg, val) __bm_out(&portal->addr, BM_REG_##reg, val) + +/* Cache-enabled (index) register access */ +#define __bm_cl_touch_ro(bm, o) dcbt_ro((bm)->ce + (o)) +#define __bm_cl_touch_rw(bm, o) dcbt_rw((bm)->ce + (o)) +#define __bm_cl_in(bm, o) be32_to_cpu(__raw_readl((bm)->ce + (o))) +#define __bm_cl_out(bm, o, val) \ + do { \ + u32 *__tmpclout = (bm)->ce + (o); \ + __raw_writel(cpu_to_be32(val), __tmpclout); \ + dcbf(__tmpclout); \ + } while (0) +#define __bm_cl_invalidate(bm, o) dccivac((bm)->ce + (o)) +#define bm_cl_touch_ro(reg) __bm_cl_touch_ro(&portal->addr, BM_CL_##reg##_CENA) +#define bm_cl_touch_rw(reg) __bm_cl_touch_rw(&portal->addr, BM_CL_##reg##_CENA) +#define bm_cl_in(reg) __bm_cl_in(&portal->addr, BM_CL_##reg##_CENA) +#define bm_cl_out(reg, val) __bm_cl_out(&portal->addr, BM_CL_##reg##_CENA, val) +#define bm_cl_invalidate(reg)\ + __bm_cl_invalidate(&portal->addr, BM_CL_##reg##_CENA) + +/* Cyclic helper for rings. FIXME: once we are able to do fine-grain perf + * analysis, look at using the "extra" bit in the ring index registers to avoid + * cyclic issues. + */ +static inline u8 bm_cyc_diff(u8 ringsize, u8 first, u8 last) +{ + /* 'first' is included, 'last' is excluded */ + if (first <= last) + return last - first; + return ringsize + last - first; +} + +/* Portal modes. + * Enum types; + * pmode == production mode + * cmode == consumption mode, + * Enum values use 3 letter codes. First letter matches the portal mode, + * remaining two letters indicate; + * ci == cache-inhibited portal register + * ce == cache-enabled portal register + * vb == in-band valid-bit (cache-enabled) + */ +enum bm_rcr_pmode { /* matches BCSP_CFG::RPM */ + bm_rcr_pci = 0, /* PI index, cache-inhibited */ + bm_rcr_pce = 1, /* PI index, cache-enabled */ + bm_rcr_pvb = 2 /* valid-bit */ +}; + +enum bm_rcr_cmode { /* s/w-only */ + bm_rcr_cci, /* CI index, cache-inhibited */ + bm_rcr_cce /* CI index, cache-enabled */ +}; + +/* --- Portal structures --- */ + +#define BM_RCR_SIZE 8 + +struct bm_rcr { + struct bm_rcr_entry *ring, *cursor; + u8 ci, available, ithresh, vbit; +#ifdef RTE_LIBRTE_DPAA_CHECKING + u32 busy; + enum bm_rcr_pmode pmode; + enum bm_rcr_cmode cmode; +#endif +}; + +struct bm_mc { + struct bm_mc_command *cr; + struct bm_mc_result *rr; + u8 rridx, vbit; +#ifdef RTE_LIBRTE_DPAA_CHECKING + enum { + /* Can only be _mc_start()ed */ + mc_idle, + /* Can only be _mc_commit()ed or _mc_abort()ed */ + mc_user, + /* Can only be _mc_retry()ed */ + mc_hw + } state; +#endif +}; + +struct bm_addr { + void __iomem *ce; /* cache-enabled */ + void __iomem *ci; /* cache-inhibited */ +}; + +struct bm_portal { + struct bm_addr addr; + struct bm_rcr rcr; + struct bm_mc mc; + struct bm_portal_config config; +} ____cacheline_aligned; + +/* Bit-wise logic to wrap a ring pointer by clearing the "carry bit" */ +#define RCR_CARRYCLEAR(p) \ + (void *)((unsigned long)(p) & (~(unsigned long)(BM_RCR_SIZE << 6))) + +/* Bit-wise logic to convert a ring pointer to a ring index */ +static inline u8 RCR_PTR2IDX(struct bm_rcr_entry *e) +{ + return ((uintptr_t)e >> 6) & (BM_RCR_SIZE - 1); +} + +/* Increment the 'cursor' ring pointer, taking 'vbit' into account */ +static inline void RCR_INC(struct bm_rcr *rcr) +{ + /* NB: this is odd-looking, but experiments show that it generates + * fast code with essentially no branching overheads. We increment to + * the next RCR pointer and handle overflow and 'vbit'. + */ + struct bm_rcr_entry *partial = rcr->cursor + 1; + + rcr->cursor = RCR_CARRYCLEAR(partial); + if (partial != rcr->cursor) + rcr->vbit ^= BM_RCR_VERB_VBIT; +} + +static inline int bm_rcr_init(struct bm_portal *portal, enum bm_rcr_pmode pmode, + __maybe_unused enum bm_rcr_cmode cmode) +{ + /* This use of 'register', as well as all other occurrences, is because + * it has been observed to generate much faster code with gcc than is + * otherwise the case. + */ + register struct bm_rcr *rcr = &portal->rcr; + u32 cfg; + u8 pi; + + rcr->ring = portal->addr.ce + BM_CL_RCR; + rcr->ci = bm_in(RCR_CI_CINH) & (BM_RCR_SIZE - 1); + + pi = bm_in(RCR_PI_CINH) & (BM_RCR_SIZE - 1); + rcr->cursor = rcr->ring + pi; + rcr->vbit = (bm_in(RCR_PI_CINH) & BM_RCR_SIZE) ? BM_RCR_VERB_VBIT : 0; + rcr->available = BM_RCR_SIZE - 1 + - bm_cyc_diff(BM_RCR_SIZE, rcr->ci, pi); + rcr->ithresh = bm_in(RCR_ITR); +#ifdef RTE_LIBRTE_DPAA_CHECKING + rcr->busy = 0; + rcr->pmode = pmode; + rcr->cmode = cmode; +#endif + cfg = (bm_in(CFG) & 0xffffffe0) | (pmode & 0x3); /* BCSP_CFG::RPM */ + bm_out(CFG, cfg); + return 0; +} + +static inline void bm_rcr_finish(struct bm_portal *portal) +{ + register struct bm_rcr *rcr = &portal->rcr; + u8 pi = bm_in(RCR_PI_CINH) & (BM_RCR_SIZE - 1); + u8 ci = bm_in(RCR_CI_CINH) & (BM_RCR_SIZE - 1); + + DPAA_ASSERT(!rcr->busy); + if (pi != RCR_PTR2IDX(rcr->cursor)) + pr_crit("losing uncommited RCR entries\n"); + if (ci != rcr->ci) + pr_crit("missing existing RCR completions\n"); + if (rcr->ci != RCR_PTR2IDX(rcr->cursor)) + pr_crit("RCR destroyed unquiesced\n"); +} + +static inline struct bm_rcr_entry *bm_rcr_start(struct bm_portal *portal) +{ + register struct bm_rcr *rcr = &portal->rcr; + + DPAA_ASSERT(!rcr->busy); + if (!rcr->available) + return NULL; +#ifdef RTE_LIBRTE_DPAA_CHECKING + rcr->busy = 1; +#endif + dcbz_64(rcr->cursor); + return rcr->cursor; +} + +static inline void bm_rcr_abort(struct bm_portal *portal) +{ + __maybe_unused register struct bm_rcr *rcr = &portal->rcr; + + DPAA_ASSERT(rcr->busy); +#ifdef RTE_LIBRTE_DPAA_CHECKING + rcr->busy = 0; +#endif +} + +static inline struct bm_rcr_entry *bm_rcr_pend_and_next( + struct bm_portal *portal, u8 myverb) +{ + register struct bm_rcr *rcr = &portal->rcr; + + DPAA_ASSERT(rcr->busy); + DPAA_ASSERT(rcr->pmode != bm_rcr_pvb); + if (rcr->available == 1) + return NULL; + rcr->cursor->__dont_write_directly__verb = myverb | rcr->vbit; + dcbf_64(rcr->cursor); + RCR_INC(rcr); + rcr->available--; + dcbz_64(rcr->cursor); + return rcr->cursor; +} + +static inline void bm_rcr_pci_commit(struct bm_portal *portal, u8 myverb) +{ + register struct bm_rcr *rcr = &portal->rcr; + + DPAA_ASSERT(rcr->busy); + DPAA_ASSERT(rcr->pmode == bm_rcr_pci); + rcr->cursor->__dont_write_directly__verb = myverb | rcr->vbit; + RCR_INC(rcr); + rcr->available--; + hwsync(); + bm_out(RCR_PI_CINH, RCR_PTR2IDX(rcr->cursor)); +#ifdef RTE_LIBRTE_DPAA_CHECKING + rcr->busy = 0; +#endif +} + +static inline void bm_rcr_pce_prefetch(struct bm_portal *portal) +{ + __maybe_unused register struct bm_rcr *rcr = &portal->rcr; + + DPAA_ASSERT(rcr->pmode == bm_rcr_pce); + bm_cl_invalidate(RCR_PI); + bm_cl_touch_rw(RCR_PI); +} + +static inline void bm_rcr_pce_commit(struct bm_portal *portal, u8 myverb) +{ + register struct bm_rcr *rcr = &portal->rcr; + + DPAA_ASSERT(rcr->busy); + DPAA_ASSERT(rcr->pmode == bm_rcr_pce); + rcr->cursor->__dont_write_directly__verb = myverb | rcr->vbit; + RCR_INC(rcr); + rcr->available--; + lwsync(); + bm_cl_out(RCR_PI, RCR_PTR2IDX(rcr->cursor)); +#ifdef RTE_LIBRTE_DPAA_CHECKING + rcr->busy = 0; +#endif +} + +static inline void bm_rcr_pvb_commit(struct bm_portal *portal, u8 myverb) +{ + register struct bm_rcr *rcr = &portal->rcr; + struct bm_rcr_entry *rcursor; + + DPAA_ASSERT(rcr->busy); + DPAA_ASSERT(rcr->pmode == bm_rcr_pvb); + lwsync(); + rcursor = rcr->cursor; + rcursor->__dont_write_directly__verb = myverb | rcr->vbit; + dcbf_64(rcursor); + RCR_INC(rcr); + rcr->available--; +#ifdef RTE_LIBRTE_DPAA_CHECKING + rcr->busy = 0; +#endif +} + +static inline u8 bm_rcr_cci_update(struct bm_portal *portal) +{ + register struct bm_rcr *rcr = &portal->rcr; + u8 diff, old_ci = rcr->ci; + + DPAA_ASSERT(rcr->cmode == bm_rcr_cci); + rcr->ci = bm_in(RCR_CI_CINH) & (BM_RCR_SIZE - 1); + diff = bm_cyc_diff(BM_RCR_SIZE, old_ci, rcr->ci); + rcr->available += diff; + return diff; +} + +static inline void bm_rcr_cce_prefetch(struct bm_portal *portal) +{ + __maybe_unused register struct bm_rcr *rcr = &portal->rcr; + + DPAA_ASSERT(rcr->cmode == bm_rcr_cce); + bm_cl_touch_ro(RCR_CI); +} + +static inline u8 bm_rcr_cce_update(struct bm_portal *portal) +{ + register struct bm_rcr *rcr = &portal->rcr; + u8 diff, old_ci = rcr->ci; + + DPAA_ASSERT(rcr->cmode == bm_rcr_cce); + rcr->ci = bm_cl_in(RCR_CI) & (BM_RCR_SIZE - 1); + bm_cl_invalidate(RCR_CI); + diff = bm_cyc_diff(BM_RCR_SIZE, old_ci, rcr->ci); + rcr->available += diff; + return diff; +} + +static inline u8 bm_rcr_get_ithresh(struct bm_portal *portal) +{ + register struct bm_rcr *rcr = &portal->rcr; + + return rcr->ithresh; +} + +static inline void bm_rcr_set_ithresh(struct bm_portal *portal, u8 ithresh) +{ + register struct bm_rcr *rcr = &portal->rcr; + + rcr->ithresh = ithresh; + bm_out(RCR_ITR, ithresh); +} + +static inline u8 bm_rcr_get_avail(struct bm_portal *portal) +{ + register struct bm_rcr *rcr = &portal->rcr; + + return rcr->available; +} + +static inline u8 bm_rcr_get_fill(struct bm_portal *portal) +{ + register struct bm_rcr *rcr = &portal->rcr; + + return BM_RCR_SIZE - 1 - rcr->available; +} + +/* --- Management command API --- */ + +static inline int bm_mc_init(struct bm_portal *portal) +{ + register struct bm_mc *mc = &portal->mc; + + mc->cr = portal->addr.ce + BM_CL_CR; + mc->rr = portal->addr.ce + BM_CL_RR0; + mc->rridx = (__raw_readb(&mc->cr->__dont_write_directly__verb) & + BM_MCC_VERB_VBIT) ? 0 : 1; + mc->vbit = mc->rridx ? BM_MCC_VERB_VBIT : 0; +#ifdef RTE_LIBRTE_DPAA_CHECKING + mc->state = mc_idle; +#endif + return 0; +} + +static inline void bm_mc_finish(struct bm_portal *portal) +{ + __maybe_unused register struct bm_mc *mc = &portal->mc; + + DPAA_ASSERT(mc->state == mc_idle); +#ifdef RTE_LIBRTE_DPAA_CHECKING + if (mc->state != mc_idle) + pr_crit("Losing incomplete MC command\n"); +#endif +} + +static inline struct bm_mc_command *bm_mc_start(struct bm_portal *portal) +{ + register struct bm_mc *mc = &portal->mc; + + DPAA_ASSERT(mc->state == mc_idle); +#ifdef RTE_LIBRTE_DPAA_CHECKING + mc->state = mc_user; +#endif + dcbz_64(mc->cr); + return mc->cr; +} + +static inline void bm_mc_abort(struct bm_portal *portal) +{ + __maybe_unused register struct bm_mc *mc = &portal->mc; + + DPAA_ASSERT(mc->state == mc_user); +#ifdef RTE_LIBRTE_DPAA_CHECKING + mc->state = mc_idle; +#endif +} + +static inline void bm_mc_commit(struct bm_portal *portal, u8 myverb) +{ + register struct bm_mc *mc = &portal->mc; + struct bm_mc_result *rr = mc->rr + mc->rridx; + + DPAA_ASSERT(mc->state == mc_user); + lwsync(); + mc->cr->__dont_write_directly__verb = myverb | mc->vbit; + dcbf(mc->cr); + dcbit_ro(rr); +#ifdef RTE_LIBRTE_DPAA_CHECKING + mc->state = mc_hw; +#endif +} + +static inline struct bm_mc_result *bm_mc_result(struct bm_portal *portal) +{ + register struct bm_mc *mc = &portal->mc; + struct bm_mc_result *rr = mc->rr + mc->rridx; + + DPAA_ASSERT(mc->state == mc_hw); + /* The inactive response register's verb byte always returns zero until + * its command is submitted and completed. This includes the valid-bit, + * in case you were wondering. + */ + if (!__raw_readb(&rr->verb)) { + dcbit_ro(rr); + return NULL; + } + mc->rridx ^= 1; + mc->vbit ^= BM_MCC_VERB_VBIT; +#ifdef RTE_LIBRTE_DPAA_CHECKING + mc->state = mc_idle; +#endif + return rr; +} + +#define SCN_REG(bpid) BM_REG_SCN((bpid) / 32) +#define SCN_BIT(bpid) (0x80000000 >> (bpid & 31)) +static inline void bm_isr_bscn_mask(struct bm_portal *portal, u8 bpid, + int enable) +{ + u32 val; + + DPAA_ASSERT(bpid < bman_pool_max); + /* REG_SCN for bpid=0..31, REG_SCN+4 for bpid=32..63 */ + val = __bm_in(&portal->addr, SCN_REG(bpid)); + if (enable) + val |= SCN_BIT(bpid); + else + val &= ~SCN_BIT(bpid); + __bm_out(&portal->addr, SCN_REG(bpid), val); +} + +static inline u32 __bm_isr_read(struct bm_portal *portal, enum bm_isr_reg n) +{ +#if defined(RTE_ARCH_ARM64) + return __bm_in(&portal->addr, BM_REG_ISR + (n << 6)); +#else + return __bm_in(&portal->addr, BM_REG_ISR + (n << 2)); +#endif +} + +static inline void __bm_isr_write(struct bm_portal *portal, enum bm_isr_reg n, + u32 val) +{ +#if defined(RTE_ARCH_ARM64) + __bm_out(&portal->addr, BM_REG_ISR + (n << 6), val); +#else + __bm_out(&portal->addr, BM_REG_ISR + (n << 2), val); +#endif +} + +/* Buffer Pool Cleanup */ +static inline int bm_shutdown_pool(struct bm_portal *p, u32 bpid) +{ + struct bm_mc_command *bm_cmd; + struct bm_mc_result *bm_res; + + int aq_count = 0; + bool stop = false; + + while (!stop) { + /* Acquire buffers until empty */ + bm_cmd = bm_mc_start(p); + bm_cmd->acquire.bpid = bpid; + bm_mc_commit(p, BM_MCC_VERB_CMD_ACQUIRE | 1); + while (!(bm_res = bm_mc_result(p))) + cpu_relax(); + if (!(bm_res->verb & BM_MCR_VERB_ACQUIRE_BUFCOUNT)) { + /* Pool is empty */ + stop = true; + } else + ++aq_count; + }; + return 0; +} + +#endif /* __BMAN_H */ diff --git a/drivers/bus/dpaa/base/qbman/bman_driver.c b/drivers/bus/dpaa/base/qbman/bman_driver.c index fb3c50e..28f2cf2 100644 --- a/drivers/bus/dpaa/base/qbman/bman_driver.c +++ b/drivers/bus/dpaa/base/qbman/bman_driver.c @@ -65,6 +65,7 @@ static __thread struct dpaa_ioctl_portal_map map = { static int fsl_bman_portal_init(uint32_t idx, int is_shared) { cpu_set_t cpuset; + struct bman_portal *portal; int loop, ret; struct dpaa_ioctl_irq_map irq_map; @@ -111,6 +112,14 @@ static int fsl_bman_portal_init(uint32_t idx, int is_shared) /* Use the IRQ FD as a unique IRQ number */ pcfg.irq = fd; + portal = bman_create_affine_portal(&pcfg); + if (!portal) { + pr_err("Bman portal initialisation failed (%d)", + pcfg.cpu); + process_portal_unmap(&map.addr); + return -EBUSY; + } + /* Set the IRQ number */ irq_map.type = dpaa_portal_bman; irq_map.portal_cinh = map.addr.cinh; @@ -120,10 +129,13 @@ static int fsl_bman_portal_init(uint32_t idx, int is_shared) static int fsl_bman_portal_finish(void) { + __maybe_unused const struct bm_portal_config *cfg; int ret; process_portal_irq_unmap(fd); + cfg = bman_destroy_affine_portal(); + BUG_ON(cfg != &pcfg); ret = process_portal_unmap(&map.addr); if (ret) error(0, ret, "process_portal_unmap()"); diff --git a/drivers/bus/dpaa/base/qbman/dpaa_alloc.c b/drivers/bus/dpaa/base/qbman/dpaa_alloc.c index 690576a..35dba7f 100644 --- a/drivers/bus/dpaa/base/qbman/dpaa_alloc.c +++ b/drivers/bus/dpaa/base/qbman/dpaa_alloc.c @@ -41,6 +41,22 @@ #include "dpaa_sys.h" #include #include +#include + +int bman_alloc_bpid_range(u32 *result, u32 count, u32 align, int partial) +{ + return process_alloc(dpaa_id_bpid, result, count, align, partial); +} + +void bman_release_bpid_range(u32 bpid, u32 count) +{ + process_release(dpaa_id_bpid, bpid, count); +} + +int bman_reserve_bpid_range(u32 bpid, u32 count) +{ + return process_reserve(dpaa_id_bpid, bpid, count); +} int qman_alloc_fqid_range(u32 *result, u32 count, u32 align, int partial) {