From patchwork Fri Feb 2 03:04:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chaoyong He X-Patchwork-Id: 136300 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 11DA243A49; Fri, 2 Feb 2024 04:04:55 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 5372E42D28; Fri, 2 Feb 2024 04:04:49 +0100 (CET) Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2094.outbound.protection.outlook.com [40.107.94.94]) by mails.dpdk.org (Postfix) with ESMTP id AC3E440268 for ; Fri, 2 Feb 2024 04:04:47 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=TG2vQgftARknsy0L2DNuCscRHx68nNtqr+ZOzbNJdw1GRZvUb1Ly6E3uQ+AUQ+7BD3YcuThp9CjG4Vlmu0AK4MrTah22zQcCi40PpYpaPMbE1pVIbc8m5Qs3q7S9f8Sf1Fzg2VnlsbCOjnlOr+eo7v32ECU6osD56TVE9N2/5HkbYBJLnxP0OB9z6lCE6SzHr5K0xscizQeCGAO4y+Dpa3XN6zX13r8WDxlrAjErkPqKYK5UD4gSurdVs1i5mlUk5dTDHrFT8a3IzGplfO2EaGd6K06OJlx0/HW4d+EmSw8WYHWMnZitnFcT8wgqMG8QLrJUm7xvVT11OEoF5qE3YQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=EDdidAtLEGr6J7F9yLfLTZ8UHNmJvBfxSXmNem/GJfk=; b=NufAKxM7lqbkp7HANFAEEym/WacMt/wqzGzD+1E6Era5vi3YpjXAL8OtxY/92TsI4Z8pan0DrW0dM8jAVCNeEzks0LNNR/ExpowU2v6n+MwtbylzIrrafAbdgw8Ga9GAMKM6rT7iXcmS1oQ29LIJ+4fwxZsxywdygRnhctwSzCVIUiBIYLpQ4GFL6pmxAySM3l9tYxA1pAhCpXd3qmi/yxVIABSuD/AxtmpLcvJ7N9RhJRoqNXP3DeD/d0PXez4/LHiEUUqREeST6BHhFkQbPei9mVZCaWQVhroRj9+HUJzQ+AJz/wJyXJFzHrTHK/ws2MpPHrIAKGeEAJgfSNU93w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=corigine.com; dmarc=pass action=none header.from=corigine.com; dkim=pass header.d=corigine.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=corigine.onmicrosoft.com; s=selector2-corigine-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=EDdidAtLEGr6J7F9yLfLTZ8UHNmJvBfxSXmNem/GJfk=; b=GE4PQSpyLs4i3lL5SidpadiCH3Gc7//Mv24Xr9XtmXH9W1Eje0T3usQKMCcM5QDfA6Xqk+sKMCI1bJZkFmTWSjlpclxErhOuqZjxQMfqHPBKhs3mNfRHA41ZWtS8MnDQ4zaDFZq9AaQ5o7+oxOe77f0z6PH2e0J1To2wAtdykbU= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=corigine.com; Received: from SJ0PR13MB5545.namprd13.prod.outlook.com (2603:10b6:a03:424::5) by SN7PR13MB6177.namprd13.prod.outlook.com (2603:10b6:806:2e3::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7249.30; Fri, 2 Feb 2024 03:04:45 +0000 Received: from SJ0PR13MB5545.namprd13.prod.outlook.com ([fe80::8e02:f738:570a:f8aa]) by SJ0PR13MB5545.namprd13.prod.outlook.com ([fe80::8e02:f738:570a:f8aa%7]) with mapi id 15.20.7249.024; Fri, 2 Feb 2024 03:04:45 +0000 From: Chaoyong He To: dev@dpdk.org Cc: oss-drivers@corigine.com, Long Wu , Chaoyong He , Peng Zhang Subject: [PATCH v2 1/3] net/nfp: add synchronize module Date: Fri, 2 Feb 2024 11:04:13 +0800 Message-Id: <20240202030415.739638-2-chaoyong.he@corigine.com> X-Mailer: git-send-email 2.39.1 In-Reply-To: <20240202030415.739638-1-chaoyong.he@corigine.com> References: <20240124092541.2011874-1-chaoyong.he@corigine.com> <20240202030415.739638-1-chaoyong.he@corigine.com> X-ClientProxiedBy: BYAPR05CA0004.namprd05.prod.outlook.com (2603:10b6:a03:c0::17) To SJ0PR13MB5545.namprd13.prod.outlook.com (2603:10b6:a03:424::5) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SJ0PR13MB5545:EE_|SN7PR13MB6177:EE_ X-MS-Office365-Filtering-Correlation-Id: 59109f1b-f34a-4c25-17b0-08dc239bb5af X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: AT/Y5NU1wTNcrAggBH9SbPP5uMKzUq1SYaDoKv8AL2g2d85wRofsNQkHkynxEPmIZ+KrQwe4hSs5eszAr5lPp7BZ0WjJfUkH0nIYDEcFHN0OeavhLg0K1a5pyZAi9PrGWkBHcFUUGj6ufEV3XyCf3G5jJR6t6aYXxzFpAqSCPCMKap7zmqGxPQzFQ1nx+bLLdydiINdCg+icnCiC0/gYppjFpbLt4/5kB6KyqONm8fmCYcbzgUfXMgIQXb4/d8SMtudAX9WDIRhhvp2xyz/lemgxkKD9TcaNGGpRJXtcR8UZsZQZvWPXZt3VyPbVimfJg6pTqFUMPtNN7ASL7AMRRsuJxLzUXtHADH/GuzGMLY7cm74zxnNBZ9YshfsJjFr4wFrsTv8So53Z97FPW2+frXQ2EO+q+mXp8nxxJcn/DbDNbeuTbWWPRiLMbzTB177787/szReXA99faMa1W0DhWUm+F6m+VJb4jDaHtLiXYoN/oTxMWO8Jker+ffSIDsgbBpdxeZgiHpOpzxFhn17poBIHyXrFwUnxjtUEntidHbTPp8ci/9LidAfkaDY9dmtKBn93yF/XYsl1RHNNVk01nKItLUE19XcBaW+/do/V4PxWv84UZNrwqm8raV1NkVpo X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SJ0PR13MB5545.namprd13.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230031)(366004)(136003)(376002)(396003)(39840400004)(346002)(230922051799003)(451199024)(186009)(64100799003)(1800799012)(41300700001)(1076003)(107886003)(26005)(2616005)(6916009)(38350700005)(316002)(36756003)(478600001)(6512007)(6486002)(83380400001)(52116002)(6506007)(6666004)(38100700002)(86362001)(2906002)(5660300002)(30864003)(66946007)(54906003)(66476007)(66556008)(8936002)(44832011)(8676002)(4326008); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: ssT5kUUZZOA7DfuvcYumF1KfX9nf9DnQ2T32FclX+FgdB9PvmhTo2karT1w5DzGknUGmOJCMA9yiLeWzcvN5NyIOxWM3vsL75i9jQgjb9aie4CiIEXXK9ctSAmz/GmDit8uGFRp94xwsx4AyPDhGq+TcFfNntMJC0RBZ/7NvjsmRZYcWgoki5sF+HXseveHRhCLoV6qmdTSwIqjJbr2/H4kH8KWfjbry9ynrQW+ly80Gdq08x6Xi9gGKNrwXg8rWS/A1vwPOFlRhQw/IrPibkzsn7ErvyHhbhZe0wI5G5zxwc7VSlxuQHce6iAFT7ka3FMSd08R3eVWhSzVjvDMngUUBsbniMP+PvMwEEjlyDRt2Zr2prmghs50+O2m1UYFCpig/npPAYYQSbkzbNemTPqoXvYUJGeNc1SlAKvAF10+OSGtoDSuorPAD+OS7/9wmnjkdzpjakKc3NCJyQQtElLIKeGnQYWcl8hpogdkLc61ZXwFH5KGb4nEudCU7X8uO6He6Sp7WlKkL0NZ02fa62jGYWZXIZVK/ioat2n2Q92Q9wc5l52nBL7rAVMUoF03+ChPDzFc1UV9Q8Ma+2aR8NumAM3wlNvLygDTT1NlHSlnCXpE5eL1A1VV18gMSFJaqrmiIvAFASd4A1EbsBn7Rix7lYZpNMgTHaA2tUx5HZQ5yYhhDurNav3wZ9AMKFYU7l0esBezAcXVc8dPC+ev5gFzzAXqgT2V+aeMHGmprW48f6yGyQJgSH2+5YdmrfYNKFg13HFdtpEtSujLmWkuRqGqjHz7oggPaiA/qX2g3PsvLEaXp9hKy/5fZrsceVO97d1AD0+yrrwziac8ZywZqTOfjR+MR7eySNlMF7lXhvxsX7NGoZE2G/FXlxmozcuaPs2uVISNkOHXVL4ptohfOEcTqf/KqnxW2EOMMD0AKID22Zh1PcOuHhAfPh5PTZLXn0W5SmiiD02CSUMHHRN9t6n4YPacvi2/G2kzhoT/ZiW170gOKRHqbWSjt5Tf6/y4VFYwmZYQG1LQA0JmA6c6r8cx0RQcXD+kFjsyQ5Ud5Ebbrcu+32RU3rKBBRB2UlFsGnghfqdkWhlDgqOMY8qQTkWyMQ8oobwkRas2+YTuVzTXreckJ1/Wab6Sv0eeugAFrCZkfqjAel2t4vAm6t0ZfxE0GlK0MMI0SY09yIrnWWVOShnPUke+5Zgn+nIoUouiM2WCd8CIDLRwC40U3FdUHpis1rCMBBoKYdaz/lfaZbk8cyzp9ndDKGBe99vAWDc9uBoDomAIwaKv+ZjYngH1LJuKK/7vc57/WVBZZ/FV0x0jvj7qKXNaLdE8/iwaeCcTGd7DfsGN6hvgc8Sz0doK87oKDL8wn+aHbJ4RkTkd9/o8cFktXjkn9cex61OGNxgigFJLdwXzCIqK7eOXQ56hiA7Bek5XB2K/U4FgowgYTjPFRijYW79llfgl4h/8XdgDgltuiHhV5s1pQZi1nRb5Jt0s/yfE+tOKT/QnlOe3gi9b2ZcEkO5ZnV8SwIX7ffwKOXtU30EVVJrbUw9tFTUU2rkKDACopJPAr1kr7sZ9S3Adeb3HonEDG+VWZT1eUrKMEjdUFHIb5VF4HzY4s1AUx0g== X-OriginatorOrg: corigine.com X-MS-Exchange-CrossTenant-Network-Message-Id: 59109f1b-f34a-4c25-17b0-08dc239bb5af X-MS-Exchange-CrossTenant-AuthSource: SJ0PR13MB5545.namprd13.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Feb 2024 03:04:45.7444 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: fe128f2c-073b-4c20-818e-7246a585940c X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 8bwwP5mK6wlldgVMh6yc9RWBwaudZCvObq7XULiJDXloh/Yysmm8uqkDm0eTPJld7wgjgjRXulJc6asM0ugbY4dzrFENhyY0LGEpeC54EeU= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN7PR13MB6177 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Long Wu PMD has some information that needs synchronized: 1. Between devices in one process. 2. Between multiple processes for one device, before creating "rte_eth_device". 3. Between multiple processes for one device, after creating "rte_eth_device". The final one is already support by framework (eth_dev->data->dev_private), and this patch aims to support the first two cases by adding a synchronize module. Signed-off-by: Long Wu Reviewed-by: Chaoyong He Reviewed-by: Peng Zhang --- drivers/net/nfp/meson.build | 1 + drivers/net/nfp/nfp_ethdev.c | 31 ++- drivers/net/nfp/nfp_net_common.h | 4 + drivers/net/nfp/nfpcore/nfp_sync.c | 382 +++++++++++++++++++++++++++++ drivers/net/nfp/nfpcore/nfp_sync.h | 29 +++ 5 files changed, 444 insertions(+), 3 deletions(-) create mode 100644 drivers/net/nfp/nfpcore/nfp_sync.c create mode 100644 drivers/net/nfp/nfpcore/nfp_sync.h diff --git a/drivers/net/nfp/meson.build b/drivers/net/nfp/meson.build index 46be6f60cd..c625355d7a 100644 --- a/drivers/net/nfp/meson.build +++ b/drivers/net/nfp/meson.build @@ -26,6 +26,7 @@ sources = files( 'nfpcore/nfp_nsp_eth.c', 'nfpcore/nfp_resource.c', 'nfpcore/nfp_rtsym.c', + 'nfpcore/nfp_sync.c', 'nfpcore/nfp_target.c', 'nfpcore/nfp6000_pcie.c', 'nfp_cpp_bridge.c', diff --git a/drivers/net/nfp/nfp_ethdev.c b/drivers/net/nfp/nfp_ethdev.c index 07cd32881c..3ece806487 100644 --- a/drivers/net/nfp/nfp_ethdev.c +++ b/drivers/net/nfp/nfp_ethdev.c @@ -19,6 +19,7 @@ #include "nfpcore/nfp_nsp.h" #include "nfpcore/nfp6000_pcie.h" #include "nfpcore/nfp_resource.h" +#include "nfpcore/nfp_sync.h" #include "nfp_cpp_bridge.h" #include "nfp_ipsec.h" @@ -551,6 +552,7 @@ nfp_pf_uninit(struct nfp_pf_dev *pf_dev) free(pf_dev->nfp_eth_table); free(pf_dev->hwinfo); nfp_cpp_free(pf_dev->cpp); + nfp_sync_free(pf_dev->sync); rte_free(pf_dev); } @@ -559,6 +561,7 @@ nfp_pf_secondary_uninit(struct nfp_pf_dev *pf_dev) { free(pf_dev->sym_tbl); nfp_cpp_free(pf_dev->cpp); + nfp_sync_free(pf_dev->sync); rte_free(pf_dev); return 0; @@ -1612,6 +1615,7 @@ nfp_net_speed_capa_get(struct nfp_pf_dev *pf_dev, static int nfp_pf_init(struct rte_pci_device *pci_dev) { + void *sync; uint32_t i; uint32_t id; int ret = 0; @@ -1652,6 +1656,13 @@ nfp_pf_init(struct rte_pci_device *pci_dev) return -ENOMEM; } + sync = nfp_sync_alloc(); + if (sync == NULL) { + PMD_INIT_LOG(ERR, "Failed to alloc sync zone."); + ret = -ENOMEM; + goto pf_cleanup; + } + /* * When device bound to UIO, the device could be used, by mistake, * by two DPDK apps, and the UIO driver does not avoid it. This @@ -1667,7 +1678,7 @@ nfp_pf_init(struct rte_pci_device *pci_dev) if (cpp == NULL) { PMD_INIT_LOG(ERR, "A CPP handle can not be obtained"); ret = -EIO; - goto pf_cleanup; + goto sync_free; } hwinfo = nfp_hwinfo_read(cpp); @@ -1734,6 +1745,7 @@ nfp_pf_init(struct rte_pci_device *pci_dev) pf_dev->sym_tbl = sym_tbl; pf_dev->pci_dev = pci_dev; pf_dev->nfp_eth_table = nfp_eth_table; + pf_dev->sync = sync; /* Get the speed capability */ for (i = 0; i < nfp_eth_table->count; i++) { @@ -1815,6 +1827,8 @@ nfp_pf_init(struct rte_pci_device *pci_dev) free(hwinfo); cpp_cleanup: nfp_cpp_free(cpp); +sync_free: + nfp_sync_free(sync); pf_cleanup: rte_free(pf_dev); @@ -1878,6 +1892,7 @@ nfp_secondary_init_app_fw_nic(struct nfp_pf_dev *pf_dev) static int nfp_pf_secondary_init(struct rte_pci_device *pci_dev) { + void *sync; int ret = 0; struct nfp_cpp *cpp; uint8_t function_id; @@ -1910,6 +1925,13 @@ nfp_pf_secondary_init(struct rte_pci_device *pci_dev) return -ENOMEM; } + sync = nfp_sync_alloc(); + if (sync == NULL) { + PMD_INIT_LOG(ERR, "Failed to alloc sync zone."); + ret = -ENOMEM; + goto pf_cleanup; + } + /* * When device bound to UIO, the device could be used, by mistake, * by two DPDK apps, and the UIO driver does not avoid it. This @@ -1925,7 +1947,7 @@ nfp_pf_secondary_init(struct rte_pci_device *pci_dev) if (cpp == NULL) { PMD_INIT_LOG(ERR, "A CPP handle can not be obtained"); ret = -EIO; - goto pf_cleanup; + goto sync_free; } /* @@ -1936,7 +1958,7 @@ nfp_pf_secondary_init(struct rte_pci_device *pci_dev) if (sym_tbl == NULL) { PMD_INIT_LOG(ERR, "Something is wrong with the firmware symbol table"); ret = -EIO; - goto pf_cleanup; + goto sync_free; } /* Read the app ID of the firmware loaded */ @@ -1954,6 +1976,7 @@ nfp_pf_secondary_init(struct rte_pci_device *pci_dev) pf_dev->cpp = cpp; pf_dev->sym_tbl = sym_tbl; pf_dev->pci_dev = pci_dev; + pf_dev->sync = sync; /* Call app specific init code now */ switch (app_fw_id) { @@ -1983,6 +2006,8 @@ nfp_pf_secondary_init(struct rte_pci_device *pci_dev) sym_tbl_cleanup: free(sym_tbl); +sync_free: + nfp_sync_free(sync); pf_cleanup: rte_free(pf_dev); diff --git a/drivers/net/nfp/nfp_net_common.h b/drivers/net/nfp/nfp_net_common.h index 5648ad32d8..3a441f49f6 100644 --- a/drivers/net/nfp/nfp_net_common.h +++ b/drivers/net/nfp/nfp_net_common.h @@ -12,6 +12,7 @@ #include #include +#include "nfpcore/nfp_sync.h" #include "nfp_net_ctrl.h" /* Interrupt definitions */ @@ -109,6 +110,9 @@ struct nfp_pf_dev { /** Supported speeds bitmap */ uint32_t speed_capa; + + /** Synchronized info */ + struct nfp_sync *sync; }; #define NFP_NET_FLOW_LIMIT 1024 diff --git a/drivers/net/nfp/nfpcore/nfp_sync.c b/drivers/net/nfp/nfpcore/nfp_sync.c new file mode 100644 index 0000000000..8bc38d6585 --- /dev/null +++ b/drivers/net/nfp/nfpcore/nfp_sync.c @@ -0,0 +1,382 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (c) 2024 Corigine, Inc. + * All rights reserved. + */ + +#include "nfp_sync.h" + +#include +#include +#include +#include + +#include "nfp_logs.h" + +#define NFP_SYNC_ELEMENT_MAX 8 +#define NFP_SYNC_PCI_MAX 32 + +struct nfp_sync_element { + uint16_t count; + /** Element ID, use ASCII - SYN<> */ + uint32_t magic; + void *handle; +}; + +struct nfp_sync_common { + char pci_name[PCI_PRI_STR_SIZE + 1]; + uint16_t avail; + struct nfp_sync_element element[NFP_SYNC_ELEMENT_MAX]; +}; + +struct nfp_sync { + rte_spinlock_t spinlock; + + uint16_t alloc_count; + + struct nfp_sync_common process; + + struct nfp_sync_common pci[NFP_SYNC_PCI_MAX]; + + const struct rte_memzone *mz; +}; + +struct nfp_sync * +nfp_sync_alloc(void) +{ + uint16_t i; + struct nfp_sync *sync; + const struct rte_memzone *mz; + + mz = rte_memzone_lookup("nfp_sync"); + if (mz != NULL) { + sync = mz->addr; + sync->alloc_count++; + + return sync; + } + + mz = rte_memzone_reserve("nfp_sync", sizeof(*sync), SOCKET_ID_ANY, + RTE_MEMZONE_SIZE_HINT_ONLY); + if (mz == NULL) + return NULL; + + sync = mz->addr; + + memset(sync, 0, sizeof(*sync)); + + rte_spinlock_init(&sync->spinlock); + sync->alloc_count = 1; + sync->mz = mz; + + for (i = 0; i < NFP_SYNC_PCI_MAX; i++) + sync->pci[i].avail = NFP_SYNC_ELEMENT_MAX; + + sync->process.avail = NFP_SYNC_ELEMENT_MAX; + + return sync; +} + +void +nfp_sync_free(struct nfp_sync *sync) +{ + uint16_t i; + + rte_spinlock_lock(&sync->spinlock); + + sync->alloc_count--; + if (sync->alloc_count != 0) { + rte_spinlock_unlock(&sync->spinlock); + return; + } + + if (sync->process.avail != NFP_SYNC_ELEMENT_MAX) + PMD_DRV_LOG(ERR, "Sync process handle residue"); + + for (i = 0; i < NFP_SYNC_PCI_MAX; i++) { + if (sync->pci[i].avail != NFP_SYNC_ELEMENT_MAX) + PMD_DRV_LOG(ERR, "Sync %s pci handle residue", + sync->pci[i].pci_name); + } + + rte_spinlock_unlock(&sync->spinlock); + + rte_memzone_free(sync->mz); +} + +static void * +nfp_sync_element_alloc(struct nfp_sync_element *element, + uint32_t magic, + uint32_t size) +{ + void *handle; + + handle = rte_zmalloc(NULL, size, 0); + if (handle == NULL) + return NULL; + + element->handle = handle; + element->count = 1; + element->magic = magic; + + return handle; +} + +static void +nfp_sync_element_free(struct nfp_sync_element *element, + void *handle) +{ + element->count--; + if (element->count != 0) + return; + + rte_free(handle); + element->handle = NULL; + element->magic = 0; +} + +static void * +nfp_sync_common_handle_alloc(struct nfp_sync_common *common, + uint32_t magic, + uint32_t size) +{ + uint16_t i; + void *handle = NULL; + uint16_t avail_slot = NFP_SYNC_ELEMENT_MAX; + + for (i = 0; i < NFP_SYNC_ELEMENT_MAX; i++) { + if (common->element[i].magic != magic) + continue; + + common->element[i].count++; + + return common->element[i].handle; + } + + if (common->avail == 0) + return NULL; + + for (i = 0; i < NFP_SYNC_ELEMENT_MAX; i++) { + if (common->element[i].magic == 0) { + avail_slot = i; + break; + } + } + + handle = nfp_sync_element_alloc(&common->element[avail_slot], magic, size); + if (handle == NULL) + return NULL; + + common->avail--; + + return handle; +} + +static void +nfp_sync_common_handle_free(struct nfp_sync_common *common, + void *handle) +{ + uint16_t i; + + if (common->avail == NFP_SYNC_ELEMENT_MAX) + return; + + for (i = 0; i < NFP_SYNC_ELEMENT_MAX; i++) { + if (common->element[i].handle == handle) + break; + } + + if (i == NFP_SYNC_ELEMENT_MAX) + return; + + nfp_sync_element_free(&common->element[i], handle); + + if (common->element[i].count == 0) + common->avail++; +} + +static void * +nfp_sync_process_inner_handle_alloc(struct nfp_sync *sync, + uint32_t magic, + uint32_t size) +{ + void *handle = NULL; + + rte_spinlock_lock(&sync->spinlock); + + handle = nfp_sync_common_handle_alloc(&sync->process, magic, size); + if (handle == NULL) + PMD_DRV_LOG(ERR, "Process handle alloc failed"); + + rte_spinlock_unlock(&sync->spinlock); + + return handle; +} + +static void +nfp_sync_process_inner_handle_free(struct nfp_sync *sync, + void *handle) +{ + rte_spinlock_lock(&sync->spinlock); + + nfp_sync_common_handle_free(&sync->process, handle); + + rte_spinlock_unlock(&sync->spinlock); +} + +static uint16_t +nfp_sync_process_handle_count_get(struct nfp_sync *sync, + void *handle) +{ + uint16_t i; + uint16_t count = 0; + + rte_spinlock_lock(&sync->spinlock); + + for (i = 0; i < NFP_SYNC_ELEMENT_MAX; i++) { + if (sync->process.element[i].handle == handle) { + count = sync->process.element[i].count; + break; + } + } + + rte_spinlock_unlock(&sync->spinlock); + + return count; +} + +static void * +nfp_sync_pci_inner_handle_alloc(struct nfp_sync *sync, + const char *pci_name, + uint32_t magic, + uint32_t size) +{ + uint16_t i; + void *handle = NULL; + uint16_t pci_avail_id = NFP_SYNC_PCI_MAX; + + rte_spinlock_lock(&sync->spinlock); + + for (i = 0; i < NFP_SYNC_PCI_MAX; i++) { + if (strcmp(pci_name, sync->pci[i].pci_name) == 0) { + pci_avail_id = i; + goto common_alloc; + } + } + + for (i = 0; i < NFP_SYNC_PCI_MAX; i++) { + if (strlen(sync->pci[i].pci_name) == 0) { + pci_avail_id = i; + strcpy(sync->pci[pci_avail_id].pci_name, pci_name); + goto common_alloc; + } + } + + rte_spinlock_unlock(&sync->spinlock); + + return NULL; + +common_alloc: + handle = nfp_sync_common_handle_alloc(&sync->pci[pci_avail_id], + magic, size); + if (handle == NULL) + PMD_DRV_LOG(ERR, "PCI handle alloc failed"); + + rte_spinlock_unlock(&sync->spinlock); + + return handle; +} + +static void +nfp_sync_pci_inner_handle_free(struct nfp_sync *sync, + const char *pci_name, + void *handle) +{ + uint16_t i; + char *name_tmp; + + rte_spinlock_lock(&sync->spinlock); + + for (i = 0; i < NFP_SYNC_PCI_MAX; i++) { + name_tmp = sync->pci[i].pci_name; + if (strlen(name_tmp) != 0 && strcmp(pci_name, name_tmp) == 0) { + nfp_sync_common_handle_free(&sync->pci[i], handle); + if (sync->pci[i].avail == NFP_SYNC_ELEMENT_MAX) + name_tmp[0] = 0; + break; + } + } + + rte_spinlock_unlock(&sync->spinlock); +} + +static uint16_t +nfp_sync_pci_handle_count_get(struct nfp_sync *sync, + const char *pci_name, + void *handle) +{ + uint16_t i; + uint16_t count = 0; + struct nfp_sync_common *pci_common; + + rte_spinlock_lock(&sync->spinlock); + + for (i = 0; i < NFP_SYNC_PCI_MAX; i++) { + if (strcmp(sync->pci[i].pci_name, pci_name) == 0) + break; + } + + if (i == NFP_SYNC_PCI_MAX) { + rte_spinlock_unlock(&sync->spinlock); + return 0; + } + + pci_common = &sync->pci[i]; + + for (i = 0; i < NFP_SYNC_ELEMENT_MAX; i++) { + if (pci_common->element[i].handle == handle) { + count = pci_common->element[i].count; + break; + } + } + + rte_spinlock_unlock(&sync->spinlock); + + return count; +} + +void * +nfp_sync_handle_alloc(struct nfp_sync *sync, + struct rte_pci_device *pci_dev, + uint32_t magic, + uint32_t size) +{ + if (pci_dev == NULL) + return nfp_sync_process_inner_handle_alloc(sync, magic, size); + + return nfp_sync_pci_inner_handle_alloc(sync, pci_dev->device.name, + magic, size); +} + +void +nfp_sync_handle_free(struct nfp_sync *sync, + struct rte_pci_device *pci_dev, + void *handle) +{ + if (pci_dev == NULL) { + nfp_sync_process_inner_handle_free(sync, handle); + return; + } + + nfp_sync_pci_inner_handle_free(sync, pci_dev->device.name, handle); +} + +uint16_t +nfp_sync_handle_count_get(struct nfp_sync *sync, + struct rte_pci_device *pci_dev, + void *handle) +{ + if (pci_dev == NULL) + return nfp_sync_process_handle_count_get(sync, handle); + + return nfp_sync_pci_handle_count_get(sync, pci_dev->device.name, handle); +} diff --git a/drivers/net/nfp/nfpcore/nfp_sync.h b/drivers/net/nfp/nfpcore/nfp_sync.h new file mode 100644 index 0000000000..82f01e2652 --- /dev/null +++ b/drivers/net/nfp/nfpcore/nfp_sync.h @@ -0,0 +1,29 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (c) 2024 Corigine, Inc. + * All rights reserved. + */ + +#ifndef __NFP_SYNC_H__ +#define __NFP_SYNC_H__ + +#include + +#include + +struct nfp_sync; + +struct nfp_sync *nfp_sync_alloc(void); +void nfp_sync_free(struct nfp_sync *sync); + +void *nfp_sync_handle_alloc(struct nfp_sync *sync, + struct rte_pci_device *pci_dev, + uint32_t magic, + uint32_t size); +void nfp_sync_handle_free(struct nfp_sync *sync, + struct rte_pci_device *pci_dev, + void *handle); +uint16_t nfp_sync_handle_count_get(struct nfp_sync *sync, + struct rte_pci_device *pci_dev, + void *handle); + +#endif /* __NFP_SYNC_H__ */ From patchwork Fri Feb 2 03:04:14 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chaoyong He X-Patchwork-Id: 136301 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 8968543A49; Fri, 2 Feb 2024 04:05:04 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id E1CBE42E04; Fri, 2 Feb 2024 04:04:51 +0100 (CET) Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2100.outbound.protection.outlook.com [40.107.94.100]) by mails.dpdk.org (Postfix) with ESMTP id 89A4F42DB0 for ; Fri, 2 Feb 2024 04:04:49 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=JDFirxTPLoN7TQTK2DjyOorPGziv/QGoGhhmfEC3+65c1KU2Yjx/8bSyokr0JR3E5wvU6YBrYOSqMK20YR2Q8oruLP2L2uHtVF08+hYns1MM2h2EFOeUJnrlMxJEPgE3GkohBJ09zuzQwZ7rb9aTkKBuFXaX4RbbBlLWjdz5faTV1F3gYMREZnLELb2UzF3nJ4alWFbO5O/uXmhFRU3G/XFQB/20n7ZfBH9mu8jhg7vYRNjcHtnQ3hvJDAgeXoUFW9V8Co5+55RsZGyQhJLXW6SLNHuP/EywBs3/wCHe92eXiI7R4fFZ31e5te6txhID1I+j0DZ4DLwQauUj9TA6tQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=mBymKcu233OT634MNGzU/vumLhHr+uy6wWgT2qVo48g=; b=Eao5W6+YWRMKRVhKvQkndCKuMpPSGbwx4G1Iy50uBa3PujX04l1VwqGxhsq43SfBHq2q7E2pvkJcBsmdO76uokgT8+p2fP6ycfYv0Gw3g+BbPo57p/hrpdZZ0LUOWsekSHjplgGdnCATfFslHLJig6XIi5hdj1EzmrGwBw64lLxWB6EvmFOBSGUMsrKFe9Lo0Tx8i571fmJaSoGxthasjsYGIZOgkzgFIFQ7A8+PZDiSG5nh85vZI5nrI+Q3B64R/m7HZGZgRN55rVUKKMdIsl5UwL47C6ANVRcQR6UvQnUbO9xaW//4iShJqGe+JRNnzlTxs5AdCWBcKYVU2k3+eQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=corigine.com; dmarc=pass action=none header.from=corigine.com; dkim=pass header.d=corigine.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=corigine.onmicrosoft.com; s=selector2-corigine-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=mBymKcu233OT634MNGzU/vumLhHr+uy6wWgT2qVo48g=; b=FcoQ/waqbRHri5jqZ7Mk/1g+Ge3dISyjGDrByiYoRWyayg+sV+AFUCDLsz4oFenBOzxaTp0RnGJIH6Ww9FIwlDZq8rxXFWNYPDgrjwCid2ouL94f4Y+P0Mz9JusYx2PYPz//gKgjZjqqJeF0U/rNVpmWg7eyVyyOAuVJHP0rSjk= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=corigine.com; Received: from SJ0PR13MB5545.namprd13.prod.outlook.com (2603:10b6:a03:424::5) by SN7PR13MB6177.namprd13.prod.outlook.com (2603:10b6:806:2e3::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7249.30; Fri, 2 Feb 2024 03:04:47 +0000 Received: from SJ0PR13MB5545.namprd13.prod.outlook.com ([fe80::8e02:f738:570a:f8aa]) by SJ0PR13MB5545.namprd13.prod.outlook.com ([fe80::8e02:f738:570a:f8aa%7]) with mapi id 15.20.7249.024; Fri, 2 Feb 2024 03:04:47 +0000 From: Chaoyong He To: dev@dpdk.org Cc: oss-drivers@corigine.com, Long Wu , Chaoyong He , Peng Zhang Subject: [PATCH v2 2/3] net/nfp: create new service code related files Date: Fri, 2 Feb 2024 11:04:14 +0800 Message-Id: <20240202030415.739638-3-chaoyong.he@corigine.com> X-Mailer: git-send-email 2.39.1 In-Reply-To: <20240202030415.739638-1-chaoyong.he@corigine.com> References: <20240124092541.2011874-1-chaoyong.he@corigine.com> <20240202030415.739638-1-chaoyong.he@corigine.com> X-ClientProxiedBy: BYAPR05CA0004.namprd05.prod.outlook.com (2603:10b6:a03:c0::17) To SJ0PR13MB5545.namprd13.prod.outlook.com (2603:10b6:a03:424::5) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SJ0PR13MB5545:EE_|SN7PR13MB6177:EE_ X-MS-Office365-Filtering-Correlation-Id: 03913fe7-bc67-4fec-71fb-08dc239bb6c0 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 6UuVRyPLjiq8XvPYt2e1WQxNuq5UZwLvWnUxSWf+dPnDO/mNfIKRA5TkTVUPUmpCzrABQfUwj3NJvasjcuq53mHhbFq+5wxb63IdXi70MMwbXPhEGyislA95LToqjjv0m63NGgubDbkb0oVDK4z79hqJ2MusXxUvEND+vMJyjLHCVTHjq4fyByvwhxGokWIx7A+/ZZ3A3MfR8UiLZ+pQDGA2Owfco9HreayMGJCeeepybiFgQvwTk4Ni89tIlrsWvSeqGhaMD9NStK9WUnU3TioH794xptwbIZRe65yAn0cAla7558ACrP8pWqKodyCRCyu2Ati7PxKI0w6v45/9EmlkhXqksdj04clRqbPTW+bGxtualRec3sdS4JodIXCpAYOx6JA4ZD9mu18XINNlPsP6VW5R2L6acDyvrVhAYDpLLV2vqLuRieRUja6xeUsgO1bKd4o9WLXygq28g0G1z/gNOup18yCbblOI9j/9ebW3ZsmYlcHYHIZt8TlinLYL1goN2uA+8VsjMgC0lwXPshSIKj5FjvZLTpPwD1pgOxkEypBG9o73O6V32FiwvjYTuZlISkyDcddAtGIt+Q44YiDSZD7DZlSbWCVezzjR9NhmvEjzKxQlTV7VkqbeVDYTpAsYL/2XiUScNYU3RaOC2gZArDnObQYQOuyPr8Z1LFA= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SJ0PR13MB5545.namprd13.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230031)(366004)(136003)(376002)(396003)(39840400004)(346002)(230273577357003)(230173577357003)(230922051799003)(451199024)(186009)(64100799003)(1800799012)(41300700001)(1076003)(107886003)(26005)(2616005)(6916009)(38350700005)(316002)(36756003)(478600001)(6512007)(6486002)(83380400001)(52116002)(6506007)(6666004)(38100700002)(86362001)(2906002)(5660300002)(30864003)(66946007)(54906003)(66476007)(66556008)(8936002)(44832011)(8676002)(4326008); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: oNxCH+opJrSoWPCm0kcPX68E//cjS/lbcI1lHsFYEnP9xBoYpLaCIENMSN19l22zu6OoOZ5Wz0xbSsOxeYb34IYtXLMUvGxtt3SRuBkjECtt6prvFBcp/MCFcfPZOTN4MpN6xoqYMzeVWRQNEmqVBnS0689c5622Z39FS5vxVrHlP6n96iiQ1jO0Cc60trsRq91UaopU2o50KIlQGDXYk9T8dSvZYgl9oARhLpybICREfP2ySQA1QkzHSB1gsNZ+61hFdgKL0G+9V0LmPNzO5wqVmlBcZWRw1C+KQfiXkLx/ZMmf1Q9/Su+J9Bqq9xVEgPLyPNUFMl7h2/wr2IxQsOVcQKIC8pHHSeNYQITc6DwEQNUAfqN/56LzRUa2McBDRLgEhP8OeP4CsmT5A5uA96zOpxqAjkHa6dHI8zzO5S673K+/ZPhrgQroeJEEZVpvGtjcoGs4CJlP9DNiGreRHTgeOfqxo4UcNVAPvuplkmo7iiZbPT4NlsP5kCDseHOW7k2OZOKgSZqImiu+2AqqqpqTTY+uAWglzr0bGPvYfMP3zrmrI8SCUvrryxbjZskU37zqbog/oOT014R2xH9Zy40FryPVdzIEeRy0aidN0U0pXKBQO8ib9ncxV9ssfq/rkEsEnFps1A/XY5oCXVvjO3d3kepcv7XByuJnHkDLhLU6nqQFo/TfkzD74/rf8lsR9yWBG628hQxGeH/IG2SnxadfkJWxJvQ2bSi3nMrAVhH36XgD/8MyhnBtF1MDG/jYz+JseXtBWOcmRFJDsluubic/KA1lvkg273aJzTAXT2AFz/yApgWnstjXk2J8e6g8owNfOG+uVKmdw8v+O3SQ0yTkWoPM4f+OD/caY95tUbxWt0SBJhKiVtXRp02+zktqEl01p9tfePYxMyTZE50uZL1zxRx2bsrmnJ8rnRPd+ct/5oQOMzEPHCpuSXJIUKTpSvwmFNHB5YOxiRjkwxJ+ISZoiF3sNRHRxkgYot1UkKcsaY2FotFCgXQWOE/OT/PcrfYrjlgKoqy95yYFiLQjQKHSJ1gCqy1aTjC5DXoMBP1Zwg/T5kNqf87bUYU94BJWhvGost0iqvvtzoWp4+Tk7rgbRvrHTaDdXOjXXzkKSVoGmMuqSD0FT5D6aYDCHLxswg7ws77cZZ+0YAfaSPzQYPwX+4dgYbn31FkvaB6CAwc8PV+Ndw3yN8UTE/+az0qgAvPIi7gYlJMvnsqCZ0WXu3Dip4Z3FUcut7bjkvRVLFqDEExvRgNv7D9lYetO3NTI/jKl34wpOO+Z7lOvP+qPlyVr3uN8GOrsTcbi1NTFjB92isXegLZn8d1AzQYjqgJLH1gHkJyj+AqqFvLb6Uax/woge0dhVevhKk1cnlOKZa/25P9OurYIZW380/4HHrmDMKcW+MERDpE62VbiAGysM7vAomTf4hff7XaH7XUiZ4cvB+nsjh6PQ1WocnRtuhzlBk5kI8/S5TVuqIZzhor+mcsq4XwqpzI0C6e1M4ZERTItFWv9G9o4B+FWtwGwpBZOlIQr4YfrEPbEoMHziAHdnm9RyQ6mFvkqOgBEIWfZwQe49SWgwnBTyleP5oi3Jc76kRaFUKqoZK1SPQnn0OS8Hg== X-OriginatorOrg: corigine.com X-MS-Exchange-CrossTenant-Network-Message-Id: 03913fe7-bc67-4fec-71fb-08dc239bb6c0 X-MS-Exchange-CrossTenant-AuthSource: SJ0PR13MB5545.namprd13.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Feb 2024 03:04:47.5218 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: fe128f2c-073b-4c20-818e-7246a585940c X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 0l7kzKvzCb0kh7omwdG5a+CbcJWM2tPw6UreRUPCUou2J8RGs5F+GN2fgtaQiShhNhm+bsXs/zrthMH0+iji7Xo5giMV6JLquj0uDCd1aT8= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN7PR13MB6177 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Long Wu For clearer service code structure and more convenient future addition of service code, NFP creates new service code related files and move service related code into new files. This commit also adds service disable interface and lets CPP service run normally if app uses several flower cards. Signed-off-by: Long Wu Reviewed-by: Chaoyong He Reviewed-by: Peng Zhang --- drivers/net/nfp/flower/nfp_flower.c | 19 +-- .../net/nfp/flower/nfp_flower_representor.c | 3 + drivers/net/nfp/meson.build | 1 + drivers/net/nfp/nfp_cpp_bridge.c | 91 +++----------- drivers/net/nfp/nfp_cpp_bridge.h | 1 - drivers/net/nfp/nfp_net_common.h | 5 +- drivers/net/nfp/nfp_service.c | 117 ++++++++++++++++++ drivers/net/nfp/nfp_service.h | 20 +++ 8 files changed, 167 insertions(+), 90 deletions(-) create mode 100644 drivers/net/nfp/nfp_service.c create mode 100644 drivers/net/nfp/nfp_service.h diff --git a/drivers/net/nfp/flower/nfp_flower.c b/drivers/net/nfp/flower/nfp_flower.c index 88a1b061e1..a29edf8ca2 100644 --- a/drivers/net/nfp/flower/nfp_flower.c +++ b/drivers/net/nfp/flower/nfp_flower.c @@ -598,29 +598,20 @@ static int nfp_flower_enable_services(struct nfp_app_fw_flower *app_fw_flower) { int ret; - uint32_t service_id; + struct nfp_service_info info; const struct rte_service_spec flower_service = { .name = "flower_ctrl_vnic_service", .callback = nfp_flower_ctrl_vnic_service, .callback_userdata = (void *)app_fw_flower, }; - /* Register the flower services */ - ret = rte_service_component_register(&flower_service, &service_id); + ret = nfp_service_enable(&flower_service, &info); if (ret != 0) { - PMD_INIT_LOG(ERR, "Could not register %s", flower_service.name); - return -EINVAL; + PMD_INIT_LOG(ERR, "Could not enable service %s", flower_service.name); + return ret; } - app_fw_flower->ctrl_vnic_id = service_id; - PMD_INIT_LOG(INFO, "%s registered", flower_service.name); - - /* Map them to available service cores */ - ret = nfp_map_service(service_id); - if (ret != 0) { - PMD_INIT_LOG(ERR, "Could not map %s", flower_service.name); - return -EINVAL; - } + app_fw_flower->ctrl_vnic_id = info.id; return 0; } diff --git a/drivers/net/nfp/flower/nfp_flower_representor.c b/drivers/net/nfp/flower/nfp_flower_representor.c index 63fe37c8d7..12e8af54fb 100644 --- a/drivers/net/nfp/flower/nfp_flower_representor.c +++ b/drivers/net/nfp/flower/nfp_flower_representor.c @@ -396,6 +396,9 @@ nfp_flower_repr_dev_close(struct rte_eth_dev *dev) if (app_fw_flower->pf_repr != NULL) return 0; + /* Disable cpp service */ + nfp_service_disable(&pf_dev->cpp_service_info); + /* Now it is safe to free all PF resources */ nfp_uninit_app_fw_flower(pf_dev); nfp_pf_uninit(pf_dev); diff --git a/drivers/net/nfp/meson.build b/drivers/net/nfp/meson.build index c625355d7a..0f4beccbdf 100644 --- a/drivers/net/nfp/meson.build +++ b/drivers/net/nfp/meson.build @@ -40,6 +40,7 @@ sources = files( 'nfp_net_ctrl.c', 'nfp_net_flow.c', 'nfp_rxtx.c', + 'nfp_service.c', ) deps += ['hash', 'security', 'common_nfp'] diff --git a/drivers/net/nfp/nfp_cpp_bridge.c b/drivers/net/nfp/nfp_cpp_bridge.c index 36dcdca9de..441c0d2843 100644 --- a/drivers/net/nfp/nfp_cpp_bridge.c +++ b/drivers/net/nfp/nfp_cpp_bridge.c @@ -8,10 +8,9 @@ #include #include -#include - #include "nfpcore/nfp_cpp.h" #include "nfp_logs.h" +#include "nfp_service.h" #define NFP_CPP_MEMIO_BOUNDARY (1 << 20) #define NFP_BRIDGE_OP_READ 20 @@ -24,82 +23,23 @@ /* Prototypes */ static int nfp_cpp_bridge_service_func(void *args); -int -nfp_map_service(uint32_t service_id) -{ - int32_t ret; - uint32_t slcore = 0; - int32_t slcore_count; - uint8_t service_count; - const char *service_name; - uint32_t slcore_array[RTE_MAX_LCORE]; - uint8_t min_service_count = UINT8_MAX; - - slcore_count = rte_service_lcore_list(slcore_array, RTE_MAX_LCORE); - if (slcore_count <= 0) { - PMD_INIT_LOG(DEBUG, "No service cores found"); - return -ENOENT; - } - - /* - * Find a service core with the least number of services already - * registered to it. - */ - while (slcore_count--) { - service_count = rte_service_lcore_count_services(slcore_array[slcore_count]); - if (service_count < min_service_count) { - slcore = slcore_array[slcore_count]; - min_service_count = service_count; - } - } - - service_name = rte_service_get_name(service_id); - PMD_INIT_LOG(INFO, "Mapping service %s to core %u", service_name, slcore); - - ret = rte_service_map_lcore_set(service_id, slcore, 1); - if (ret != 0) { - PMD_INIT_LOG(DEBUG, "Could not map flower service"); - return -ENOENT; - } - - rte_service_runstate_set(service_id, 1); - rte_service_component_runstate_set(service_id, 1); - rte_service_lcore_start(slcore); - if (rte_service_may_be_active(slcore) != 0) - PMD_INIT_LOG(INFO, "The service %s is running", service_name); - else - PMD_INIT_LOG(ERR, "The service %s is not running", service_name); - - return 0; -} - int nfp_enable_cpp_service(struct nfp_pf_dev *pf_dev) { int ret; - uint32_t service_id = 0; + const char *pci_name; struct rte_service_spec cpp_service = { - .name = "nfp_cpp_service", - .callback = nfp_cpp_bridge_service_func, + .callback = nfp_cpp_bridge_service_func, + .callback_userdata = (void *)pf_dev, }; - cpp_service.callback_userdata = (void *)pf_dev; + pci_name = strchr(pf_dev->pci_dev->name, ':') + 1; + snprintf(cpp_service.name, sizeof(cpp_service.name), "%s_cpp_service", pci_name); - /* Register the cpp service */ - ret = rte_service_component_register(&cpp_service, &service_id); + ret = nfp_service_enable(&cpp_service, &pf_dev->cpp_service_info); if (ret != 0) { - PMD_INIT_LOG(WARNING, "Could not register nfp cpp service"); - return -EINVAL; - } - - pf_dev->cpp_bridge_id = service_id; - PMD_INIT_LOG(INFO, "NFP cpp service registered"); - - /* Map it to available service core */ - ret = nfp_map_service(service_id); - if (ret != 0) { - PMD_INIT_LOG(DEBUG, "Could not map nfp cpp service"); - return -EINVAL; + PMD_INIT_LOG(DEBUG, "Could not enable service %s", cpp_service.name); + return ret; } return 0; @@ -387,12 +327,18 @@ nfp_cpp_bridge_service_func(void *args) int sockfd; int datafd; struct nfp_cpp *cpp; + const char *pci_name; + char socket_handle[14]; struct sockaddr address; struct nfp_pf_dev *pf_dev; struct timeval timeout = {1, 0}; - unlink("/tmp/nfp_cpp"); + pf_dev = args; + + pci_name = strchr(pf_dev->pci_dev->name, ':') + 1; + snprintf(socket_handle, sizeof(socket_handle), "/tmp/%s", pci_name); + unlink(socket_handle); sockfd = socket(AF_UNIX, SOCK_STREAM, 0); if (sockfd < 0) { PMD_CPP_LOG(ERR, "socket creation error. Service failed"); @@ -404,7 +350,7 @@ nfp_cpp_bridge_service_func(void *args) memset(&address, 0, sizeof(struct sockaddr)); address.sa_family = AF_UNIX; - strcpy(address.sa_data, "/tmp/nfp_cpp"); + strcpy(address.sa_data, socket_handle); ret = bind(sockfd, (const struct sockaddr *)&address, sizeof(struct sockaddr)); @@ -421,9 +367,8 @@ nfp_cpp_bridge_service_func(void *args) return ret; } - pf_dev = args; cpp = pf_dev->cpp; - while (rte_service_runstate_get(pf_dev->cpp_bridge_id) != 0) { + while (rte_service_runstate_get(pf_dev->cpp_service_info.id) != 0) { datafd = accept(sockfd, NULL, NULL); if (datafd < 0) { if (errno == EAGAIN || errno == EWOULDBLOCK) diff --git a/drivers/net/nfp/nfp_cpp_bridge.h b/drivers/net/nfp/nfp_cpp_bridge.h index bf975ce7ba..406c961dce 100644 --- a/drivers/net/nfp/nfp_cpp_bridge.h +++ b/drivers/net/nfp/nfp_cpp_bridge.h @@ -9,6 +9,5 @@ #include "nfp_net_common.h" int nfp_enable_cpp_service(struct nfp_pf_dev *pf_dev); -int nfp_map_service(uint32_t service_id); #endif /* __NFP_CPP_BRIDGE_H__ */ diff --git a/drivers/net/nfp/nfp_net_common.h b/drivers/net/nfp/nfp_net_common.h index 3a441f49f6..eaa7501e8e 100644 --- a/drivers/net/nfp/nfp_net_common.h +++ b/drivers/net/nfp/nfp_net_common.h @@ -14,6 +14,7 @@ #include "nfpcore/nfp_sync.h" #include "nfp_net_ctrl.h" +#include "nfp_service.h" /* Interrupt definitions */ #define NFP_NET_IRQ_LSC_IDX 0 @@ -102,8 +103,8 @@ struct nfp_pf_dev { struct nfp_hwinfo *hwinfo; struct nfp_rtsym_table *sym_tbl; - /** Service id of cpp bridge service */ - uint32_t cpp_bridge_id; + /** Service info of cpp bridge service */ + struct nfp_service_info cpp_service_info; /** Multiple PF configuration */ struct nfp_multi_pf multi_pf; diff --git a/drivers/net/nfp/nfp_service.c b/drivers/net/nfp/nfp_service.c new file mode 100644 index 0000000000..f49fa6addf --- /dev/null +++ b/drivers/net/nfp/nfp_service.c @@ -0,0 +1,117 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (c) 2023 Corigine, Inc. + * All rights reserved. + */ + +#include "nfp_service.h" + +#include "nfpcore/nfp_cpp.h" +#include "nfp_logs.h" + +/* Disable service and try to get service status */ +#define NFP_SERVICE_DISABLE_WAIT_COUNT 3000 + +static int +nfp_map_service(struct nfp_service_info *info) +{ + int32_t ret; + uint32_t slcore = 0; + int32_t slcore_count; + uint8_t service_count; + const char *service_name; + uint32_t slcore_array[RTE_MAX_LCORE]; + uint8_t min_service_count = UINT8_MAX; + + slcore_count = rte_service_lcore_list(slcore_array, RTE_MAX_LCORE); + if (slcore_count <= 0) { + PMD_DRV_LOG(DEBUG, "No service cores found"); + return -ENOENT; + } + + /* + * Find a service core with the least number of services already + * registered to it + */ + while (slcore_count--) { + service_count = rte_service_lcore_count_services(slcore_array[slcore_count]); + if (service_count < min_service_count) { + slcore = slcore_array[slcore_count]; + min_service_count = service_count; + } + } + + service_name = rte_service_get_name(info->id); + PMD_INIT_LOG(INFO, "Mapping service %s to core %u", service_name, slcore); + + ret = rte_service_map_lcore_set(info->id, slcore, 1); + if (ret != 0) { + PMD_DRV_LOG(DEBUG, "Could not map flower service"); + return -ENOENT; + } + + rte_service_runstate_set(info->id, 1); + rte_service_component_runstate_set(info->id, 1); + rte_service_lcore_start(slcore); + if (rte_service_may_be_active(slcore)) + PMD_DRV_LOG(INFO, "The service %s is running", service_name); + else + PMD_DRV_LOG(ERR, "The service %s is not running", service_name); + + info->lcore = slcore; + + return 0; +} + +int +nfp_service_enable(const struct rte_service_spec *service_spec, + struct nfp_service_info *info) +{ + int ret; + + /* Register the service */ + ret = rte_service_component_register(service_spec, &info->id); + if (ret != 0) { + PMD_DRV_LOG(DEBUG, "Could not register %s", service_spec->name); + return -EINVAL; + } + + /* Map it to available service core */ + ret = nfp_map_service(info); + if (ret != 0) { + PMD_DRV_LOG(DEBUG, "Could not map %s", service_spec->name); + return -EINVAL; + } + + PMD_DRV_LOG(DEBUG, "Enable service %s successfully", service_spec->name); + + return 0; +} + +int +nfp_service_disable(struct nfp_service_info *info) +{ + uint32_t i; + const char *service_name; + + service_name = rte_service_get_name(info->id); + if (service_name == NULL) { + PMD_DRV_LOG(ERR, "Could not find service %u", info->id); + return -EINVAL; + } + + rte_service_runstate_set(info->id, 0); + rte_service_component_runstate_set(info->id, 0); + + for (i = 0; i < NFP_SERVICE_DISABLE_WAIT_COUNT; i++) { + if (rte_service_may_be_active(info->id) == 0) + break; + rte_delay_ms(1); + } + if (i == NFP_SERVICE_DISABLE_WAIT_COUNT) + PMD_DRV_LOG(ERR, "Could not stop service %s", service_name); + + rte_service_map_lcore_set(info->id, info->lcore, 0); + rte_service_component_unregister(info->id); + + return 0; +} diff --git a/drivers/net/nfp/nfp_service.h b/drivers/net/nfp/nfp_service.h new file mode 100644 index 0000000000..4b99d5b973 --- /dev/null +++ b/drivers/net/nfp/nfp_service.h @@ -0,0 +1,20 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (c) 2023 Corigine, Inc. + * All rights reserved. + */ + +#ifndef __NFP_SERVICE_H__ +#define __NFP_SERVICE_H__ + +#include + +struct nfp_service_info { + uint32_t id; + uint32_t lcore; +}; + +int nfp_service_disable(struct nfp_service_info *info); +int nfp_service_enable(const struct rte_service_spec *service_spec, + struct nfp_service_info *info); + +#endif /* __NFP_SERVICE_H__ */ From patchwork Fri Feb 2 03:04:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chaoyong He X-Patchwork-Id: 136302 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 81E4043A49; Fri, 2 Feb 2024 04:05:11 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 1039D42E1E; Fri, 2 Feb 2024 04:04:53 +0100 (CET) Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2099.outbound.protection.outlook.com [40.107.94.99]) by mails.dpdk.org (Postfix) with ESMTP id 5499542E00 for ; Fri, 2 Feb 2024 04:04:51 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=h1pN7Qhd0r0VvffH2+HiyZqDCpYvrqXHSf07KdwsEwfZUTbyIBb+HEoRWPg+YVGI39VeHOsZ3qkhlxQaXHAFWjHQ41Nlol+4ZwgVQc/cSOnZcmmcEE/3TPA6TLEkq7SM076UXV7VwWsbQqVuVhA6Za3JemLxQR/cEy1e1HKl5yd7Q6MLvjANVIJH1yQpiGngxd5vpeFU7KvRFUCbN0HO2r3RYOxkFIjEiou3r5q/BTTfV4C1QjPerRspIq1z3xxjvm1tksV4x+wPj8It+Nd3NNgSHJoJFn7sZQ76q1TsGC14AFKMrZZVel1EPcjQn/FpkLPUtAEW3Bb5W+UBjOgrIg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=zp0fJdpiBzHiCWMscHB0du8tHEo925iRHXtIoc6iU2Y=; b=EZidUHzs3kF0TV2WxxW7wAuijXKv270kzSNDHtQPX8uOMV2/HRNrhcC2UpdzthP/G/2sz/lO14MxAPKbEij2QvtSRiPK6uTGfLU1EgVkUy1PPYOMs7B4pcnjYZ1qtbR0J+fjzuECJGu3AKrXLQ/8dcWqm3/GLPczkO4N0z6+AqzbFhwzvJmaHxI2DW8nUFdS7ZQTX4R1tfYP/3EreVwBDJfM5ri9/JPWHhkADdYC9Xun60v7AEf/n0K/JTG6LdQ6an3GnRv4upnl3rKMPyZmgxU/AgPSeEIMqbfoV1Gt/2W5egulnmv6DT91RyBwam6Wtm+8vXUlkqbMqienxXFsdQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=corigine.com; dmarc=pass action=none header.from=corigine.com; dkim=pass header.d=corigine.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=corigine.onmicrosoft.com; s=selector2-corigine-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=zp0fJdpiBzHiCWMscHB0du8tHEo925iRHXtIoc6iU2Y=; b=Khw8p+hsmDIJpTYdDV5KJwMEqQyg4cj26ZlWQFTiMh81ncfaKco8Ss48th01jMiR3rM7zMl1r7Gv/6eSLGCZfRQvIBfRhdahAfABAKlHGlXilRNTiIAfNBvsqrLnN62dY1q9jsKKVASdtvKZUEsXNN6LznFanzljSWfXOxJbvJA= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=corigine.com; Received: from SJ0PR13MB5545.namprd13.prod.outlook.com (2603:10b6:a03:424::5) by SN7PR13MB6177.namprd13.prod.outlook.com (2603:10b6:806:2e3::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7249.30; Fri, 2 Feb 2024 03:04:49 +0000 Received: from SJ0PR13MB5545.namprd13.prod.outlook.com ([fe80::8e02:f738:570a:f8aa]) by SJ0PR13MB5545.namprd13.prod.outlook.com ([fe80::8e02:f738:570a:f8aa%7]) with mapi id 15.20.7249.024; Fri, 2 Feb 2024 03:04:49 +0000 From: Chaoyong He To: dev@dpdk.org Cc: oss-drivers@corigine.com, Long Wu , Chaoyong He , Peng Zhang Subject: [PATCH v2 3/3] net/nfp: flower driver uses one service core Date: Fri, 2 Feb 2024 11:04:15 +0800 Message-Id: <20240202030415.739638-4-chaoyong.he@corigine.com> X-Mailer: git-send-email 2.39.1 In-Reply-To: <20240202030415.739638-1-chaoyong.he@corigine.com> References: <20240124092541.2011874-1-chaoyong.he@corigine.com> <20240202030415.739638-1-chaoyong.he@corigine.com> X-ClientProxiedBy: BYAPR05CA0004.namprd05.prod.outlook.com (2603:10b6:a03:c0::17) To SJ0PR13MB5545.namprd13.prod.outlook.com (2603:10b6:a03:424::5) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SJ0PR13MB5545:EE_|SN7PR13MB6177:EE_ X-MS-Office365-Filtering-Correlation-Id: ce652371-1503-4330-be2d-08dc239bb7d0 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 5CTEAbs7zZJ4j/Ivtg2Mj8xvfIzrXTu0Gf5Rw0rNu5XW055QBMplxA4bUHmG+h9ANMnFVYdIKyZ+xsYcTzUiChwTzNs6wEi5fnZpau7UUgiBacYN0waAlwmWJ8daCqzR/KYVKAc4xsRsaRToOaRybJoFhKf7jODRf6Mp9OnRJ8KsP9q/wGksTCY1XU/D6CjPE/lcYdaTwPL/FdzeaN05u5JKq+X0mcm0kluqW4GqOgYpevjCIJ5RX9dUoSu/yXUWFF82dmxYVmx4TkGlZyTzMDxR1q6UXuS+3uhUXJobWoN38vWxB9OiiLH46FM2M17Hx/Q04WtIaH8UTIMEEFQwQCwc2e7/PKpjvTkO7dBxkhxkWSDk26OB1BLpwVZuJEIn7ZjWhNNeOtbiiqEaYISDBmk/wr/L5Q8IMuBp2gl7HqnyXUDhbBewZb3nPACD6GrX8H10gZxO/KiWBkL5wbKvjA9MjMKNV2AxnYyTC88uZidOmrQ6oOyksN+MubNNJutpHLX3BY6Oi3SU/99lL+B0kcn0nwY1CQpNdaOLVV016uMkhknlQSCBDJSpq2pMK3UmhmWWrnyupTFNh61IA+Mry0LOFXl6Oawqe746hWGN4IRPJdQAXY2f3w/8upx0pswt X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SJ0PR13MB5545.namprd13.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230031)(366004)(136003)(376002)(396003)(39840400004)(346002)(230922051799003)(451199024)(186009)(64100799003)(1800799012)(41300700001)(1076003)(107886003)(26005)(2616005)(6916009)(38350700005)(316002)(36756003)(478600001)(6512007)(6486002)(83380400001)(52116002)(6506007)(6666004)(38100700002)(86362001)(2906002)(5660300002)(30864003)(66946007)(54906003)(66476007)(66556008)(8936002)(44832011)(8676002)(4326008); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: KcReKh636kJIqbdum8CMA5SO14zAj6emBPFV7Zhpy+m7970Axa7cHbDMeH961hT4FBhNqQOC7IfnCWdriPYsRntx80G0Qx6+oORj4vwZWeYazl56Ayhd+KWQNYPSD+58cvqE/fx/G6d2B59riETsP5npycsLdAAuOp0JbnjqFFF8jLFd0BLn/s793qEMQE/8Ww4fOLxkwKr2bNqttAzLWQfRHzU0MKJBAf3W8cDgg+T28EBsTKntDWPauE8ZqKjqtxWQkqgldBQmiwtMjoap9qHBF2D4BWFVAFesb4DrVYLZuqJE2KYLkLpaNodEHKkrvSxuSj06fKm4FWcYBFb6KrZlQlYBk7JTfEcrUxYpf8oXzgbOV6v8QFNQ/VOfCcL4Fsb9UsnIFP0ckeOS4UiC4P7aHVGVBItg3kiNEqLFWicmycRDH3asAidep/4ovmJlwXFH308+PHXpTvxi/EZqd02A2OUOvXZNiuKCsg5QowMqLwJ5lHffMGz8gV9JvhcCDE6R68TLXf+wtT5P2NhYOl/ZE9LpDkFzi0RaBaQPGDnSzwhFyiWW1WX6CqwPKHOtnK0gRh33WEbqptKpDkZVeyrACiZ4rWLl/SUvDF0ybDV74YfWLBu/VJ5A9ZxPHzwQiZlVhbFt4SuHJNg9nS4X96N0iQsH/KUOY5qyfBlM8R26BWMucGgBI0958uoQlpGTUEsF5BLCQ36ICHWrEi3H6FQAdzYUW6mrgVanDR5709u0lEiO7KyrYmBwwCI2qmhq9kzGR3w4Z9O0vIxjX4+szUqqkEAL4k//rBFw0EZI1AjWTngAngzrvqX28dBSIzkCKu0E2QdbB+mWz4ngV7+wszuB1kwu59SnenjiB7rJgdEUSEPVq3KpNgJ7rdhcvpoW6i3fkGplBe9rhqBNzPxZ9/IVroflVTjqiKFp56GoeYNloPqsUWkkceclHD7vwxmKlx+n8bre03qgSTBI/GV8sJpJM4ywWspDh50n566B1WFWF6It3PfZlk9+t35Mp5bKgRSYRfqhE0sHk3mR5YFlQ9Nwlf6gNYpFjjpXEiQZfPOU5MNoT4sMj+cqAdCBW/Rp4d8Yo3ypOwU1AWtpNUPhLzSU6fox3VuJE93ABFVF97zwq/xP996VciP7EyCo9fXJ2JlRz8AtjQE+jJb4zx/v9AzoUop9Z7UOJhnM0p65bd1bFpcjCIjSNOz69/BMSUiMkcoiElQTektjqntO+ANXzqQDmyEx6idTMghPPfnw+SGfUxF0HCA3bJpjXo+OVeg12b+2BVYIH9Iuhp9RFci6gEgtOVZ/x0qT2NU5RvhC31qVzVu8GAweZFlYTS4luTdi7ZMZ+VehNxtGrMtrcVbCVhwM0Otr/j7qMMG77H9vH5yjHg5+sO+mITbEMUm/OplVKn3XBg1bb1QRSGN9I3bNMw0T4YADm41QYLxhKTU25Rw37BcbiT5tSv6tz6+Y0HdauIIEyOiEzXLo00Vjc6n44snbwWbo54m+eSrfFrLhRSwl2Xl6Qcvriz6NRo0sW6de3+588usED8OyaA6sig/Gzke2JuwsHPlT7rE6rChdiD4E45RtvYT9z0L+JbZ5DFLNvMpMfwfXhB6xVBw3j39kcA== X-OriginatorOrg: corigine.com X-MS-Exchange-CrossTenant-Network-Message-Id: ce652371-1503-4330-be2d-08dc239bb7d0 X-MS-Exchange-CrossTenant-AuthSource: SJ0PR13MB5545.namprd13.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Feb 2024 03:04:49.3184 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: fe128f2c-073b-4c20-818e-7246a585940c X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: cGujUR6c/M37rcp5PKP/VJCxr5/jXhhWdoKmovhy1oi/HJvs6ysIF1hJgWOtxST/laDnkRGCqmKiPFHH6rH5QTN5Wns/H+uaDuF5P648wHo= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN7PR13MB6177 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Long Wu Now app must provide a service core for each NFP pci device that uses flower driver to run flower service, which will cause huge resource waste. This patch is to solve this problem and app can only use one core to run NFP flower service though it uses several NFP NICs. Signed-off-by: Long Wu Reviewed-by: Chaoyong He Reviewed-by: Peng Zhang --- drivers/net/nfp/flower/nfp_flower.c | 47 ++--- drivers/net/nfp/flower/nfp_flower_ctrl.c | 20 +- drivers/net/nfp/flower/nfp_flower_ctrl.h | 2 +- .../net/nfp/flower/nfp_flower_representor.c | 4 + drivers/net/nfp/flower/nfp_flower_service.c | 196 ++++++++++++++++++ drivers/net/nfp/flower/nfp_flower_service.h | 17 ++ drivers/net/nfp/meson.build | 1 + drivers/net/nfp/nfp_net_common.h | 6 + drivers/net/nfp/nfpcore/nfp_sync.h | 2 + 9 files changed, 248 insertions(+), 47 deletions(-) create mode 100644 drivers/net/nfp/flower/nfp_flower_service.c create mode 100644 drivers/net/nfp/flower/nfp_flower_service.h diff --git a/drivers/net/nfp/flower/nfp_flower.c b/drivers/net/nfp/flower/nfp_flower.c index a29edf8ca2..e84e6ebbff 100644 --- a/drivers/net/nfp/flower/nfp_flower.c +++ b/drivers/net/nfp/flower/nfp_flower.c @@ -18,6 +18,7 @@ #include "../nfp_mtr.h" #include "nfp_flower_ctrl.h" #include "nfp_flower_representor.h" +#include "nfp_flower_service.h" #define CTRL_VNIC_NB_DESC 512 @@ -461,6 +462,13 @@ nfp_flower_init_ctrl_vnic(struct nfp_net_hw *hw) nn_cfg_writeb(&hw->super, NFP_NET_CFG_TXR_SZ(i), rte_log2_u32(CTRL_VNIC_NB_DESC)); } + /* Alloc sync memory zone */ + ret = nfp_flower_service_sync_alloc(app_fw_flower); + if (ret != 0) { + PMD_INIT_LOG(ERR, "Alloc sync memory zone failed"); + goto tx_queue_setup_cleanup; + } + return 0; tx_queue_setup_cleanup: @@ -531,6 +539,7 @@ nfp_flower_cleanup_ctrl_vnic(struct nfp_net_hw *hw) } } + nfp_flower_service_sync_free(app_fw_flower); rte_free(eth_dev->data->tx_queues); rte_free(eth_dev->data->rx_queues); rte_mempool_free(app_fw_flower->ctrl_pktmbuf_pool); @@ -584,38 +593,6 @@ nfp_flower_start_ctrl_vnic(struct nfp_net_hw *net_hw) return 0; } -static int -nfp_flower_ctrl_vnic_service(void *arg) -{ - struct nfp_app_fw_flower *app_fw_flower = arg; - - nfp_flower_ctrl_vnic_poll(app_fw_flower); - - return 0; -} - -static int -nfp_flower_enable_services(struct nfp_app_fw_flower *app_fw_flower) -{ - int ret; - struct nfp_service_info info; - const struct rte_service_spec flower_service = { - .name = "flower_ctrl_vnic_service", - .callback = nfp_flower_ctrl_vnic_service, - .callback_userdata = (void *)app_fw_flower, - }; - - ret = nfp_service_enable(&flower_service, &info); - if (ret != 0) { - PMD_INIT_LOG(ERR, "Could not enable service %s", flower_service.name); - return ret; - } - - app_fw_flower->ctrl_vnic_id = info.id; - - return 0; -} - static void nfp_flower_pkt_add_metadata_register(struct nfp_app_fw_flower *app_fw_flower) { @@ -760,7 +737,7 @@ nfp_init_app_fw_flower(struct nfp_pf_dev *pf_dev, } /* Start up flower services */ - ret = nfp_flower_enable_services(app_fw_flower); + ret = nfp_flower_service_start(app_fw_flower); if (ret != 0) { PMD_INIT_LOG(ERR, "Could not enable flower services"); ret = -ESRCH; @@ -770,11 +747,13 @@ nfp_init_app_fw_flower(struct nfp_pf_dev *pf_dev, ret = nfp_flower_repr_create(app_fw_flower); if (ret != 0) { PMD_INIT_LOG(ERR, "Could not create representor ports"); - goto ctrl_vnic_cleanup; + goto ctrl_vnic_service_stop; } return 0; +ctrl_vnic_service_stop: + nfp_flower_service_stop(app_fw_flower); ctrl_vnic_cleanup: nfp_flower_cleanup_ctrl_vnic(app_fw_flower->ctrl_hw); ctrl_cpp_area_cleanup: diff --git a/drivers/net/nfp/flower/nfp_flower_ctrl.c b/drivers/net/nfp/flower/nfp_flower_ctrl.c index c25487c277..bcb325d475 100644 --- a/drivers/net/nfp/flower/nfp_flower_ctrl.c +++ b/drivers/net/nfp/flower/nfp_flower_ctrl.c @@ -12,6 +12,7 @@ #include "../nfp_logs.h" #include "nfp_flower_representor.h" #include "nfp_mtr.h" +#include "nfp_flower_service.h" #define MAX_PKT_BURST 32 @@ -502,26 +503,21 @@ nfp_flower_cmsg_rx(struct nfp_app_fw_flower *app_fw_flower, } void -nfp_flower_ctrl_vnic_poll(struct nfp_app_fw_flower *app_fw_flower) +nfp_flower_ctrl_vnic_process(struct nfp_app_fw_flower *app_fw_flower) { uint16_t count; struct nfp_net_rxq *rxq; - struct nfp_net_hw *ctrl_hw; struct rte_eth_dev *ctrl_eth_dev; struct rte_mbuf *pkts_burst[MAX_PKT_BURST]; - ctrl_hw = app_fw_flower->ctrl_hw; - ctrl_eth_dev = ctrl_hw->eth_dev; + ctrl_eth_dev = app_fw_flower->ctrl_hw->eth_dev; /* Ctrl vNIC only has a single Rx queue */ rxq = ctrl_eth_dev->data->rx_queues[0]; - - while (rte_service_runstate_get(app_fw_flower->ctrl_vnic_id) != 0) { - count = nfp_flower_ctrl_vnic_recv(rxq, pkts_burst, MAX_PKT_BURST); - if (count != 0) { - app_fw_flower->ctrl_vnic_rx_count += count; - /* Process cmsgs here */ - nfp_flower_cmsg_rx(app_fw_flower, pkts_burst, count); - } + count = nfp_flower_ctrl_vnic_recv(rxq, pkts_burst, MAX_PKT_BURST); + if (count != 0) { + app_fw_flower->ctrl_vnic_rx_count += count; + /* Process cmsgs here */ + nfp_flower_cmsg_rx(app_fw_flower, pkts_burst, count); } } diff --git a/drivers/net/nfp/flower/nfp_flower_ctrl.h b/drivers/net/nfp/flower/nfp_flower_ctrl.h index 4c94d36847..131d002ac6 100644 --- a/drivers/net/nfp/flower/nfp_flower_ctrl.h +++ b/drivers/net/nfp/flower/nfp_flower_ctrl.h @@ -8,7 +8,7 @@ #include "nfp_flower.h" -void nfp_flower_ctrl_vnic_poll(struct nfp_app_fw_flower *app_fw_flower); +void nfp_flower_ctrl_vnic_process(struct nfp_app_fw_flower *app_fw_flower); uint16_t nfp_flower_ctrl_vnic_xmit(struct nfp_app_fw_flower *app_fw_flower, struct rte_mbuf *mbuf); void nfp_flower_ctrl_vnic_xmit_register(struct nfp_app_fw_flower *app_fw_flower); diff --git a/drivers/net/nfp/flower/nfp_flower_representor.c b/drivers/net/nfp/flower/nfp_flower_representor.c index 12e8af54fb..4f4df0cd2e 100644 --- a/drivers/net/nfp/flower/nfp_flower_representor.c +++ b/drivers/net/nfp/flower/nfp_flower_representor.c @@ -9,6 +9,7 @@ #include "../nfpcore/nfp_nsp.h" #include "../nfp_logs.h" #include "../nfp_mtr.h" +#include "nfp_flower_service.h" /* Type of representor */ enum nfp_repr_type { @@ -396,6 +397,9 @@ nfp_flower_repr_dev_close(struct rte_eth_dev *dev) if (app_fw_flower->pf_repr != NULL) return 0; + /* Stop flower service first */ + nfp_flower_service_stop(app_fw_flower); + /* Disable cpp service */ nfp_service_disable(&pf_dev->cpp_service_info); diff --git a/drivers/net/nfp/flower/nfp_flower_service.c b/drivers/net/nfp/flower/nfp_flower_service.c new file mode 100644 index 0000000000..5e8811fe24 --- /dev/null +++ b/drivers/net/nfp/flower/nfp_flower_service.c @@ -0,0 +1,196 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (c) 2024 Corigine, Inc. + * All rights reserved. + */ + +#include "nfp_flower_service.h" + +#include + +#include "nfp_flower_ctrl.h" +#include "../nfpcore/nfp_cpp.h" +#include "../nfpcore/nfp_sync.h" +#include "../nfp_logs.h" +#include "../nfp_service.h" + +/* Driver limitation, PMD can enlarge it if need. */ +#define MAX_FLOWER_SERVICE_SLOT 8 + +struct nfp_flower_service { + /** Flower service is enabled */ + bool enabled; + /** Flower service info */ + struct nfp_service_info info; + /** Store flower cards' information */ + struct nfp_app_fw_flower *slots[MAX_FLOWER_SERVICE_SLOT]; + /** Spinlock for sync slots when add/remove card */ + rte_spinlock_t spinlock; +}; + +static struct nfp_flower_service * +nfp_flower_service_handle_get(struct nfp_app_fw_flower *app) +{ + return app->pf_hw->pf_dev->process_share.fl_service; +} + +static int +nfp_flower_service_loop(void *arg) +{ + uint16_t slot; + struct nfp_app_fw_flower *app; + struct nfp_flower_service *service_handle; + + service_handle = arg; + /* Waiting for enabling service */ + while (!service_handle->enabled) + rte_delay_ms(1); + + while (rte_service_runstate_get(service_handle->info.id) != 0) { + rte_spinlock_lock(&service_handle->spinlock); + for (slot = 0; slot < MAX_FLOWER_SERVICE_SLOT; slot++) { + app = service_handle->slots[slot]; + if (app == NULL) + continue; + + nfp_flower_ctrl_vnic_process(app); + } + rte_spinlock_unlock(&service_handle->spinlock); + } + + return 0; +} + +static int +nfp_flower_service_enable(struct nfp_flower_service *service_handle) +{ + int ret; + + const struct rte_service_spec flower_service = { + .name = "flower_ctrl_vnic_service", + .callback = nfp_flower_service_loop, + .callback_userdata = (void *)service_handle, + }; + + ret = nfp_service_enable(&flower_service, &service_handle->info); + if (ret != 0) + return ret; + + rte_spinlock_init(&service_handle->spinlock); + service_handle->enabled = true; + + return 0; +} + +static uint16_t +nfp_flower_service_insert(struct nfp_app_fw_flower *app, + struct nfp_flower_service *service_handle) +{ + uint16_t slot; + + rte_spinlock_lock(&service_handle->spinlock); + for (slot = 0; slot < MAX_FLOWER_SERVICE_SLOT; slot++) { + if (service_handle->slots[slot] == NULL) { + service_handle->slots[slot] = app; + break; + } + } + rte_spinlock_unlock(&service_handle->spinlock); + + return slot; +} + +int +nfp_flower_service_start(void *app_fw_flower) +{ + int ret; + struct nfp_flower_service *service_handle; + struct nfp_app_fw_flower *app = app_fw_flower; + + service_handle = nfp_flower_service_handle_get(app); + if (service_handle == NULL) { + PMD_DRV_LOG(ERR, "Can not get service handle"); + return -EINVAL; + } + + /* Enable flower service when driver initializes the first NIC */ + if (!service_handle->enabled) { + ret = nfp_flower_service_enable(service_handle); + if (ret != 0) { + PMD_DRV_LOG(ERR, "Could not enable flower service"); + return -ESRCH; + } + } + + /* Insert the NIC to flower service slot */ + ret = nfp_flower_service_insert(app, service_handle); + if (ret == MAX_FLOWER_SERVICE_SLOT) { + PMD_DRV_LOG(ERR, "Flower ctrl vnic service slot over %u", + MAX_FLOWER_SERVICE_SLOT); + return -ENOSPC; + } + + return 0; +} + +void +nfp_flower_service_stop(void *app_fw_flower) +{ + uint16_t slot; + uint16_t count; + struct nfp_flower_service *service_handle; + struct nfp_app_fw_flower *app = app_fw_flower; + + service_handle = nfp_flower_service_handle_get(app); + if (service_handle == NULL) { + PMD_DRV_LOG(ERR, "Can not get service handle"); + return; + } + + rte_spinlock_lock(&service_handle->spinlock); + for (slot = 0; slot < MAX_FLOWER_SERVICE_SLOT; slot++) { + /* The app only in one slot */ + if (service_handle->slots[slot] != app) + continue; + + service_handle->slots[slot] = NULL; + } + rte_spinlock_unlock(&service_handle->spinlock); + + /* Determine whether to disable service */ + count = nfp_sync_handle_count_get(app->pf_hw->pf_dev->sync, NULL, + service_handle); + if (count > 1) + return; + + if (nfp_service_disable(&service_handle->info) != 0) + PMD_DRV_LOG(ERR, "Could not disable service"); +} + +int +nfp_flower_service_sync_alloc(void *app_fw_flower) +{ + struct nfp_flower_service *service_handle; + struct nfp_app_fw_flower *app = app_fw_flower; + struct nfp_pf_dev *pf_dev = app->pf_hw->pf_dev; + + service_handle = nfp_sync_handle_alloc(pf_dev->sync, NULL, + NFP_SYNC_MAGIC_FL_SERVICE, + sizeof(struct nfp_flower_service)); + if (service_handle == NULL) + return -ENOMEM; + + pf_dev->process_share.fl_service = service_handle; + + return 0; +} + +void +nfp_flower_service_sync_free(void *app_fw_flower) +{ + struct nfp_app_fw_flower *app = app_fw_flower; + struct nfp_pf_dev *pf_dev = app->pf_hw->pf_dev; + + nfp_sync_handle_free(pf_dev->sync, NULL, pf_dev->process_share.fl_service); + + pf_dev->process_share.fl_service = NULL; +} diff --git a/drivers/net/nfp/flower/nfp_flower_service.h b/drivers/net/nfp/flower/nfp_flower_service.h new file mode 100644 index 0000000000..a75780274f --- /dev/null +++ b/drivers/net/nfp/flower/nfp_flower_service.h @@ -0,0 +1,17 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (c) 2024 Corigine, Inc. + * All rights reserved. + */ + +#ifndef __NFP_FLOWER_SERVICE_H__ +#define __NFP_FLOWER_SERVICE_H__ + +struct nfp_flower_service; + +int nfp_flower_service_start(void *app_fw_flower); +void nfp_flower_service_stop(void *app_fw_flower); + +int nfp_flower_service_sync_alloc(void *app_fw_flower); +void nfp_flower_service_sync_free(void *app_fw_flower); + +#endif /* __NFP_FLOWER_SERVICE_H__ */ diff --git a/drivers/net/nfp/meson.build b/drivers/net/nfp/meson.build index 0f4beccbdf..e376fd328f 100644 --- a/drivers/net/nfp/meson.build +++ b/drivers/net/nfp/meson.build @@ -13,6 +13,7 @@ sources = files( 'flower/nfp_flower_ctrl.c', 'flower/nfp_flower_flow.c', 'flower/nfp_flower_representor.c', + 'flower/nfp_flower_service.c', 'nfd3/nfp_nfd3_dp.c', 'nfdk/nfp_nfdk_dp.c', 'nfpcore/nfp_cppcore.c', diff --git a/drivers/net/nfp/nfp_net_common.h b/drivers/net/nfp/nfp_net_common.h index eaa7501e8e..e374739022 100644 --- a/drivers/net/nfp/nfp_net_common.h +++ b/drivers/net/nfp/nfp_net_common.h @@ -12,6 +12,7 @@ #include #include +#include "flower/nfp_flower_service.h" #include "nfpcore/nfp_sync.h" #include "nfp_net_ctrl.h" #include "nfp_service.h" @@ -80,6 +81,10 @@ struct nfp_multi_pf { uint8_t *beat_addr; }; +struct nfp_process_share { + struct nfp_flower_service *fl_service; +}; + struct nfp_pf_dev { /** Backpointer to associated pci device */ struct rte_pci_device *pci_dev; @@ -114,6 +119,7 @@ struct nfp_pf_dev { /** Synchronized info */ struct nfp_sync *sync; + struct nfp_process_share process_share; }; #define NFP_NET_FLOW_LIMIT 1024 diff --git a/drivers/net/nfp/nfpcore/nfp_sync.h b/drivers/net/nfp/nfpcore/nfp_sync.h index 82f01e2652..ff3f60937e 100644 --- a/drivers/net/nfp/nfpcore/nfp_sync.h +++ b/drivers/net/nfp/nfpcore/nfp_sync.h @@ -10,6 +10,8 @@ #include +#define NFP_SYNC_MAGIC_FL_SERVICE 0x53594e41 /**< ASCII - SYNA */ + struct nfp_sync; struct nfp_sync *nfp_sync_alloc(void);