From patchwork Mon Sep 26 06:59:45 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Chaoyong He X-Patchwork-Id: 116842 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id A030FA00C2; Mon, 26 Sep 2022 09:00:25 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id E6E5641133; Mon, 26 Sep 2022 09:00:23 +0200 (CEST) Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2136.outbound.protection.outlook.com [40.107.244.136]) by mails.dpdk.org (Postfix) with ESMTP id 0AB1E4069B for ; Mon, 26 Sep 2022 09:00:21 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=V+xxyFd4g1n9t931uCPzICRs4WKf+mKqHwDeRtvW2Az1T56WgFLlBIcXoknALk3eqSH+ZUO5jWJFLwcHwlMbB1DEvriavGizKs2r5T7PLTIlraNBLQTiG5hBEL0845aLaZINGIZpk2s0jc2ieIpxXLBQ5XofWVJngdwqOjthOoG5pxn1We2j9DhKXuzsEIdiOEBznJoKhZM5hHRE3lQK0rutxOF4CWpP3alf2pDHkKzkfcE+QsEYOMbYP4Z7LwSyA0Cy2Xj2mqmrPmDlMjEfldvVIGJ7EhvEOotLFMnn3Rez6qc0s9cQEdWUR+nAGe3S36jYWi58Ys2CegfvoEg/Rg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=2kWgqlAKKKOe5RANaE5XV6q8qDXVSmaKiPw4VtEaric=; b=G1713xqSaEBE0N9ok1gGNkeSx5ESvyEN3nkRLIViwHyHNkPfCxY3ZYnG82nu0zXZfGAePXwfBx62mcLZatkLI6E3YMkOondM5DNVsqBE1Afmwj+LOTWq64pQqLzgHeXnJDXDPo2BefKM3G/PrEP1AOYQbtCkhj0wzCmgwhyIy4ACi4Pi8K2vh/gBjdG4Bw0HUP6Uif62arabpwYCDouJiuUYTkeGDc5UqTWE8d0LPgbVvMhjsI1yUlfXbgQSOaNwlEUSxtC1eiyraLlJTk3oTuNjncLnBSdRNe7HdH2epUzae8rL9SkCfWRZp+YSBZsJQG9eRRMwTHWit5xBJG3r2Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=corigine.com; dmarc=pass action=none header.from=corigine.com; dkim=pass header.d=corigine.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=corigine.onmicrosoft.com; s=selector2-corigine-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=2kWgqlAKKKOe5RANaE5XV6q8qDXVSmaKiPw4VtEaric=; b=QgHbaNsBCfz5JDGB5becUVb8DAdVzd0K21JhdazHI7UUMwWClWJUPHmPNT0fsgRslRUQYab0P3Ie5AM0qXyt3HSRqTa4Jka7szm6+r2XFJFHh0QrPQf6AhFCPywS0faDUejxLYKNmqm2IIbGGpygLkAJwGEuXiyVYqSlSsrZ/zU= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=corigine.com; Received: from SJ0PR13MB5545.namprd13.prod.outlook.com (2603:10b6:a03:424::5) by SA1PR13MB4893.namprd13.prod.outlook.com (2603:10b6:806:18a::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5676.14; Mon, 26 Sep 2022 07:00:19 +0000 Received: from SJ0PR13MB5545.namprd13.prod.outlook.com ([fe80::779:2722:a8e5:503b]) by SJ0PR13MB5545.namprd13.prod.outlook.com ([fe80::779:2722:a8e5:503b%7]) with mapi id 15.20.5676.014; Mon, 26 Sep 2022 07:00:19 +0000 From: Chaoyong He To: dev@dpdk.org Cc: oss-drivers@corigine.com, niklas.soderlund@corigine.com, Chaoyong He , Heinrich Kuhn Subject: [PATCH v10 01/13] net/nfp: move app specific attributes to own struct Date: Mon, 26 Sep 2022 14:59:45 +0800 Message-Id: <1664175597-37248-2-git-send-email-chaoyong.he@corigine.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1664175597-37248-1-git-send-email-chaoyong.he@corigine.com> References: <1664175597-37248-1-git-send-email-chaoyong.he@corigine.com> X-ClientProxiedBy: SG2P153CA0005.APCP153.PROD.OUTLOOK.COM (2603:1096::15) To SJ0PR13MB5545.namprd13.prod.outlook.com (2603:10b6:a03:424::5) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SJ0PR13MB5545:EE_|SA1PR13MB4893:EE_ X-MS-Office365-Filtering-Correlation-Id: 18711a61-44d8-4734-f445-08da9f8cc5e9 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: eP6A74m3JC5VGT7H/+i2199fA2UzgS9IT5EJKJEQcdSDZ57OxRsrwGMgs3M8xMrIKepCUzGWdREQbdcEt24OXjACxSg7wcSThWKzCY+4TSg8cw4dPPEtRbCKeqgWfHLE2AJLFfl0QN37ptDOzJl1YpcI6J/6JJK+k7LvgsG1NAEnJiGM2k6/7nQxY/bkfRW7Aq/PDQnMQu/WKT9thbZ9f4zuhkDQXI/QQz3aJ/595icObpZQFYYUSrcFgAdA58hOvgSO8XyYnC/nrpCRTc7AXnENs6K2GIQJhJqinbhsrIJ72+E3UuSq0vJkPspd4E6GhxWzc/72sBMasAkRD1klYOdmIsTfEtUb+UZmN9Dv+fPgzWhsAkjFcObr6dBbKyrCFydZiozvNjJpul1cs++Qet+M+nEN909so+g0mEGT7EFBjkn8xRvTsXc5FYJbpWV5QZCagfMtYCTacxhp5qKlmf52pzcioJUTD5BxIf/qqdAhkbSrU/jTkqRBpfwPf5KlrubBkfxWeSyXEWBRpquIRnp2oDT+4ZS6EEi0UrZu4mEtpIeNEQa/yeY4wYb92PncZZwlzv7/fY/mK1IZ71m92onFtO59bAhoTa4d3ajJwoozLL6Ra5MSSA1SmEbQSab04NZXlQgBnLmJWIw1MIpYntBn0PQ3mcu+xpsvuG9qED73H1gXgbgjUVVYIOfEwzzba+GwT4UcUOTGT3mChkHL2sfrcSNTo6gqyvZ6kK5mVSTi0OvR2FQYbZtVg3eJz3Gssi2fpQYI0aeveU3WnUDPoQ== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SJ0PR13MB5545.namprd13.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230022)(4636009)(366004)(396003)(39830400003)(136003)(376002)(346002)(451199015)(36756003)(38100700002)(38350700002)(2906002)(86362001)(44832011)(30864003)(66946007)(8936002)(66556008)(66476007)(5660300002)(8676002)(4326008)(83380400001)(186003)(66574015)(54906003)(6512007)(6916009)(316002)(52116002)(6666004)(2616005)(26005)(6506007)(478600001)(41300700001)(107886003)(6486002); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?pUAszTfd6OwqtKlM6kvUfTSul/T8?= =?utf-8?q?E+UKTJL4WOKaeU7ZWSG4w7F7OkunanJRKlv0ahw6Day4QWu20kWrGlDrvWkJeaATp?= =?utf-8?q?Zv6Poyzp8l1PTWoxrlHlRya4Un7ubWKQnSWtXUJy0MdXIYzxgKYaUtFa9Q9Bp/PQK?= =?utf-8?q?T8idrKA4mTcS/3emI9eQkeOIMrRDRIsHIU02L5Xa36N+pM/mEyN4JcBoBkJq8YKF/?= =?utf-8?q?ZEpRfzl+bcHPcpGAM0OCKHrfyV0Xn7yC3bGV5cUEnDvH+BbIIQkVdDNgXs+HzgFPd?= =?utf-8?q?/HPqeOHY91c4w2TKdfUmq8fsDkzQdgJJWyRLHTsAekgndlkqSIYoZBObagMHjk4Ie?= =?utf-8?q?YISJcC8s3+2XAqPTif0k526e7v+T45l0cb94GhipJGq+HRe1g3sJK1+Ga1LlGe87+?= =?utf-8?q?NcqzkSHc8wITB5Eg4EVsojRmiVU+24syd3zmpbLrdrOzkkLj7fQbKDiUuQLyXTjdk?= =?utf-8?q?dBHOsIYk0gAHQLkTwK9L3el9PSo2dlPfSdOI05mWMK2y8/Dk0nUJVLMen/xmhPEsr?= =?utf-8?q?ub4rmHHJXwFDI2+5P5d1daD5vNnP0fFpSkfJvmdg/YjuyaRqJT5LW2KRr+inFhfms?= =?utf-8?q?5OJI016GIGU4FWUbtNUn3sy/xrY31sZAWH6bqL87XPqwes332bhNsnUCjfylFMnBA?= =?utf-8?q?PivhTRx+b+tc8rg7Pz80O3wGY/4TRphkELNtfD0hBUuyLaKLZOimDxHxS0ZsSLACx?= =?utf-8?q?IO1W1p+ekC7a7eg2SFP3jNwDpWoyJ7Tt5bHh3XBegG2DopfzkbZnF+miUXW4h9XHB?= =?utf-8?q?lr7R3P7jItPs8cLC8mIbcwgSyTV/08TG/q4PptISP6Z0JyedpmdsTtqBS7hp8UU5G?= =?utf-8?q?sqDseUgpg8GSNGcCK9YBzEopxLOOlzW5vPxdpsO6WDIFy18iJjatqke3xNehOOb7B?= =?utf-8?q?47IRNDTh9lJP3ELxwNTpds+FbOZO/0NN+Bj2aN6JWTtJ8lsRCiJ8vhcjUA1skTrZo?= =?utf-8?q?rbjRq1fPYwY2Tf3F3rXUom1Xw1M5fQgBFXyr9UNt2NYETYnOuPl1+R9Ob/v/L57Wm?= =?utf-8?q?F2AEK2nQJ6BY0dljCjWx7v9sRtcDCCLbq1oxQeiACIOiPKYl9RJ5FWPwD4BWA+3vN?= =?utf-8?q?Hmox36zwC6lcoQAJ8hCRCIg3/xW7PmNL41wzM3ATlWKtVZ05VZwGZZqk5BRs8TTXX?= =?utf-8?q?ex6DPifuseJnSuZRZfvAAEqoVryWjngtZX7fXkaOp3hJ/6fvJHbsDcBFBbmpO9Czc?= =?utf-8?q?MG8EdP/l9b/zilJDYHlJvhADp3RQuG4Cp//K74IhGaetxcAytK4zg+8PRAo2Ypw9Q?= =?utf-8?q?lGE9/5DtPYL5hZHKXzPvR5UF927q8Nx/Y/smBfYOhAwxvowp5y3CE5usE3MbiYNqx?= =?utf-8?q?ZcKtP0KkujD2z6Smvhp0bVMFEPUary50N9L/++Q2RmLDZboWgonVU55U+LtHNkNaF?= =?utf-8?q?L4ZGOPzajO9nBU/dhpIvoJiBK6ytV/VBbLpveD61e/8xktDE/V26594Jdzm5eK0Pa?= =?utf-8?q?1inWWx4wsJIqdA18UkApU7ob+KU85QfdePUGVF6s1yP0ouGZ201+wXg/4Qf4W0WMq?= =?utf-8?q?PS1U2lr3gaBYeTtmK3Skp9WPNyNUfOK7dQ=3D=3D?= X-OriginatorOrg: corigine.com X-MS-Exchange-CrossTenant-Network-Message-Id: 18711a61-44d8-4734-f445-08da9f8cc5e9 X-MS-Exchange-CrossTenant-AuthSource: SJ0PR13MB5545.namprd13.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 26 Sep 2022 07:00:19.2122 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: fe128f2c-073b-4c20-818e-7246a585940c X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: LMtKTqM72FuAfrpGSr0JdMh4EsJtijYYnD06CIxzf9UwgJ0aXMWlxaq45tkJjPvsCsZPPqBkTRbkshGEQDmO4m8jfM4qaULBt+NcwMQBpFY= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR13MB4893 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org The NFP card can load different application firmwares. Currently only the CoreNIC application firmware is supported. This commit makes needed infrastructure changes in order to support other application firmwares too. Clearer separation is made between the PF device and any application firmware specific concepts. The PF struct is now generic regardless of the application firmware loaded. A new struct is also made for the CoreNIC application firmware. Future additions to support other application firmwares should also add an application firmware specific struct. Signed-off-by: Chaoyong He Signed-off-by: Heinrich Kuhn Reviewed-by: Niklas Söderlund --- drivers/net/nfp/nfp_common.h | 33 +++++++- drivers/net/nfp/nfp_ethdev.c | 194 +++++++++++++++++++++++++++---------------- 2 files changed, 153 insertions(+), 74 deletions(-) diff --git a/drivers/net/nfp/nfp_common.h b/drivers/net/nfp/nfp_common.h index 6ceb7e9..d9548d6 100644 --- a/drivers/net/nfp/nfp_common.h +++ b/drivers/net/nfp/nfp_common.h @@ -113,6 +113,11 @@ #include #include +/* Firmware application ID's */ +enum nfp_app_fw_id { + NFP_APP_FW_CORE_NIC = 0x1, +}; + /* nfp_qcp_ptr - Read or Write Pointer of a queue */ enum nfp_qcp_ptr { NFP_QCP_READ_PTR = 0, @@ -123,8 +128,13 @@ struct nfp_pf_dev { /* Backpointer to associated pci device */ struct rte_pci_device *pci_dev; - /* Array of physical ports belonging to this PF */ - struct nfp_net_hw *ports[NFP_MAX_PHYPORTS]; + enum nfp_app_fw_id app_fw_id; + + /* Pointer to the app running on the PF */ + void *app_fw_priv; + + /* The eth table reported by firmware */ + struct nfp_eth_table *nfp_eth_table; /* Current values for control */ uint32_t ctrl; @@ -153,8 +163,6 @@ struct nfp_pf_dev { struct nfp_cpp_area *msix_area; uint8_t *hw_queues; - uint8_t total_phyports; - bool multiport; union eth_table_entry *eth_table; @@ -163,6 +171,20 @@ struct nfp_pf_dev { uint32_t nfp_cpp_service_id; }; +struct nfp_app_fw_nic { + /* Backpointer to the PF device */ + struct nfp_pf_dev *pf_dev; + + /* + * Array of physical ports belonging to the this CoreNIC app + * This is really a list of vNIC's. One for each physical port + */ + struct nfp_net_hw *ports[NFP_MAX_PHYPORTS]; + + bool multiport; + uint8_t total_phyports; +}; + struct nfp_net_hw { /* Backpointer to the PF this port belongs to */ struct nfp_pf_dev *pf_dev; @@ -426,6 +448,9 @@ int nfp_net_rss_hash_conf_get(struct rte_eth_dev *dev, #define NFP_NET_DEV_PRIVATE_TO_PF(dev_priv)\ (((struct nfp_net_hw *)dev_priv)->pf_dev) +#define NFP_PRIV_TO_APP_FW_NIC(app_fw_priv)\ + ((struct nfp_app_fw_nic *)app_fw_priv) + #endif /* _NFP_COMMON_H_ */ /* * Local variables: diff --git a/drivers/net/nfp/nfp_ethdev.c b/drivers/net/nfp/nfp_ethdev.c index 2e546cf..47ee418 100644 --- a/drivers/net/nfp/nfp_ethdev.c +++ b/drivers/net/nfp/nfp_ethdev.c @@ -39,15 +39,15 @@ #include "nfp_cpp_bridge.h" static int -nfp_net_pf_read_mac(struct nfp_pf_dev *pf_dev, int port) +nfp_net_pf_read_mac(struct nfp_app_fw_nic *app_fw_nic, int port) { struct nfp_eth_table *nfp_eth_table; struct nfp_net_hw *hw = NULL; /* Grab a pointer to the correct physical port */ - hw = pf_dev->ports[port]; + hw = app_fw_nic->ports[port]; - nfp_eth_table = nfp_eth_read_ports(pf_dev->cpp); + nfp_eth_table = nfp_eth_read_ports(app_fw_nic->pf_dev->cpp); nfp_eth_copy_mac((uint8_t *)&hw->mac_addr, (uint8_t *)&nfp_eth_table->ports[port].mac_addr); @@ -64,6 +64,7 @@ uint32_t new_ctrl, update = 0; struct nfp_net_hw *hw; struct nfp_pf_dev *pf_dev; + struct nfp_app_fw_nic *app_fw_nic; struct rte_eth_conf *dev_conf; struct rte_eth_rxmode *rxmode; uint32_t intr_vector; @@ -71,6 +72,7 @@ hw = NFP_NET_DEV_PRIVATE_TO_HW(dev->data->dev_private); pf_dev = NFP_NET_DEV_PRIVATE_TO_PF(dev->data->dev_private); + app_fw_nic = NFP_PRIV_TO_APP_FW_NIC(pf_dev->app_fw_priv); PMD_INIT_LOG(DEBUG, "Start"); @@ -82,7 +84,7 @@ /* check and configure queue intr-vector mapping */ if (dev->data->dev_conf.intr_conf.rxq != 0) { - if (pf_dev->multiport) { + if (app_fw_nic->multiport) { PMD_INIT_LOG(ERR, "PMD rx interrupt is not supported " "with NFP multiport PF"); return -EINVAL; @@ -250,6 +252,7 @@ struct nfp_net_hw *hw; struct rte_pci_device *pci_dev; struct nfp_pf_dev *pf_dev; + struct nfp_app_fw_nic *app_fw_nic; int i; if (rte_eal_process_type() != RTE_PROC_PRIMARY) @@ -260,6 +263,7 @@ pf_dev = NFP_NET_DEV_PRIVATE_TO_PF(dev->data->dev_private); hw = NFP_NET_DEV_PRIVATE_TO_HW(dev->data->dev_private); pci_dev = RTE_ETH_DEV_TO_PCI(dev); + app_fw_nic = NFP_PRIV_TO_APP_FW_NIC(pf_dev->app_fw_priv); /* * We assume that the DPDK application is stopping all the @@ -280,12 +284,12 @@ /* Only free PF resources after all physical ports have been closed */ /* Mark this port as unused and free device priv resources*/ nn_cfg_writeb(hw, NFP_NET_CFG_LSC, 0xff); - pf_dev->ports[hw->idx] = NULL; + app_fw_nic->ports[hw->idx] = NULL; rte_eth_dev_release_port(dev); - for (i = 0; i < pf_dev->total_phyports; i++) { + for (i = 0; i < app_fw_nic->total_phyports; i++) { /* Check to see if ports are still in use */ - if (pf_dev->ports[i]) + if (app_fw_nic->ports[i]) return 0; } @@ -296,6 +300,7 @@ free(pf_dev->hwinfo); free(pf_dev->sym_tbl); nfp_cpp_free(pf_dev->cpp); + rte_free(app_fw_nic); rte_free(pf_dev); rte_intr_disable(pci_dev->intr_handle); @@ -404,6 +409,7 @@ { struct rte_pci_device *pci_dev; struct nfp_pf_dev *pf_dev; + struct nfp_app_fw_nic *app_fw_nic; struct nfp_net_hw *hw; struct rte_ether_addr *tmp_ether_addr; uint64_t rx_bar_off = 0; @@ -420,6 +426,9 @@ /* Use backpointer here to the PF of this eth_dev */ pf_dev = NFP_NET_DEV_PRIVATE_TO_PF(eth_dev->data->dev_private); + /* Use backpointer to the CoreNIC app struct */ + app_fw_nic = NFP_PRIV_TO_APP_FW_NIC(pf_dev->app_fw_priv); + /* NFP can not handle DMA addresses requiring more than 40 bits */ if (rte_mem_check_dma_mask(40)) { RTE_LOG(ERR, PMD, @@ -438,7 +447,7 @@ * Use PF array of physical ports to get pointer to * this specific port */ - hw = pf_dev->ports[port]; + hw = app_fw_nic->ports[port]; PMD_INIT_LOG(DEBUG, "Working with physical port number: %d, " "NFP internal port number: %d", port, hw->nfp_idx); @@ -568,7 +577,7 @@ goto dev_err_queues_map; } - nfp_net_pf_read_mac(pf_dev, port); + nfp_net_pf_read_mac(app_fw_nic, port); nfp_net_write_mac(hw, (uint8_t *)&hw->mac_addr); tmp_ether_addr = (struct rte_ether_addr *)&hw->mac_addr; @@ -720,25 +729,68 @@ } static int -nfp_init_phyports(struct nfp_pf_dev *pf_dev) +nfp_init_app_fw_nic(struct nfp_pf_dev *pf_dev) { int i; - int ret = 0; + int ret; + int err = 0; + int total_vnics; struct nfp_net_hw *hw; + unsigned int numa_node; struct rte_eth_dev *eth_dev; + struct nfp_app_fw_nic *app_fw_nic; struct nfp_eth_table *nfp_eth_table; + char port_name[RTE_ETH_NAME_MAX_LEN]; - nfp_eth_table = nfp_eth_read_ports(pf_dev->cpp); - if (nfp_eth_table == NULL) { - PMD_INIT_LOG(ERR, "Error reading NFP ethernet table"); - return -EIO; + nfp_eth_table = pf_dev->nfp_eth_table; + PMD_INIT_LOG(INFO, "Total physical ports: %d", nfp_eth_table->count); + + /* Allocate memory for the CoreNIC app */ + app_fw_nic = rte_zmalloc("nfp_app_fw_nic", sizeof(*app_fw_nic), 0); + if (app_fw_nic == NULL) + return -ENOMEM; + + /* Point the app_fw_priv pointer in the PF to the coreNIC app */ + pf_dev->app_fw_priv = app_fw_nic; + + /* Read the number of vNIC's created for the PF */ + total_vnics = nfp_rtsym_read_le(pf_dev->sym_tbl, "nfd_cfg_pf0_num_ports", &err); + if (err != 0 || total_vnics <= 0 || total_vnics > 8) { + PMD_INIT_LOG(ERR, "nfd_cfg_pf0_num_ports symbol with wrong value"); + ret = -ENODEV; + goto app_cleanup; } - /* Loop through all physical ports on PF */ - for (i = 0; i < pf_dev->total_phyports; i++) { - const unsigned int numa_node = rte_socket_id(); - char port_name[RTE_ETH_NAME_MAX_LEN]; + /* + * For coreNIC the number of vNICs exposed should be the same as the + * number of physical ports + */ + if (total_vnics != (int)nfp_eth_table->count) { + PMD_INIT_LOG(ERR, "Total physical ports do not match number of vNICs"); + ret = -ENODEV; + goto app_cleanup; + } + + /* Populate coreNIC app properties*/ + app_fw_nic->total_phyports = total_vnics; + app_fw_nic->pf_dev = pf_dev; + if (total_vnics > 1) + app_fw_nic->multiport = true; + + /* Map the symbol table */ + pf_dev->ctrl_bar = nfp_rtsym_map(pf_dev->sym_tbl, "_pf0_net_bar0", + app_fw_nic->total_phyports * 32768, &pf_dev->ctrl_area); + if (pf_dev->ctrl_bar == NULL) { + PMD_INIT_LOG(ERR, "nfp_rtsym_map fails for _pf0_net_ctrl_bar"); + ret = -EIO; + goto app_cleanup; + } + + PMD_INIT_LOG(DEBUG, "ctrl bar: %p", pf_dev->ctrl_bar); + /* Loop through all physical ports on PF */ + numa_node = rte_socket_id(); + for (i = 0; i < app_fw_nic->total_phyports; i++) { snprintf(port_name, sizeof(port_name), "%s_port%d", pf_dev->pci_dev->device.name, i); @@ -762,7 +814,7 @@ hw = NFP_NET_DEV_PRIVATE_TO_HW(eth_dev->data->dev_private); /* Add this device to the PF's array of physical ports */ - pf_dev->ports[i] = hw; + app_fw_nic->ports[i] = hw; hw->pf_dev = pf_dev; hw->cpp = pf_dev->cpp; @@ -785,20 +837,21 @@ rte_eth_dev_probing_finish(eth_dev); } /* End loop, all ports on this PF */ - ret = 0; - goto eth_table_cleanup; + + return 0; port_cleanup: - for (i = 0; i < pf_dev->total_phyports; i++) { - if (pf_dev->ports[i] && pf_dev->ports[i]->eth_dev) { + for (i = 0; i < app_fw_nic->total_phyports; i++) { + if (app_fw_nic->ports[i] && app_fw_nic->ports[i]->eth_dev) { struct rte_eth_dev *tmp_dev; - tmp_dev = pf_dev->ports[i]->eth_dev; + tmp_dev = app_fw_nic->ports[i]->eth_dev; rte_eth_dev_release_port(tmp_dev); - pf_dev->ports[i] = NULL; + app_fw_nic->ports[i] = NULL; } } -eth_table_cleanup: - free(nfp_eth_table); + nfp_cpp_area_free(pf_dev->ctrl_area); +app_cleanup: + rte_free(app_fw_nic); return ret; } @@ -806,11 +859,11 @@ static int nfp_pf_init(struct rte_pci_device *pci_dev) { - int err; - int ret = 0; + int ret; + int err = 0; uint64_t addr; - int total_ports; struct nfp_cpp *cpp; + enum nfp_app_fw_id app_fw_id; struct nfp_pf_dev *pf_dev; struct nfp_hwinfo *hwinfo; char name[RTE_ETH_NAME_MAX_LEN]; @@ -842,9 +895,10 @@ if (hwinfo == NULL) { PMD_INIT_LOG(ERR, "Error reading hwinfo table"); ret = -EIO; - goto error; + goto cpp_cleanup; } + /* Read the number of physical ports from hardware */ nfp_eth_table = nfp_eth_read_ports(cpp); if (nfp_eth_table == NULL) { PMD_INIT_LOG(ERR, "Error reading NFP ethernet table"); @@ -867,20 +921,14 @@ goto eth_table_cleanup; } - total_ports = nfp_rtsym_read_le(sym_tbl, "nfd_cfg_pf0_num_ports", &err); - if (total_ports != (int)nfp_eth_table->count) { - PMD_DRV_LOG(ERR, "Inconsistent number of ports"); + /* Read the app ID of the firmware loaded */ + app_fw_id = nfp_rtsym_read_le(sym_tbl, "_pf0_net_app_id", &err); + if (err != 0) { + PMD_INIT_LOG(ERR, "Couldn't read app_fw_id from fw"); ret = -EIO; goto sym_tbl_cleanup; } - PMD_INIT_LOG(INFO, "Total physical ports: %d", total_ports); - - if (total_ports <= 0 || total_ports > 8) { - PMD_INIT_LOG(ERR, "nfd_cfg_pf0_num_ports symbol with wrong value"); - ret = -ENODEV; - goto sym_tbl_cleanup; - } /* Allocate memory for the PF "device" */ snprintf(name, sizeof(name), "nfp_pf%d", 0); pf_dev = rte_zmalloc(name, sizeof(*pf_dev), 0); @@ -890,26 +938,12 @@ } /* Populate the newly created PF device */ + pf_dev->app_fw_id = app_fw_id; pf_dev->cpp = cpp; pf_dev->hwinfo = hwinfo; pf_dev->sym_tbl = sym_tbl; - pf_dev->total_phyports = total_ports; - - if (total_ports > 1) - pf_dev->multiport = true; - pf_dev->pci_dev = pci_dev; - - /* Map the symbol table */ - pf_dev->ctrl_bar = nfp_rtsym_map(pf_dev->sym_tbl, "_pf0_net_bar0", - pf_dev->total_phyports * 32768, &pf_dev->ctrl_area); - if (pf_dev->ctrl_bar == NULL) { - PMD_INIT_LOG(ERR, "nfp_rtsym_map fails for _pf0_net_ctrl_bar"); - ret = -EIO; - goto pf_cleanup; - } - - PMD_INIT_LOG(DEBUG, "ctrl bar: %p", pf_dev->ctrl_bar); + pf_dev->nfp_eth_table = nfp_eth_table; /* configure access to tx/rx vNIC BARs */ switch (pci_dev->id.device_id) { @@ -925,7 +959,8 @@ default: PMD_INIT_LOG(ERR, "nfp_net: no device ID matching"); err = -ENODEV; - goto ctrl_area_cleanup; + ret = -ENODEV; + goto pf_cleanup; } pf_dev->hw_queues = nfp_cpp_map_area(pf_dev->cpp, 0, 0, @@ -934,18 +969,27 @@ if (pf_dev->hw_queues == NULL) { PMD_INIT_LOG(ERR, "nfp_rtsym_map fails for net.qc"); ret = -EIO; - goto ctrl_area_cleanup; + goto pf_cleanup; } PMD_INIT_LOG(DEBUG, "tx/rx bar address: 0x%p", pf_dev->hw_queues); /* - * Initialize and prep physical ports now - * This will loop through all physical ports + * PF initialization has been done at this point. Call app specific + * init code now */ - ret = nfp_init_phyports(pf_dev); - if (ret) { - PMD_INIT_LOG(ERR, "Could not create physical ports"); + switch (pf_dev->app_fw_id) { + case NFP_APP_FW_CORE_NIC: + PMD_INIT_LOG(INFO, "Initializing coreNIC"); + ret = nfp_init_app_fw_nic(pf_dev); + if (ret != 0) { + PMD_INIT_LOG(ERR, "Could not initialize coreNIC!"); + goto hwqueues_cleanup; + } + break; + default: + PMD_INIT_LOG(ERR, "Unsupported Firmware loaded"); + ret = -EINVAL; goto hwqueues_cleanup; } @@ -956,8 +1000,6 @@ hwqueues_cleanup: nfp_cpp_area_free(pf_dev->hwqueues_area); -ctrl_area_cleanup: - nfp_cpp_area_free(pf_dev->ctrl_area); pf_cleanup: rte_free(pf_dev); sym_tbl_cleanup: @@ -966,6 +1008,8 @@ free(nfp_eth_table); hwinfo_cleanup: free(hwinfo); +cpp_cleanup: + nfp_cpp_free(cpp); error: return ret; } @@ -979,7 +1023,8 @@ nfp_pf_secondary_init(struct rte_pci_device *pci_dev) { int i; - int err; + int err = 0; + int ret = 0; int total_ports; struct nfp_cpp *cpp; struct nfp_net_hw *hw; @@ -1017,6 +1062,11 @@ } total_ports = nfp_rtsym_read_le(sym_tbl, "nfd_cfg_pf0_num_ports", &err); + if (err != 0 || total_ports <= 0 || total_ports > 8) { + PMD_INIT_LOG(ERR, "nfd_cfg_pf0_num_ports symbol with wrong value"); + ret = -ENODEV; + goto sym_tbl_cleanup; + } for (i = 0; i < total_ports; i++) { struct rte_eth_dev *eth_dev; @@ -1030,7 +1080,8 @@ if (eth_dev == NULL) { RTE_LOG(ERR, EAL, "secondary process attach failed, ethdev doesn't exist"); - return -ENODEV; + ret = -ENODEV; + break; } hw = NFP_NET_DEV_PRIVATE_TO_HW(eth_dev->data->dev_private); @@ -1046,7 +1097,10 @@ /* Register the CPP bridge service for the secondary too */ nfp_register_cpp_service(cpp); - return 0; +sym_tbl_cleanup: + free(sym_tbl); + + return ret; } static int From patchwork Mon Sep 26 06:59:46 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Chaoyong He X-Patchwork-Id: 116843 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 112DEA00C2; Mon, 26 Sep 2022 09:00:32 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id E8364415D7; Mon, 26 Sep 2022 09:00:24 +0200 (CEST) Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2092.outbound.protection.outlook.com [40.107.244.92]) by mails.dpdk.org (Postfix) with ESMTP id CFE2A41133 for ; Mon, 26 Sep 2022 09:00:22 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=crhUBixuhoZEMJ0fxHJCkypdPxMXinjeC+9mNMjD7f3qP6qoHHRSURgO6SugJlRr+wwwj0vjA4JzAY4lwWiRDhDKwdjQEWMyYVSv39UR8v5YcpUa8sved1qDP3J2updQl9nHMs788ThtidJpDYZeaJ0BlJiRN1IryQROBDGUu39GkTVyic4s6yo9NOiyO9haZNWCLDMqZgzHAMfG3g/zXpzzugJLwlFuGnKAE8aFHveI0GN8GbwhOuYpazQEl3hVonTrFRL6iOgOKErGGcNjSrKFw2b1AJ6ASeEKCc1eSPXskxmXJVUPay0gC36XjKqLwam+zgtWifQrccAudbTypg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=ikUdERG8KQObCNmcGR3HIW7FUmEAJQPS2ukxe60dbwg=; b=FMkAP9AWwHDYLAqNiQ6nOtaKVanoaxb5w+9mNTaabVB1yr+YgzYONr7QeAZ0F1RPwCLSXxf8XhQ/g5Z48IY2SzlVkT7EUxu6r06DRVcdEm5G7DxcnjFUb+X5WTZ1DNb0bLJcn1zq8X27CkUvL86F6i/wLs0BCM3An03FngUUjZvxLsX3MxES3F1OCiHNbzn5BERkEFU4jLdH+y78yntAy5jrkneARfp3wn0/YEUYvEG6W54Zar4S0ai3pNe4KShk6aHpMy6mzdBSvlzrB93QTxv+GOFA3y0EPtZlcg76XUvp9DJdUSObaGGHLcfspGkskmbToX0GGlf8xE6eRbFnIw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=corigine.com; dmarc=pass action=none header.from=corigine.com; dkim=pass header.d=corigine.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=corigine.onmicrosoft.com; s=selector2-corigine-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ikUdERG8KQObCNmcGR3HIW7FUmEAJQPS2ukxe60dbwg=; b=Z8j8rFy3e8bcT4Bkr0S3pwdk5ja2tAWll+g/4tgL2hXF1E7IGPX/Yt8nYEX7vd/g1E9+AeBYQYyG4RgzRCwYFv26AS0/sM9WVGS4Wmt7V7aFkyLP2vgUqE59QVbBCJzFGMuOhteXouNw6f6ZWPeQrZwWejntVtBaVPB4Ad+FLjU= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=corigine.com; Received: from SJ0PR13MB5545.namprd13.prod.outlook.com (2603:10b6:a03:424::5) by SA1PR13MB4893.namprd13.prod.outlook.com (2603:10b6:806:18a::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5676.14; Mon, 26 Sep 2022 07:00:21 +0000 Received: from SJ0PR13MB5545.namprd13.prod.outlook.com ([fe80::779:2722:a8e5:503b]) by SJ0PR13MB5545.namprd13.prod.outlook.com ([fe80::779:2722:a8e5:503b%7]) with mapi id 15.20.5676.014; Mon, 26 Sep 2022 07:00:21 +0000 From: Chaoyong He To: dev@dpdk.org Cc: oss-drivers@corigine.com, niklas.soderlund@corigine.com, Chaoyong He Subject: [PATCH v10 02/13] net/nfp: simplify initialization and remove dead code Date: Mon, 26 Sep 2022 14:59:46 +0800 Message-Id: <1664175597-37248-3-git-send-email-chaoyong.he@corigine.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1664175597-37248-1-git-send-email-chaoyong.he@corigine.com> References: <1664175597-37248-1-git-send-email-chaoyong.he@corigine.com> X-ClientProxiedBy: SG2P153CA0005.APCP153.PROD.OUTLOOK.COM (2603:1096::15) To SJ0PR13MB5545.namprd13.prod.outlook.com (2603:10b6:a03:424::5) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SJ0PR13MB5545:EE_|SA1PR13MB4893:EE_ X-MS-Office365-Filtering-Correlation-Id: 88525d89-c20d-4574-59df-08da9f8cc70f X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 4csSskRWnKg0YVtXVUKxXhflC+Dv0ptePC7JtQ8xuVKKS2c6clZ/lvruECDKLw0VGDEow0omE+Byr4qsB3B21e89FCIPc7ovzUQXLbYw4sO/66vy4zMw4w8wveTXbEhx3tcV0pJUPld4mM96RGGnqUltLvah+EoBXj7oB7lM1T1pNxVR3sQHc/eHxkjOyD5N/+J0RsA2w+arWvmLjJ/jB0VYCMGDpIkhueI85p0SP0ky8hf858T+PdIC7jQGHb1OeH2um2cTioAelGuNiSru1csLvAF8M1bCd7QYE8+/LsPKVviH1ov4OpQdvoD87OIrbcr2DIvxqNzfRkBlqENqdG3ZaTj9JQqXf6jVjEyFxIlugOscfzbpEVce7lwz6u1xkxzVQgu5kHoIqWjjM0dMBgBv2yunoaOzwbPwrfYlx1ZJMwvLE6EyePsbT3/o49lPZdx2WXT/kMcnwf5h/CdLR1mZyGhFwdptjgVpfpo/THLONt8vxfK1XCNlyiXJKpaX7PuCCZmJlXZqdt/Wt3lEO9oaOz9tMh1nidOq/yO1Q3zbSxxVkMEW8ydqgl18GIbiRC1/aRDH4xI6LHIEv0DO1U5Rj8nPOtLUZ99h4Sa1T9FHtjfOE+tf8GrYT+cCtaQSbRmuSooC4rpDhUu//UiWQTqHkr/Kpy42MXML3MHWXlWiqYVJ1gjb/y0JThTM8U7XFO6JnBNl2X42OX2ntk0tkVPcbV5+vH0i+kUiMDdp6mnX0odYId1Lkyku2rXhj5zDjAyovrQXp4dcwa00iFhIog== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SJ0PR13MB5545.namprd13.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230022)(4636009)(366004)(396003)(39830400003)(136003)(376002)(346002)(451199015)(36756003)(38100700002)(38350700002)(2906002)(86362001)(44832011)(66946007)(8936002)(66556008)(66476007)(5660300002)(8676002)(4326008)(83380400001)(186003)(6512007)(6916009)(316002)(52116002)(6666004)(2616005)(26005)(6506007)(478600001)(41300700001)(107886003)(6486002); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?OfaW7B/KAbxztgVnNHSR4Cjaw+vk?= =?utf-8?q?LiEKBAs6MHA6iWeh/U4+tEWrl5OpAJ2AyWC/S0MzhHZ5grm1R/F3d+qxqVxwDfmyt?= =?utf-8?q?Ri1wknwuyDdVD6KP8ForNlV1ppaeluZcUfiSgv/5NTUnwb9lrixpCN+REm7XfKiDB?= =?utf-8?q?R25Z5Z5764KeqU+nrbF0q/EDwbif2NzHsHm8JaPfx+XM5v9P8As+UD9HNmhJCz74C?= =?utf-8?q?wbi6G6JoOAZgC1NTXW265bobdyn9QOXFbzZYwuhnh3POvPxcMvPwKxo2o3s54FHdb?= =?utf-8?q?x6lsA4Cvd55oJVj72ay/nic4VSM+Dn4Dk4lp79wbmqLuMOzOF5WcP1LNgaMizXoAr?= =?utf-8?q?hEnPqozjtCOmaNVWeAHgHYnAQ4PKm9sF2xFiOclGPXqmDw4z5bePEvRywwkxrZH5/?= =?utf-8?q?9UdFN7v+5D0ni2D70dQXNxn1I4egwx9sEvAU3QGwqooEQif7Uv2a3Q8lhhq2Iv57G?= =?utf-8?q?QG8XcDxElhqZskRya7E5DdCBV9MPlaW4tmBUG4t3qNp7dhpKCWQ9NwFBI7q5f7ux8?= =?utf-8?q?o6Gr+jMLVEodhKNvG8ALQ3drH7oiljRbO8asgKG4M5huk91Z85rtI0HCCw/wlA8+4?= =?utf-8?q?pSF+CnbWb2roaqxpfVDVT/NhNBD8AE/0iFiN+rkDbcCm80hMtxuYTAYICumtZlp4F?= =?utf-8?q?DJ131hwpMk87TqFrVv6aaIOrt+LspTyZsWGRh6AqvM6nTthlxYdYnlQq1TsLS3f6f?= =?utf-8?q?lU3z/f/XNmJZ0H8mPCGuzi21OcYDJD0DBqkclz4fKtz4vo1xBFaM91oBZNDcMUVMJ?= =?utf-8?q?+fd1jecJ1FYrrrL3VzqGvVbAytq8+cm1VrPJIUBemO3ybZYwDwXwzM4Y8YL9iSskQ?= =?utf-8?q?zZgvZswvVBnYcdr/kCDzjeARkMBgMAh9Moh04XmNeRoQLqQsIT6VQ9y1QhrhPUhKk?= =?utf-8?q?/0cYj8h+Kifr8df+nKb3FqFwHOfNvBFNmhc3KFW+c2F+lUJgInfHzjy7eGpe1rewH?= =?utf-8?q?y8j4nDaL/HxWpgp+S8fLcYwVuyA70ZxpzJE0K9SNnkNwMys8F2ZJOzZnDwJwW7soQ?= =?utf-8?q?tAv9eb2FzbIWAtwmEz0qDP2UQCWOC/QSKrnA3GCOinp4GON7UAZnro9ib/YU7JxsB?= =?utf-8?q?3kNhL9Pr65DvXp4L+bKB1aEIlo2ceV6Q8FaPmvk7jQije6O7xKESoqXLltxMrDEPg?= =?utf-8?q?h0bhN94PG67sHltnbCKxRI420JVIZHAHnM57cUt0teqJvxYdql+kz92RtF+j/AQrw?= =?utf-8?q?bHbGlYh9VwGONwqcrBGKn5vaNTUf/sZykUgkrtydqj3L4xd8YAbDqaKhNWG8KC91A?= =?utf-8?q?9kiYhDKxeMtBv1Rm+qCt+dxAz1VRZ394EhGJHkOgn9lQR0y0F7Ds+roYpSEiiQaB6?= =?utf-8?q?egSzVKiuSv/Yt/L6W1Lo4TQT1I9Y8PjtmuCisraqiZQKeOk1bWp0cnzQ+wlMKvKu0?= =?utf-8?q?YzjJXOgTlgda4WOz4PaIlKU8Koh0w0jUK0Y6qSQ8/467u3mcXKzHkzrtxmsRGWkl8?= =?utf-8?q?ODJL8pFraB+n1TvHPMRXYU8BPBWHKJTIsgzCVp1Ibk7F9YnoU1FurOlIghYmTXpRH?= =?utf-8?q?XeRbfvSyY9YorM2rETnvpVZoJOrwSdoQBA=3D=3D?= X-OriginatorOrg: corigine.com X-MS-Exchange-CrossTenant-Network-Message-Id: 88525d89-c20d-4574-59df-08da9f8cc70f X-MS-Exchange-CrossTenant-AuthSource: SJ0PR13MB5545.namprd13.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 26 Sep 2022 07:00:21.3225 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: fe128f2c-073b-4c20-818e-7246a585940c X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: jeNBk/EKy17hmYVIIjyvKV9N1elORUILPzwjG73L2X7CPhxJd8RKukhzYgQpSBXcgGimtcEHvyK4j3txWnsJ0R69YgYdHqZDF4DFERH6MRs= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR13MB4893 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Calling nfp_net_init() is only done for the corenic firmware flavor and it is guaranteed to always be called from the primary process, so the explicit check for RTE_PROC_PRIMARY can be dropped. The calling graph of nfp_net_init() already guaranteed the free of resources when it fail, so remove the necessary free logics inside it. While at it remove the unused member is_phyport from struct nfp_net_hw. Signed-off-by: Chaoyong He Reviewed-by: Niklas Söderlund --- drivers/net/nfp/nfp_common.h | 4 ---- drivers/net/nfp/nfp_ethdev.c | 40 +++++++++++----------------------------- 2 files changed, 11 insertions(+), 33 deletions(-) diff --git a/drivers/net/nfp/nfp_common.h b/drivers/net/nfp/nfp_common.h index d9548d6..5bdf028 100644 --- a/drivers/net/nfp/nfp_common.h +++ b/drivers/net/nfp/nfp_common.h @@ -168,7 +168,6 @@ struct nfp_pf_dev { struct nfp_hwinfo *hwinfo; struct nfp_rtsym_table *sym_tbl; - uint32_t nfp_cpp_service_id; }; struct nfp_app_fw_nic { @@ -240,11 +239,8 @@ struct nfp_net_hw { uint8_t idx; /* Internal port number as seen from NFP */ uint8_t nfp_idx; - bool is_phyport; union eth_table_entry *eth_table; - - uint32_t nfp_cpp_service_id; }; struct nfp_net_adapter { diff --git a/drivers/net/nfp/nfp_ethdev.c b/drivers/net/nfp/nfp_ethdev.c index 47ee418..14c6d69 100644 --- a/drivers/net/nfp/nfp_ethdev.c +++ b/drivers/net/nfp/nfp_ethdev.c @@ -417,7 +417,6 @@ uint32_t start_q; int stride = 4; int port = 0; - int err; PMD_INIT_FUNC_TRACE(); @@ -452,10 +451,6 @@ PMD_INIT_LOG(DEBUG, "Working with physical port number: %d, " "NFP internal port number: %d", port, hw->nfp_idx); - /* For secondary processes, the primary has done all the work */ - if (rte_eal_process_type() != RTE_PROC_PRIMARY) - return 0; - rte_eth_copy_pci_info(eth_dev, pci_dev); hw->device_id = pci_dev->id.device_id; @@ -506,8 +501,7 @@ break; default: PMD_DRV_LOG(ERR, "nfp_net: no device ID matching"); - err = -ENODEV; - goto dev_err_ctrl_map; + return -ENODEV; } PMD_INIT_LOG(DEBUG, "tx_bar_off: 0x%" PRIx64 "", tx_bar_off); @@ -573,8 +567,7 @@ RTE_ETHER_ADDR_LEN, 0); if (eth_dev->data->mac_addrs == NULL) { PMD_INIT_LOG(ERR, "Failed to space for MAC address"); - err = -ENOMEM; - goto dev_err_queues_map; + return -ENOMEM; } nfp_net_pf_read_mac(app_fw_nic, port); @@ -604,24 +597,15 @@ hw->mac_addr[0], hw->mac_addr[1], hw->mac_addr[2], hw->mac_addr[3], hw->mac_addr[4], hw->mac_addr[5]); - if (rte_eal_process_type() == RTE_PROC_PRIMARY) { - /* Registering LSC interrupt handler */ - rte_intr_callback_register(pci_dev->intr_handle, - nfp_net_dev_interrupt_handler, (void *)eth_dev); - /* Telling the firmware about the LSC interrupt entry */ - nn_cfg_writeb(hw, NFP_NET_CFG_LSC, NFP_NET_IRQ_LSC_IDX); - /* Recording current stats counters values */ - nfp_net_stats_reset(eth_dev); - } + /* Registering LSC interrupt handler */ + rte_intr_callback_register(pci_dev->intr_handle, + nfp_net_dev_interrupt_handler, (void *)eth_dev); + /* Telling the firmware about the LSC interrupt entry */ + nn_cfg_writeb(hw, NFP_NET_CFG_LSC, NFP_NET_IRQ_LSC_IDX); + /* Recording current stats counters values */ + nfp_net_stats_reset(eth_dev); return 0; - -dev_err_queues_map: - nfp_cpp_area_free(hw->hwqueues_area); -dev_err_ctrl_map: - nfp_cpp_area_free(hw->ctrl_area); - - return err; } #define DEFAULT_FW_PATH "/lib/firmware/netronome" @@ -821,7 +805,6 @@ hw->eth_dev = eth_dev; hw->idx = i; hw->nfp_idx = nfp_eth_table->ports[i].index; - hw->is_phyport = true; eth_dev->device = &pf_dev->pci_dev->device; @@ -887,8 +870,7 @@ if (cpp == NULL) { PMD_INIT_LOG(ERR, "A CPP handle can not be obtained"); - ret = -EIO; - goto error; + return -EIO; } hwinfo = nfp_hwinfo_read(cpp); @@ -1010,7 +992,7 @@ free(hwinfo); cpp_cleanup: nfp_cpp_free(cpp); -error: + return ret; } From patchwork Mon Sep 26 06:59:47 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Chaoyong He X-Patchwork-Id: 116844 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 211ACA00C2; Mon, 26 Sep 2022 09:00:42 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 4D63C427F0; Mon, 26 Sep 2022 09:00:28 +0200 (CEST) Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2111.outbound.protection.outlook.com [40.107.244.111]) by mails.dpdk.org (Postfix) with ESMTP id D68B441143 for ; Mon, 26 Sep 2022 09:00:24 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=SelnhHNT257rna7z9XybG5OHTRmIG6NJ43z/kDBRD+slNTe/dNfUMv6JMxpHIjxP+e2pOcIuT3R7417+1MrHxh0a37xR/FtuAkiSZc5I36keVheoQfnesX5bfZ+eXLKBpt6LrWpIlTWNZs8LNFB1c5A/p0jcWvyNV3IXRydcnquyPfSTPT/DxF7hn+jlGGXTYLO/CswzBd1SkFgt3IneEYlEdUqHzzS3kpHbxF4ZYxniZ++WeQZOsRMNZ2a5POXcBNqB8JS1YULYtz6DhKyrWLeX0w65EuyelkEmBMN9BfdxadaWZZAcsOUwSNImK2lNt8uSTumkhMB8prA3T+MnFA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=cL+9gRDq+npF4lcT9e5VMG7tdwhBmtk8YZ/5m4preSo=; b=aEpDM8sO4kbzvQz2YaQA4tIgMBlfZYPmfeBUC4rWbOBlZdQHv7c0SWRInmQsxTXaDPJ023yd4VI/3Pk6W0XawRoZpvtk4PiqsE+/13mPv6Vkv8zNegF1rci1BmJAeaxqE67ZCV2ZYfT1bsVSoTIx6+G0ozYUaCbgRHTd1PDmsZcP0al8Ii8xEl+B3bbXkOTWIQrXLnQilwpWOyn7+L+QsENLePAIPeUhA6X4t3n49hJOBTpF5MvygV6MNri5DqGSq/ATdkr5I2W3Sc0uyGwcYrFoyluDigQU1ERkhR21wUYdOUuGniY20PPnK8k6w70tECoe0XLkmpHFUnFuhZpOrw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=corigine.com; dmarc=pass action=none header.from=corigine.com; dkim=pass header.d=corigine.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=corigine.onmicrosoft.com; s=selector2-corigine-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=cL+9gRDq+npF4lcT9e5VMG7tdwhBmtk8YZ/5m4preSo=; b=O+lvNcJTt9W2BAifKqJg4u4QwBcVn8Dxs6cPU/t/laNLdpVZu6p7LTFn3XQn1iEtkm7gg2/42rM17aduSb0CqRz19gLIAt8LLM6B519+qLYL8YFraVUrE5xse2GkeGEXBk0DES/eRuwyrrG8SGRniIi2mmdEEMS9VaxQ47znCfU= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=corigine.com; Received: from SJ0PR13MB5545.namprd13.prod.outlook.com (2603:10b6:a03:424::5) by SA1PR13MB4893.namprd13.prod.outlook.com (2603:10b6:806:18a::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5676.14; Mon, 26 Sep 2022 07:00:23 +0000 Received: from SJ0PR13MB5545.namprd13.prod.outlook.com ([fe80::779:2722:a8e5:503b]) by SJ0PR13MB5545.namprd13.prod.outlook.com ([fe80::779:2722:a8e5:503b%7]) with mapi id 15.20.5676.014; Mon, 26 Sep 2022 07:00:23 +0000 From: Chaoyong He To: dev@dpdk.org Cc: oss-drivers@corigine.com, niklas.soderlund@corigine.com, Chaoyong He Subject: [PATCH v10 03/13] net/nfp: move app specific init logic to own function Date: Mon, 26 Sep 2022 14:59:47 +0800 Message-Id: <1664175597-37248-4-git-send-email-chaoyong.he@corigine.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1664175597-37248-1-git-send-email-chaoyong.he@corigine.com> References: <1664175597-37248-1-git-send-email-chaoyong.he@corigine.com> X-ClientProxiedBy: SG2P153CA0005.APCP153.PROD.OUTLOOK.COM (2603:1096::15) To SJ0PR13MB5545.namprd13.prod.outlook.com (2603:10b6:a03:424::5) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SJ0PR13MB5545:EE_|SA1PR13MB4893:EE_ X-MS-Office365-Filtering-Correlation-Id: a51722af-2e26-44ec-d574-08da9f8cc847 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: YGjMXApWvEFW4ryvMIc7oJNCi3Fkm8vsdX+uLY2XMumhORFWedQ/Po/4QAccEgYuwQrcNVj0HcUorE+Hg3oEW8LtN3lOLAnaVQgn9b9Hhy7MvrNRswOLDwG/UVq0FX7T7ZCNzyfFpdgO/+9Fo1ZiQNvPfyvGjay5D6tAFiqySsPpUfr8wTCZaU7wE2LaOx14BX7ks59E6Gj1+UMMVjJrQJS75bckV2ptaAO3xxTL+wIuaCzPDizjNSQDxrgUFXdZaXLHiHuBc1UrLZSDBhKltKdRdTFsl2rUKK8BfEg6ZbupVoELu6Ik6IToxbrdr6e4/45Y8jodiyo304rDFbZb23UnLy4fOhjYjvmqmU1p8JaZ/GtjekDhb3of2RvsAXba7bdLaUNd7HvTHjl1w4voeZwCMG/BNk4TqAQGclCnpUDBbxUrMwiTWWklaGlNczeSmGDBVu1RYUEgEPwROUyKWAFC9UdxnH7JgpCczXSQra7fh8i2vAJ7rxN7rzRBCjgRqsz3e05P2ywSMaj7qf6+NafOMwQBNawkJiwHR7eE+7o0uab21lQyavMWYIhPtVVBGCBRZpEJ67/npSx33hvcUg6Om/14E7MA1aukWts0TBA2bsXl5xPHNesPonS44tA6FfJGNn45vuLEaM6LCbvnM0UJDO4Ako1QuuPzRZPfBNfXoAA+udZgZjQCSN/ruGatVJm2JKIifIC3IEOPuDVKXGfrmD1OeDN0m58A8TSAboBdULCMiCcdo126VXHQ9xEaQPx/SPIzC7cphA0+/SYc1g== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SJ0PR13MB5545.namprd13.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230022)(4636009)(366004)(396003)(39830400003)(136003)(376002)(346002)(451199015)(36756003)(38100700002)(38350700002)(2906002)(86362001)(44832011)(66946007)(8936002)(66556008)(66476007)(5660300002)(8676002)(4326008)(83380400001)(186003)(6512007)(6916009)(316002)(52116002)(6666004)(2616005)(26005)(6506007)(478600001)(41300700001)(107886003)(6486002); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?gtFAOoEa8V+EIckH2kT3yZ2+PagN?= =?utf-8?q?BKWORyjIYrr2p2Vmebwq4Vpiuo8f6MpeNu8MPEfTyookhK4l0q+WO/eY1sYDHFAPP?= =?utf-8?q?3q0nvqhjBdzHj/Jy0JZGgMgzOTY9BHwmJu8hkeGJBdR8eP8/MVywRvESbEIYqILHr?= =?utf-8?q?DIVkb5kEeJdVreIx6AffjOF3QK4WMdlozXD6liJfZLYZB3rwSmHwxFuQ6lAcsKImD?= =?utf-8?q?RAH7k7vmJYSWsWj0hIpf+zJpcbUy2XKDKQZNH4WHndxThVT4iev3ShucrpDiY/zfL?= =?utf-8?q?KhoaHqs6/Cr5zryIYThaOBhslX2hl1SNYz8pitCM8Hq6OBXst7Qri5QRzK4+o43EB?= =?utf-8?q?8G/krb/K/arPZQFXIc9qVVoZjl3o+JVBj5oN/J6015crLqyD2LNgmQAlVsFHickWl?= =?utf-8?q?bGAiFdsFOf4ciTmKwYzUEl9KqhIWiYG95YW5oMWYde4PUGJkoC7TRfJ5ZMm+S0KQb?= =?utf-8?q?Yqcjl2vu1Ufa998brYcjCW2DYDW4U2jGMx/5vellrLyXxgNhLglACB1Tf/6b0tMY+?= =?utf-8?q?uthHCKAYmXC1c+xRR0YDYig33SDo/ldHvNGMA0ajLBKmM/LgNNZrB+m2pluDsEJ+4?= =?utf-8?q?C1wRCQONfR+H6WJ4VtBBRyhRhU6SRan9yfbuhcpZwZBOyn0ubeoCspsCz6tFlFWJU?= =?utf-8?q?9itCnX8Z3nDuNuqc/q1O+orBLCLUtD8h/DTVlmkAmnpy0WAvaQBS1QXjgUSr6AalE?= =?utf-8?q?/ktf2zju+2tTGwFTPumZ3ogFQqVdHYFSA4w0YBRx+gEatxwSTM1TY+ixXdGnfEtQS?= =?utf-8?q?pcmXZDZdQtwGhLjT2SzxtMjDitAmze2U9Yf34I92MTt8U8U2TbIxPnTm/phtLmOl7?= =?utf-8?q?Sw/1Hnuek0RwRvcwEo+Nasj0x+FaZUOPBY70bEWZzAdVztoyUECDbBDv+0iBmPoj8?= =?utf-8?q?I5Ram/50JDV8wfjCKulw8AMLp/VkwM4jPtZAkiFoCUKlawXpNyXHLSYW4+0Hcq3g2?= =?utf-8?q?JCMcUkd1JsDwqz9ZOTH41ffZuNvpst/pD+wuQF7uyBF+5P3L26pZ/1xdm4CnCuY0f?= =?utf-8?q?ivZyE0zrublrKbWWbrO+Llb/9KihvMv/VyzFA3kqlrvIKVVnZUF3KkAawbcJIUWJ4?= =?utf-8?q?GXihoRbo7P6RmyxV3dGGUzatplhi4eVNJT3QP8dH9yJ8dMCNAvDQjn89If7MZWQCN?= =?utf-8?q?Y84PbiPaPXeu1Ay3U7T8LjSc2A0qY8UU7u/fOZZD9R5ZjCGmr3T/p1eWBAbmRcKGN?= =?utf-8?q?Ynw2ZQPVAztivKGErCKUjX37MMYS5f9KeJGAt9wdsn+wWcihbP8o8r/qFWsnwyrIy?= =?utf-8?q?4KHwME5w0aQwUtO5eeMzKcbGD+GczPj7wkMIJi1yDXNJsJiOQ/hz4iuKDBRdK5srS?= =?utf-8?q?C78PHj2yrxljiCkIvs3VFyC1A6cFBE8Q/UL8fMUaWyR/owQ+YJhnTPuK7/sDJsyYz?= =?utf-8?q?gew0/L7CDyzYgqx2b0WFGdmgVSjIbSgD3Xt9z7JFxnQuoithGH7Xo5HfEi6KU/wPE?= =?utf-8?q?jqKFMI2/6qiYd6+THMVZFK80fi3p9vMot0z/U71Hkc9ykgSQxLtdEBkgIATOCwmVc?= =?utf-8?q?w8eViQKPNlkHJ7B9jth/+8USRSO5Kh5M1Q=3D=3D?= X-OriginatorOrg: corigine.com X-MS-Exchange-CrossTenant-Network-Message-Id: a51722af-2e26-44ec-d574-08da9f8cc847 X-MS-Exchange-CrossTenant-AuthSource: SJ0PR13MB5545.namprd13.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 26 Sep 2022 07:00:23.3380 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: fe128f2c-073b-4c20-818e-7246a585940c X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: rbFaM0iHNr68znFJznFJ1FypoOAksmhqGnM4xGtfbcrC7Js+Jkf1ZjqaC4n0SwK3GRYbwxF0L9w70CtbcKj2n413E2qqTMsRVTYNnc5YX8w= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR13MB4893 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org The NFP card can load different firmware applications. This commit move the init logic of corenic app of the secondary process into its own function. Signed-off-by: Chaoyong He Reviewed-by: Niklas Söderlund --- drivers/net/nfp/nfp_ethdev.c | 90 +++++++++++++++++++++++++++++--------------- 1 file changed, 60 insertions(+), 30 deletions(-) diff --git a/drivers/net/nfp/nfp_ethdev.c b/drivers/net/nfp/nfp_ethdev.c index 14c6d69..b0b3ef8 100644 --- a/drivers/net/nfp/nfp_ethdev.c +++ b/drivers/net/nfp/nfp_ethdev.c @@ -996,6 +996,49 @@ return ret; } +static int +nfp_secondary_init_app_fw_nic(struct rte_pci_device *pci_dev, + struct nfp_rtsym_table *sym_tbl, + struct nfp_cpp *cpp) +{ + int i; + int err = 0; + int ret = 0; + int total_vnics; + struct nfp_net_hw *hw; + + /* Read the number of vNIC's created for the PF */ + total_vnics = nfp_rtsym_read_le(sym_tbl, "nfd_cfg_pf0_num_ports", &err); + if (err != 0 || total_vnics <= 0 || total_vnics > 8) { + PMD_INIT_LOG(ERR, "nfd_cfg_pf0_num_ports symbol with wrong value"); + return -ENODEV; + } + + for (i = 0; i < total_vnics; i++) { + struct rte_eth_dev *eth_dev; + char port_name[RTE_ETH_NAME_MAX_LEN]; + snprintf(port_name, sizeof(port_name), "%s_port%d", + pci_dev->device.name, i); + + PMD_INIT_LOG(DEBUG, "Secondary attaching to port %s", port_name); + eth_dev = rte_eth_dev_attach_secondary(port_name); + if (eth_dev == NULL) { + PMD_INIT_LOG(ERR, "Secondary process attach to port %s failed", port_name); + ret = -ENODEV; + break; + } + + eth_dev->process_private = cpp; + hw = NFP_NET_DEV_PRIVATE_TO_HW(eth_dev->data->dev_private); + if (nfp_net_ethdev_ops_mount(hw, eth_dev)) + return -EINVAL; + + rte_eth_dev_probing_finish(eth_dev); + } + + return ret; +} + /* * When attaching to the NFP4000/6000 PF on a secondary process there * is no need to initialise the PF again. Only minimal work is required @@ -1004,12 +1047,10 @@ static int nfp_pf_secondary_init(struct rte_pci_device *pci_dev) { - int i; int err = 0; int ret = 0; - int total_ports; struct nfp_cpp *cpp; - struct nfp_net_hw *hw; + enum nfp_app_fw_id app_fw_id; struct nfp_rtsym_table *sym_tbl; if (pci_dev == NULL) @@ -1043,37 +1084,26 @@ return -EIO; } - total_ports = nfp_rtsym_read_le(sym_tbl, "nfd_cfg_pf0_num_ports", &err); - if (err != 0 || total_ports <= 0 || total_ports > 8) { - PMD_INIT_LOG(ERR, "nfd_cfg_pf0_num_ports symbol with wrong value"); - ret = -ENODEV; + /* Read the app ID of the firmware loaded */ + app_fw_id = nfp_rtsym_read_le(sym_tbl, "_pf0_net_app_id", &err); + if (err != 0) { + PMD_INIT_LOG(ERR, "Couldn't read app_fw_id from fw"); goto sym_tbl_cleanup; } - for (i = 0; i < total_ports; i++) { - struct rte_eth_dev *eth_dev; - char port_name[RTE_ETH_NAME_MAX_LEN]; - - snprintf(port_name, sizeof(port_name), "%s_port%d", - pci_dev->device.name, i); - - PMD_DRV_LOG(DEBUG, "Secondary attaching to port %s", port_name); - eth_dev = rte_eth_dev_attach_secondary(port_name); - if (eth_dev == NULL) { - RTE_LOG(ERR, EAL, - "secondary process attach failed, ethdev doesn't exist"); - ret = -ENODEV; - break; + switch (app_fw_id) { + case NFP_APP_FW_CORE_NIC: + PMD_INIT_LOG(INFO, "Initializing coreNIC"); + ret = nfp_secondary_init_app_fw_nic(pci_dev, sym_tbl, cpp); + if (ret != 0) { + PMD_INIT_LOG(ERR, "Could not initialize coreNIC!"); + goto sym_tbl_cleanup; } - - hw = NFP_NET_DEV_PRIVATE_TO_HW(eth_dev->data->dev_private); - - if (nfp_net_ethdev_ops_mount(hw, eth_dev)) - return -EINVAL; - - eth_dev->process_private = cpp; - - rte_eth_dev_probing_finish(eth_dev); + break; + default: + PMD_INIT_LOG(ERR, "Unsupported Firmware loaded"); + ret = -EINVAL; + goto sym_tbl_cleanup; } /* Register the CPP bridge service for the secondary too */ From patchwork Mon Sep 26 06:59:48 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Chaoyong He X-Patchwork-Id: 116845 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 0FA1DA00C2; Mon, 26 Sep 2022 09:00:50 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 5BF20427ED; Mon, 26 Sep 2022 09:00:30 +0200 (CEST) Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2117.outbound.protection.outlook.com [40.107.244.117]) by mails.dpdk.org (Postfix) with ESMTP id C1C4241145 for ; Mon, 26 Sep 2022 09:00:27 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=F8A2UAYF9p/OE2clpDOr83Uwexqolxnc2rG26Jz4AcveSQXaCmXNioX1DGZNfyCUqMXVmHIU/PRtmSrNYQMAPfQVkSn/NIxNx3QSo2CCpZMjyF4IgVU3gYzSmkrs8RnJZqFVm1rAM5OSFcSZfKZP0YWvSHriZyJi2Opth1XWJP+B19oV/dyWZwJE7TugUFXbPFIbXMkQj5F0Vb4fYnj3hXt9i9o3f8gzrmRdn7HguSFAwgJcuCu9E7XOjRZwOgHOmqkdyQ1YJVOrnQnchywTtvgwnrSWrY/KPVvyvnl4b6EUEOqyFk9T/GHc6n5DC048JtL2DTALHQeW30L/p5SBOg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=4pfIN3sUaBfdwVLXUCt6XZMhxr5jP1y5pehzoRF8Hb8=; b=Wfst82L2dRYr/g9Yqdudz6GQg7dq04dgq5sGfyFUERDHz/a0anqwlunlawsPWCWhxVuFsQ467GJqV5sRplb/TcMVDlfAHBlPNJOP0BY0N1r15mRfKOooKcZP/HkL0nZxKSrouqKsPUe6ldQKc5n+pJS9HtdG7McAaP25VZPoocl0RdDtaMYjrRANzQww1CkfVjfevPGx2x9A4qliQueGks/lYyd7S0RaMEZtIhCwgAr3fj1omR3EMeP8IccTgGL5H08alVhDQB0U8EycplE1PMNrxdkQ5M0hfmBMJ//TeI1prlYpx+c6d2doEuj2DAdyy0YQ7AdTwwoAVyMwDgtB1g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=corigine.com; dmarc=pass action=none header.from=corigine.com; dkim=pass header.d=corigine.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=corigine.onmicrosoft.com; s=selector2-corigine-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=4pfIN3sUaBfdwVLXUCt6XZMhxr5jP1y5pehzoRF8Hb8=; b=Wd0EV9zFh8lZ+xG5UG47WnACAj9+lUTlFm/vAd184rppcRyNX6aWDgyvCPpe6d20C8puEbPD6Yfrk9zY4/JCB/C9metjwCGqD1ss0cX67xzmxR0qcOCN/djH/4iR8DO1gHjutwtGDSj01WnR7ABPnXwS7Dtp5LZXJk3xTPeUJdA= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=corigine.com; Received: from SJ0PR13MB5545.namprd13.prod.outlook.com (2603:10b6:a03:424::5) by SA1PR13MB4893.namprd13.prod.outlook.com (2603:10b6:806:18a::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5676.14; Mon, 26 Sep 2022 07:00:26 +0000 Received: from SJ0PR13MB5545.namprd13.prod.outlook.com ([fe80::779:2722:a8e5:503b]) by SJ0PR13MB5545.namprd13.prod.outlook.com ([fe80::779:2722:a8e5:503b%7]) with mapi id 15.20.5676.014; Mon, 26 Sep 2022 07:00:26 +0000 From: Chaoyong He To: dev@dpdk.org Cc: oss-drivers@corigine.com, niklas.soderlund@corigine.com, Chaoyong He , Heinrich Kuhn Subject: [PATCH v10 04/13] net/nfp: add initial flower firmware support Date: Mon, 26 Sep 2022 14:59:48 +0800 Message-Id: <1664175597-37248-5-git-send-email-chaoyong.he@corigine.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1664175597-37248-1-git-send-email-chaoyong.he@corigine.com> References: <1664175597-37248-1-git-send-email-chaoyong.he@corigine.com> X-ClientProxiedBy: SG2P153CA0005.APCP153.PROD.OUTLOOK.COM (2603:1096::15) To SJ0PR13MB5545.namprd13.prod.outlook.com (2603:10b6:a03:424::5) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SJ0PR13MB5545:EE_|SA1PR13MB4893:EE_ X-MS-Office365-Filtering-Correlation-Id: f264fdab-985b-4ae4-8099-08da9f8cc9e4 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: avIJAIbi9p6HBdxRsjfHljGeA0tBDI63c18Tee8ED1Dxgl9b8Ujq0umpBWl5WnsSLaf5sjAVu9DxAP92Y1Vjp9ETWHRC46e63HDFT+9Lvy5pppdtYDMigqZFNsrmA5JxcDV/1tAgP8/aDnaTmulRyzP6rWRPiY9SPNP/RnkoIGuDUlwxAQYUSjYi3W7iu91Tu31PLIK7l0b7dbBdzJRpADUMOVWmYlWL+C9nY+uIWbCqVN57BIH96GqPe1vj4GmxcyPl85UO7mLvC5GkdnDxjZqClQpc3e2CQzco5fJ4mCIX6CNRxpgzjaaHzawv9jJGisaf6bOzopWi0FiFJ7/gXzRJykH9KkyoFzKAiwuS7xq4f/QJ7Z2BnyWP9CyHMcTApRahp9mZbdQW8a8ZNvscU87Rmlbvg3KZ9frsWtz7PwiOyg9Re/NH8gksM0t96D9cPv1ADgEnZuz1A2PO2UGXcTU+8HqNJFdWo9Ok7YpVbHfzfzAvqlbsn9VgBkPgNmb9tR0dT7f7WyGE67+kfWoLRjb0WjB7qykb6W+w3apIuCUwuVfy7NeouNHK0c3HBGEV4sHxOBjlgCEJrbWp1k1sMEUV0+isfnI+0iqE6LZULCx/iY/cuudZFMqu88iGyOH+DLo7f1mPxLe5yKWULVLDzQ2RtEix4PRCALDqz0MqnehaimMAro68RZsXUdikZNY1r8P44eW5FN+B/mikAD1xOIRLmoRX4axBHlmzGmpxn0DySxbeDCyreo5kOWnMoP5WPEKbL4s0YaI2VjBtanXreA== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SJ0PR13MB5545.namprd13.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230022)(4636009)(366004)(396003)(39830400003)(136003)(376002)(346002)(451199015)(36756003)(38100700002)(38350700002)(2906002)(86362001)(44832011)(30864003)(66946007)(8936002)(66556008)(66476007)(5660300002)(8676002)(4326008)(83380400001)(186003)(66574015)(54906003)(6512007)(6916009)(316002)(52116002)(6666004)(2616005)(26005)(6506007)(478600001)(41300700001)(107886003)(6486002); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?snggjyrlQEEc7GV3T1e0bzAPpEQt?= =?utf-8?q?jF5jTdRrUNlxLpn0WDywAqJW+WdcXumkapAwxjTNzZpX0+86wtwN22hXYp80TILWV?= =?utf-8?q?vv28AlxKrRDj0I/egQm0H7ezeIiELdwr+qojxLyYlLkB1GrCZ1q3eiJ9WaC9Nt3xW?= =?utf-8?q?FYt5IjJlPBXxUDDpBMqiqAG8UH3NAGz/JgqBxJeHi4E3uQ68IKW/hOovEnHbjRvF6?= =?utf-8?q?Ky3MPPNeYhsxLMsLRJtExWjBM4i4vPpuzMHdLMpq8KZmJb4VVjWLQYJP+yg5amwGW?= =?utf-8?q?mjxoaRDDZSxI4dn8D8bM6TA7zkeJkpaVKW/R40GtuWNEfai5QIxtMr2Miq6t0K7r5?= =?utf-8?q?VhIrqiW7WqRgUVmTpmF5hAAXif5ILzPtbnGJqnki+ugKcXj2uBPf7lxWaiImFLRx5?= =?utf-8?q?FgeP8siUMkreK/RqfpH0N2VjQgtrUZN9fHRL8G5m+dxt1ceSW2cQMR1rXEuY1fMog?= =?utf-8?q?D85wk/p4hoOGpXB4vQnDDudz40SABrLBwjGCUOpx7MyJ4O2TD15SXW/8TSuPRPHkf?= =?utf-8?q?6YmU/LM+5uOXqDvNW2v9EtmuPmwBTWzIrZAXcf0DvBJzW1qUVTkTLRoU+9iqbHYPO?= =?utf-8?q?PlIkCIlpSHeDwU1jg3w7Uza4FRWMUCMvRzecNH8iuICpNgI4ZjEfjAuYJJxPWY+h4?= =?utf-8?q?JHQ7Ebio+dcRE5kJ7ZOVAWknL+7ahHAfFCI12rzYN52H7sC8zDc4oAwNPJMmq/fx4?= =?utf-8?q?n8Mczjqw3+HF23sIMUfK8z/GNHw4/vpZKQHhYBOAzMw1HzH3wFn93CoUxgraYwYDB?= =?utf-8?q?UYbkBoDgQ0U2xYwfPppxEqXNTtM9NCotb/4a3Go4gvMHWhRj41l0TIpXUKY8MCnye?= =?utf-8?q?YWhPRY67G3q2cdFbSdyvayRPXjAZUa5IVkz5q8xpeMeqaXOGCHJ0/T8YlCWhJsRHA?= =?utf-8?q?qgyWF262lHoifuvpC9d0HXvBgH+kLo3FkT0bvbVINNXigA97rjbRdurxbzGM6ScTo?= =?utf-8?q?nJhxMsUUFb+jSOf9q933szuIhbeQMcBw+BS+14iVuN3741w2+Wd2W8MxY4gTs8zyB?= =?utf-8?q?NtzOPygg7X509RPOzifM5cNlp8caZs6rV1yNiLw2+gAArNFACwRjrrvfu6joO4i8G?= =?utf-8?q?O93VyRINQvuV2p9fhiJut/Goce/jU+QwPW1sa29yHEtceF25Y/MKCJcYP+ABjiSuJ?= =?utf-8?q?mTpOoU3Rym1X5eKxVaT9FFNp+ZaUp4VxlnHPhFTd6+NcrTFJ9aZNGLcGsPNx++F7C?= =?utf-8?q?Qmn08mTFZOv5gCV736N1IeMd3AZxyluxAyrT0TtbNEaJB694BNc69V4mEkWmKbziF?= =?utf-8?q?dk04UMomCQzbPFHQxu/BU0lPpZ/sVJHmA6I0FqLCqQOAFLyYBLZZG0O2fBtYoutP3?= =?utf-8?q?KpytU+YiSRdFNa09zLiZWhrtOm9UCqVSqnFjRasL6CXlNMu/HCWKEqnCY7nYNfoxe?= =?utf-8?q?G2dRiB9NDYmfiAODm/GlLrytzoM2HCgd28XTWjNVRcLmR86OiGQ6E2InAOU4cecvu?= =?utf-8?q?DUr5aEYSqD50InZ+WpIYFet8fqLWTYgtIhwg84oLG5HaCJTIJoIPsPiWoCmLXJ8RL?= =?utf-8?q?dqi0pxcfdvjlzn4mzXbcMrx0y8pNu9U8Fg=3D=3D?= X-OriginatorOrg: corigine.com X-MS-Exchange-CrossTenant-Network-Message-Id: f264fdab-985b-4ae4-8099-08da9f8cc9e4 X-MS-Exchange-CrossTenant-AuthSource: SJ0PR13MB5545.namprd13.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 26 Sep 2022 07:00:25.8717 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: fe128f2c-073b-4c20-818e-7246a585940c X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: NpLQVOPTYqlX8+99o1XFRLB5wOZGyk2+FJdXXjm4bNAvOygpZ3DqzMUvJgEnk4aYbjViYljuJli55bzqDzVgB57+jY4GqFYs1apArwPUdzA= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR13MB4893 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Adds the basic probing infrastructure to support the flower firmware application. Adds the cpp service, used for some user tools. Signed-off-by: Chaoyong He Signed-off-by: Heinrich Kuhn Reviewed-by: Niklas Söderlund --- doc/guides/nics/nfp.rst | 13 +++++ doc/guides/rel_notes/release_22_11.rst | 7 +++ drivers/net/nfp/flower/nfp_flower.c | 45 +++++++++++++++++ drivers/net/nfp/flower/nfp_flower.h | 16 +++++++ drivers/net/nfp/meson.build | 1 + drivers/net/nfp/nfp_common.h | 1 + drivers/net/nfp/nfp_cpp_bridge.c | 88 +++++++++++++++++++++++++++++----- drivers/net/nfp/nfp_cpp_bridge.h | 6 ++- drivers/net/nfp/nfp_ethdev.c | 31 +++++++++++- 9 files changed, 192 insertions(+), 16 deletions(-) create mode 100644 drivers/net/nfp/flower/nfp_flower.c create mode 100644 drivers/net/nfp/flower/nfp_flower.h diff --git a/doc/guides/nics/nfp.rst b/doc/guides/nics/nfp.rst index 55539ac..4faab39 100644 --- a/doc/guides/nics/nfp.rst +++ b/doc/guides/nics/nfp.rst @@ -181,3 +181,16 @@ System configuration -k option shows the device driver, if any, that devices are bound to. Depending on the modules loaded at this point the new PCI devices may be bound to nfp_netvf driver. + + +Flow offload +------------ + +Use the flower firmware application, some type of Netronome's SmartNICs can +offload the flow into cards. + +The flower firmware application requires the PMD running two services: + + * PF vNIC service: handling the feedback traffic. + * ctrl vNIC service: communicate between PMD and firmware through + control message. diff --git a/doc/guides/rel_notes/release_22_11.rst b/doc/guides/rel_notes/release_22_11.rst index 2e076ba..ad438b2 100644 --- a/doc/guides/rel_notes/release_22_11.rst +++ b/doc/guides/rel_notes/release_22_11.rst @@ -55,6 +55,13 @@ New Features Also, make sure to start the actual text at the margin. ======================================================= +* **Updated Netronome nfp driver.** + + Add the needed data structures and logics to support the offload of rte_flow: + + * Added the support of flower firmware. + * Added the flower service infrastructure. + * **Updated Wangxun ngbe driver.** * Added support to set device link down/up. diff --git a/drivers/net/nfp/flower/nfp_flower.c b/drivers/net/nfp/flower/nfp_flower.c new file mode 100644 index 0000000..87cb922 --- /dev/null +++ b/drivers/net/nfp/flower/nfp_flower.c @@ -0,0 +1,45 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2022 Corigine, Inc. + * All rights reserved. + */ + +#include +#include +#include +#include +#include +#include + +#include "../nfp_common.h" +#include "../nfp_logs.h" +#include "../nfp_ctrl.h" +#include "../nfp_cpp_bridge.h" +#include "nfp_flower.h" + +int +nfp_init_app_fw_flower(struct nfp_pf_dev *pf_dev) +{ + unsigned int numa_node; + struct nfp_app_fw_flower *app_fw_flower; + + numa_node = rte_socket_id(); + + /* Allocate memory for the Flower app */ + app_fw_flower = rte_zmalloc_socket("nfp_app_fw_flower", sizeof(*app_fw_flower), + RTE_CACHE_LINE_SIZE, numa_node); + if (app_fw_flower == NULL) { + PMD_INIT_LOG(ERR, "Could not malloc app fw flower"); + return -ENOMEM; + } + + pf_dev->app_fw_priv = app_fw_flower; + + return 0; +} + +int +nfp_secondary_init_app_fw_flower(__rte_unused struct nfp_cpp *cpp) +{ + PMD_INIT_LOG(ERR, "Flower firmware not supported"); + return -ENOTSUP; +} diff --git a/drivers/net/nfp/flower/nfp_flower.h b/drivers/net/nfp/flower/nfp_flower.h new file mode 100644 index 0000000..8b9ef95 --- /dev/null +++ b/drivers/net/nfp/flower/nfp_flower.h @@ -0,0 +1,16 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2022 Corigine, Inc. + * All rights reserved. + */ + +#ifndef _NFP_FLOWER_H_ +#define _NFP_FLOWER_H_ + +/* The flower application's private structure */ +struct nfp_app_fw_flower { +}; + +int nfp_init_app_fw_flower(struct nfp_pf_dev *pf_dev); +int nfp_secondary_init_app_fw_flower(struct nfp_cpp *cpp); + +#endif /* _NFP_FLOWER_H_ */ diff --git a/drivers/net/nfp/meson.build b/drivers/net/nfp/meson.build index 810f02a..7ae3115 100644 --- a/drivers/net/nfp/meson.build +++ b/drivers/net/nfp/meson.build @@ -6,6 +6,7 @@ if not is_linux or not dpdk_conf.get('RTE_ARCH_64') reason = 'only supported on 64-bit Linux' endif sources = files( + 'flower/nfp_flower.c', 'nfpcore/nfp_cpp_pcie_ops.c', 'nfpcore/nfp_nsp.c', 'nfpcore/nfp_cppcore.c', diff --git a/drivers/net/nfp/nfp_common.h b/drivers/net/nfp/nfp_common.h index 5bdf028..52dc3ca 100644 --- a/drivers/net/nfp/nfp_common.h +++ b/drivers/net/nfp/nfp_common.h @@ -116,6 +116,7 @@ /* Firmware application ID's */ enum nfp_app_fw_id { NFP_APP_FW_CORE_NIC = 0x1, + NFP_APP_FW_FLOWER_NIC = 0x3, }; /* nfp_qcp_ptr - Read or Write Pointer of a queue */ diff --git a/drivers/net/nfp/nfp_cpp_bridge.c b/drivers/net/nfp/nfp_cpp_bridge.c index 0922ea9..155628d 100644 --- a/drivers/net/nfp/nfp_cpp_bridge.c +++ b/drivers/net/nfp/nfp_cpp_bridge.c @@ -28,22 +28,86 @@ static int nfp_cpp_bridge_serve_write(int sockfd, struct nfp_cpp *cpp); static int nfp_cpp_bridge_serve_read(int sockfd, struct nfp_cpp *cpp); static int nfp_cpp_bridge_serve_ioctl(int sockfd, struct nfp_cpp *cpp); +static int nfp_cpp_bridge_service_func(void *args); -void nfp_register_cpp_service(struct nfp_cpp *cpp) +int +nfp_map_service(uint32_t service_id) { - uint32_t *cpp_service_id = NULL; - struct rte_service_spec service; + int32_t ret; + uint32_t slcore = 0; + int32_t slcore_count; + uint8_t service_count; + const char *service_name; + uint32_t slcore_array[RTE_MAX_LCORE]; + uint8_t min_service_count = UINT8_MAX; + + slcore_count = rte_service_lcore_list(slcore_array, RTE_MAX_LCORE); + if (slcore_count <= 0) { + PMD_INIT_LOG(DEBUG, "No service cores found"); + return -ENOENT; + } + + /* + * Find a service core with the least number of services already + * registered to it + */ + while (slcore_count--) { + service_count = rte_service_lcore_count_services(slcore_array[slcore_count]); + if (service_count < min_service_count) { + slcore = slcore_array[slcore_count]; + min_service_count = service_count; + } + } + + service_name = rte_service_get_name(service_id); + PMD_INIT_LOG(INFO, "Mapping service %s to core %u", service_name, slcore); - memset(&service, 0, sizeof(struct rte_service_spec)); - snprintf(service.name, sizeof(service.name), "nfp_cpp_service"); - service.callback = nfp_cpp_bridge_service_func; - service.callback_userdata = (void *)cpp; + ret = rte_service_map_lcore_set(service_id, slcore, 1); + if (ret != 0) { + PMD_INIT_LOG(DEBUG, "Could not map flower service"); + return -ENOENT; + } - if (rte_service_component_register(&service, - cpp_service_id)) - RTE_LOG(WARNING, PMD, "NFP CPP bridge service register() failed"); + rte_service_runstate_set(service_id, 1); + rte_service_component_runstate_set(service_id, 1); + rte_service_lcore_start(slcore); + if (rte_service_may_be_active(slcore)) + PMD_INIT_LOG(INFO, "The service %s is running", service_name); else - RTE_LOG(DEBUG, PMD, "NFP CPP bridge service registered"); + PMD_INIT_LOG(ERR, "The service %s is not running", service_name); + + return 0; +} + +int +nfp_enable_cpp_service(struct nfp_cpp *cpp) +{ + int ret; + uint32_t service_id = 0; + struct rte_service_spec cpp_service = { + .name = "nfp_cpp_service", + .callback = nfp_cpp_bridge_service_func, + }; + + cpp_service.callback_userdata = (void *)cpp; + + /* Register the cpp service */ + ret = rte_service_component_register(&cpp_service, &service_id); + if (ret != 0) { + PMD_INIT_LOG(WARNING, "Could not register nfp cpp service"); + return -EINVAL; + } + + PMD_INIT_LOG(INFO, "NFP cpp service registered"); + + /* Map it to available service core*/ + ret = nfp_map_service(service_id); + if (ret != 0) { + PMD_INIT_LOG(DEBUG, "Could not map nfp cpp service"); + return -EINVAL; + } + + return 0; } /* @@ -307,7 +371,7 @@ void nfp_register_cpp_service(struct nfp_cpp *cpp) * unaware of the CPP bridge performing the NFP kernel char driver for CPP * accesses. */ -int32_t +static int nfp_cpp_bridge_service_func(void *args) { struct sockaddr address; diff --git a/drivers/net/nfp/nfp_cpp_bridge.h b/drivers/net/nfp/nfp_cpp_bridge.h index aea5fdc..7fee3a9 100644 --- a/drivers/net/nfp/nfp_cpp_bridge.h +++ b/drivers/net/nfp/nfp_cpp_bridge.h @@ -16,6 +16,8 @@ #ifndef _NFP_CPP_BRIDGE_H_ #define _NFP_CPP_BRIDGE_H_ +#include "nfp_common.h" + #define NFP_CPP_MEMIO_BOUNDARY (1 << 20) #define NFP_BRIDGE_OP_READ 20 #define NFP_BRIDGE_OP_WRITE 30 @@ -24,8 +26,8 @@ #define NFP_IOCTL 'n' #define NFP_IOCTL_CPP_IDENTIFICATION _IOW(NFP_IOCTL, 0x8f, uint32_t) -void nfp_register_cpp_service(struct nfp_cpp *cpp); -int32_t nfp_cpp_bridge_service_func(void *args); +int nfp_enable_cpp_service(struct nfp_cpp *cpp); +int nfp_map_service(uint32_t service_id); #endif /* _NFP_CPP_BRIDGE_H_ */ /* diff --git a/drivers/net/nfp/nfp_ethdev.c b/drivers/net/nfp/nfp_ethdev.c index b0b3ef8..5a9a1bd 100644 --- a/drivers/net/nfp/nfp_ethdev.c +++ b/drivers/net/nfp/nfp_ethdev.c @@ -38,6 +38,8 @@ #include "nfp_ctrl.h" #include "nfp_cpp_bridge.h" +#include "flower/nfp_flower.h" + static int nfp_net_pf_read_mac(struct nfp_app_fw_nic *app_fw_nic, int port) { @@ -969,6 +971,14 @@ goto hwqueues_cleanup; } break; + case NFP_APP_FW_FLOWER_NIC: + PMD_INIT_LOG(INFO, "Initializing Flower"); + ret = nfp_init_app_fw_flower(pf_dev); + if (ret != 0) { + PMD_INIT_LOG(ERR, "Could not initialize Flower!"); + goto hwqueues_cleanup; + } + break; default: PMD_INIT_LOG(ERR, "Unsupported Firmware loaded"); ret = -EINVAL; @@ -976,7 +986,12 @@ } /* register the CPP bridge service here for primary use */ - nfp_register_cpp_service(pf_dev->cpp); + ret = nfp_enable_cpp_service(pf_dev->cpp); + if (ret != 0) { + PMD_INIT_LOG(ERR, "Enable cpp service failed."); + ret = -EINVAL; + goto hwqueues_cleanup; + } return 0; @@ -1100,6 +1115,14 @@ goto sym_tbl_cleanup; } break; + case NFP_APP_FW_FLOWER_NIC: + PMD_INIT_LOG(INFO, "Initializing Flower"); + ret = nfp_secondary_init_app_fw_flower(cpp); + if (ret != 0) { + PMD_INIT_LOG(ERR, "Could not initialize Flower!"); + goto sym_tbl_cleanup; + } + break; default: PMD_INIT_LOG(ERR, "Unsupported Firmware loaded"); ret = -EINVAL; @@ -1107,7 +1130,11 @@ } /* Register the CPP bridge service for the secondary too */ - nfp_register_cpp_service(cpp); + ret = nfp_enable_cpp_service(cpp); + if (ret != 0) { + PMD_INIT_LOG(ERR, "Enable cpp service failed."); + ret = -EINVAL; + } sym_tbl_cleanup: free(sym_tbl); From patchwork Mon Sep 26 06:59:49 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Chaoyong He X-Patchwork-Id: 116846 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id C7998A00C2; Mon, 26 Sep 2022 09:00:57 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 52AC14282E; Mon, 26 Sep 2022 09:00:31 +0200 (CEST) Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2132.outbound.protection.outlook.com [40.107.244.132]) by mails.dpdk.org (Postfix) with ESMTP id AEF2B42802 for ; Mon, 26 Sep 2022 09:00:29 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=W2gRKluZRXNejn6lxWlzbMzLAL32xXjr/s1zWq6WKxo1h2DKl0359q/1fRDNloFuNwX2oBpVM9qyJZnLypvgKc8wk4CkgRIz2QQP/4pXT+wpL2RRTrs702d5iAixPomJrsewJFZoq1VAnC+yhFfj5xcic9/9Fn02s5dpPRLe/vpfqZ0M/J+QXWBEt4uQYG1hWrYSgTKR+rlUAG/rd78G0YtnShaw8Lg+O/ChGiAIGQo9TbM6rCV6pkYlzLCjUpxOpP8rlQzndq/rgH7uy3GZsDfcfBdj77VBzZFp8Q6oSpFJlrdZJm2pDdX4uRQo5zTZyRP1nnqF0GTo5GqQlDXCJA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=bRkGUuhLSajcrO7T7RT3v6WzgxY0ddN5pk0bv0nKcao=; b=lvXKw2wm5OHfq5pmyVS4M8DOa7dBc6pKNx6ZZr+LoYrRL49DxUT5JLUUMPwqSiNVL7Ccrbs0x0bBZx6rwWv2HLJd3e4UtRnzVaDSJ6SUQWDYbK9FAkoYUjQ5PbgD2CeCC83q1AJUIMr8MMze4DzVTfREUTiLjy9qXRE4cGjpy/D0agPoO0QXOu1ggB2Stmboed5SIQznqCITuken6KvsWXhtPWpUFs9dj5UGhaHVJZWT6jkzYfSpo4wl5I+KoKZNgduV5B/ns2mznYP4T7ju9Vn22tX/ZkEfOr0TSya4XEi5JuMoZ5MaNBndzDu2mg3RUkYjqbv6/XGrya25hIdQ0w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=corigine.com; dmarc=pass action=none header.from=corigine.com; dkim=pass header.d=corigine.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=corigine.onmicrosoft.com; s=selector2-corigine-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=bRkGUuhLSajcrO7T7RT3v6WzgxY0ddN5pk0bv0nKcao=; b=P9CQM8L3AwCIIzpLDKhr4A21YKrNQDNyOjzsvd/SKNMnfix3feXegEkl3Jc494NMmg+W6oNCrZeSboGpDGrrfhxBBs1HNeYT5mMVhwXXZMs9IfgSPYcpSqsdBWY0Y9B7YRM2hP3NC8ynSTQjUZJXt6vAFq6FSzds9HvIw1n+5y4= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=corigine.com; Received: from SJ0PR13MB5545.namprd13.prod.outlook.com (2603:10b6:a03:424::5) by SA1PR13MB4893.namprd13.prod.outlook.com (2603:10b6:806:18a::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5676.14; Mon, 26 Sep 2022 07:00:28 +0000 Received: from SJ0PR13MB5545.namprd13.prod.outlook.com ([fe80::779:2722:a8e5:503b]) by SJ0PR13MB5545.namprd13.prod.outlook.com ([fe80::779:2722:a8e5:503b%7]) with mapi id 15.20.5676.014; Mon, 26 Sep 2022 07:00:28 +0000 From: Chaoyong He To: dev@dpdk.org Cc: oss-drivers@corigine.com, niklas.soderlund@corigine.com, Chaoyong He Subject: [PATCH v10 05/13] net/nfp: add flower PF setup logic Date: Mon, 26 Sep 2022 14:59:49 +0800 Message-Id: <1664175597-37248-6-git-send-email-chaoyong.he@corigine.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1664175597-37248-1-git-send-email-chaoyong.he@corigine.com> References: <1664175597-37248-1-git-send-email-chaoyong.he@corigine.com> X-ClientProxiedBy: SG2P153CA0005.APCP153.PROD.OUTLOOK.COM (2603:1096::15) To SJ0PR13MB5545.namprd13.prod.outlook.com (2603:10b6:a03:424::5) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SJ0PR13MB5545:EE_|SA1PR13MB4893:EE_ X-MS-Office365-Filtering-Correlation-Id: 0a822a80-a050-4a28-2073-08da9f8ccb05 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: IpVnxFYfI7r2wp6gfF/1iJ5FvubwzigN+Z85Y+o7njwPr+eRf6U+yMtmB4LbfPuFonVfmQgmCYtUd/vGn6BRv3IpZ5dL5cl1TirOhx+IOMTLg8QUf8dBr3ehKu7VzNhr74paggTDwNHdOaKd+0+B8Qt4bUkwuWvz/8nXbtY3qvZe93ikYJT4/XS03MFOMlmDpj2uOU6+tTiqRDiLOL5jRkFfRDbvFIxuTR1z/yAe3/Hjw8xefEI+ROIlHLtbo65bBx7dfwu0D7x2uCzcFMHUu9a4q7mvMEwya+l/KtrsMnJctjCKJYbd50VXaGUvOjObOJpARghcwFZzKv4ZXJUSOBHAJ7LRIforBEdAeeHvkXYhNSUvihwk0kiXkI2gj8ipmro+G9ThAr+MEZbSGlzcyHFU6+Nq2Su24vOkhTZrA21WenyjOwOu9MLv57sQHte0oaTaI/fyXReojSWwaiCZ18L17bRyeWZD18hig3H/nGDZOmKYxLqDmgUxpDWRqQhWg2FpBWITbIsLAXAUVbNFQAsoGmGRhVdh1Pl8953GCZuHYm9dnAnaHjV3x6TZgvElfAjO+jttsUZYtn7Ntj+zC3EqmefzMVC0WbCf5LerPcF/KrUxjQLYCMg1HHQ/bfCa3Cv2B/5hCE7tXwCuV++CgtjMJ7vFCdJuWdbpbH0g4QQhJ8S4I33sC5VyAGAktJpgAQU/EP5Kvmrw+6NoyuBtyzcLaDebwfvWi/0f/zQ0eqm6GMGEU3wJm7v70AXfwr26KukMlPHGYD7VeYSDgiPvRw== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SJ0PR13MB5545.namprd13.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230022)(4636009)(366004)(396003)(39830400003)(136003)(376002)(346002)(451199015)(36756003)(38100700002)(38350700002)(2906002)(86362001)(44832011)(66946007)(8936002)(66556008)(66476007)(5660300002)(8676002)(4326008)(83380400001)(186003)(66574015)(6512007)(6916009)(316002)(52116002)(6666004)(2616005)(26005)(6506007)(478600001)(41300700001)(107886003)(6486002); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?6qOVdAp5aEA2OhS8JWJuHjBb7uLB?= =?utf-8?q?BrqB4iE5hvHpRAYrk31NDAVqYnYisfSseDdY6czgtwikp6p6ZY8I2XTI3sJ9HTxAo?= =?utf-8?q?y5PS+XQSB9CG/dslF2dLCbmJAhrQV7Ly3Y4XYB4wAzGbH9eYxdfcYIMjmenf+4flb?= =?utf-8?q?oUPhyqFTtUwgS9jqd4hlKJV0iTWs4YsLKIelFhczslsQmSNDXTEduBVy5FvvfUPTd?= =?utf-8?q?okeiElfVzZ3oeaOGNxBjEvZUxnOoc7lCayS2jFWkw741UAChOib1W85V5qI61O20q?= =?utf-8?q?dOvQsqtqXLmAhiDM0PuA6aD3NOoZjNdRsOdkmyC/dXXiafn9A7felRx90AqrDqFXO?= =?utf-8?q?Myws48icQMiakO1KWXRoFLgmsw+12XLTbYcFa0+xhANlDqOzGfoFyLxaSh/G2fVWV?= =?utf-8?q?NqIb6Vlc4u5arMnp0kbe8WeJAl9V8zxCq4kGXW7xCkJhGE6lz15QmOm/6IjGbKIB1?= =?utf-8?q?Ezi3eFD2Rx/gQCU3vsA6j/O2Xk6pLEEJTZd3EGqZ1nscq+S5L7MnDQByDvrlhZ8Ii?= =?utf-8?q?FO1V5vK3rgMefuAzBznNw3lAK4/7sMsD+oPxeoV9eyZ6R0CztweYkQuLLJzxp9fgL?= =?utf-8?q?8oDOEv0rSCIeZiW8b6NQP1tlOyU+14PIJ8WS8ugHxs1b937dcvhp0jOtEC8BkU6zI?= =?utf-8?q?VPo2UEVsdw2g5x8yZRGauFFRQwE+7YyKIQ6abQcu7UMFGiT6ngX6FNtfQR84IPyeT?= =?utf-8?q?sC0RHdoz6tzcY2BLd2br9Xk1mnSgycQTuTuO4RzfpSRBPbAk2/KnnlREUBrkiegFo?= =?utf-8?q?ijSxyaIQGCjcWUnTWx9sdwqnv1te471B9VsyH0JQZG2TAdo4NmIDfHqlo4oZoL6N6?= =?utf-8?q?V5y1cE84fMZ/yN+MTYI74WyZc1MC5+fzCPQUyzAft5tiUsaA+FD+mssjM/M8Q8lOV?= =?utf-8?q?X9PboygLpwPGPyTPGbfZydo9lU9jtvWZhcVozg7APL4PoO5ieUz0m/g7ej97vkXMR?= =?utf-8?q?HOg8J00juHbNd7mz5oRTG3ENbi38H5mFpcgjS43c7jwQN1n0fts18ESN8Cck7H3G7?= =?utf-8?q?l7SOoqzsRtR42QTXW+98AkGBkrW1v7WdlJRI7MtZ7vdS0QUvuZCXqDiwQYJSDwvZD?= =?utf-8?q?O7WqMAyUiaLZe7g9x5yDp80K7r3vWd3YbMFxIDq24dLiltBAjXwUBotfwGCIaDZFV?= =?utf-8?q?zIfzGFb2DmMC0Pw3dVYnU63ddXAzsafrmKs56AY7/BdZ8ToUEM+c/YqqwS90lRRoK?= =?utf-8?q?s5BbEINSe1RzeZb3cVkCkrMfM2kLKIjgWh7QmfCCLZLLyBp87u2e9Sr1OVEzooQvZ?= =?utf-8?q?GLRP1cBg8GktI5dcFlgZn4wE4Saqxk6yPvu1r8bsEGH1kXKg2Eg9ZXFtBL7qiU0A7?= =?utf-8?q?V6SA5fZDt/DrJ7zI9YPp0OKGkXfZCOHrSahWTYNrS6wluf+6s6ATr0V9G4KkjGc4O?= =?utf-8?q?wnRQXGP4I02SANZDevsuOaoCLiLX+byTkXZp+cAW1f1q6LOHKD7a8f/n0rG9KZ03R?= =?utf-8?q?+Uqo9xG7J3WIwb+K6kDlSJCAdtpyklnvS48VWtl0yBQ9L7B459ZVx2J9o6pH/3pSJ?= =?utf-8?q?VeRdhOCjAg5DbeEB1FvhDhXspXgbC0hWNw=3D=3D?= X-OriginatorOrg: corigine.com X-MS-Exchange-CrossTenant-Network-Message-Id: 0a822a80-a050-4a28-2073-08da9f8ccb05 X-MS-Exchange-CrossTenant-AuthSource: SJ0PR13MB5545.namprd13.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 26 Sep 2022 07:00:27.9362 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: fe128f2c-073b-4c20-818e-7246a585940c X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 8udOwCQn8UhNuWPT+62yEdu8Vlg/0U55UbS9bMTT4+4Z64Qcu24//0J2+mL9jb7PyV5ir+n9oHNH659eeM2gbT6nnwgy1+rMI+Rl1jnM2FY= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR13MB4893 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Adds the vNIC initialization logic for the flower PF vNIC. The flower firmware application exposes this vNIC for the purposes of fallback traffic in the switchdev use-case. Adds minimal dev_ops for this PF vNIC device. Because the device is being exposed externally to DPDK it needs to implements a minimal set of dev_ops. Signed-off-by: Chaoyong He Reviewed-by: Niklas Söderlund --- drivers/net/nfp/flower/nfp_flower.c | 143 +++++++++++++++++++++++++++++++++++- drivers/net/nfp/flower/nfp_flower.h | 2 + drivers/net/nfp/nfp_common.h | 3 + 3 files changed, 145 insertions(+), 3 deletions(-) diff --git a/drivers/net/nfp/flower/nfp_flower.c b/drivers/net/nfp/flower/nfp_flower.c index 87cb922..aec9e28 100644 --- a/drivers/net/nfp/flower/nfp_flower.c +++ b/drivers/net/nfp/flower/nfp_flower.c @@ -14,12 +14,96 @@ #include "../nfp_logs.h" #include "../nfp_ctrl.h" #include "../nfp_cpp_bridge.h" +#include "../nfp_rxtx.h" +#include "../nfpcore/nfp_mip.h" +#include "../nfpcore/nfp_rtsym.h" +#include "../nfpcore/nfp_nsp.h" #include "nfp_flower.h" +#define DEFAULT_FLBUF_SIZE 9216 + +static const struct eth_dev_ops nfp_flower_pf_vnic_ops = { + .dev_infos_get = nfp_net_infos_get, + .link_update = nfp_net_link_update, + .dev_configure = nfp_net_configure, +}; + +static int +nfp_flower_init_vnic_common(struct nfp_net_hw *hw, const char *vnic_type) +{ + uint32_t start_q; + uint64_t rx_bar_off; + uint64_t tx_bar_off; + const int stride = 4; + struct nfp_pf_dev *pf_dev; + struct rte_pci_device *pci_dev; + + pf_dev = hw->pf_dev; + pci_dev = hw->pf_dev->pci_dev; + + /* NFP can not handle DMA addresses requiring more than 40 bits */ + if (rte_mem_check_dma_mask(40)) { + PMD_INIT_LOG(ERR, "Device %s can not be used: restricted dma mask to 40 bits!\n", + pci_dev->device.name); + return -ENODEV; + }; + + hw->device_id = pci_dev->id.device_id; + hw->vendor_id = pci_dev->id.vendor_id; + hw->subsystem_device_id = pci_dev->id.subsystem_device_id; + hw->subsystem_vendor_id = pci_dev->id.subsystem_vendor_id; + + PMD_INIT_LOG(DEBUG, "%s vNIC ctrl bar: %p", vnic_type, hw->ctrl_bar); + + /* Read the number of available rx/tx queues from hardware */ + hw->max_rx_queues = nn_cfg_readl(hw, NFP_NET_CFG_MAX_RXRINGS); + hw->max_tx_queues = nn_cfg_readl(hw, NFP_NET_CFG_MAX_TXRINGS); + + /* Work out where in the BAR the queues start */ + start_q = nn_cfg_readl(hw, NFP_NET_CFG_START_TXQ); + tx_bar_off = (uint64_t)start_q * NFP_QCP_QUEUE_ADDR_SZ; + start_q = nn_cfg_readl(hw, NFP_NET_CFG_START_RXQ); + rx_bar_off = (uint64_t)start_q * NFP_QCP_QUEUE_ADDR_SZ; + + hw->tx_bar = pf_dev->hw_queues + tx_bar_off; + hw->rx_bar = pf_dev->hw_queues + rx_bar_off; + + /* Get some of the read-only fields from the config BAR */ + hw->ver = nn_cfg_readl(hw, NFP_NET_CFG_VERSION); + hw->cap = nn_cfg_readl(hw, NFP_NET_CFG_CAP); + hw->max_mtu = nn_cfg_readl(hw, NFP_NET_CFG_MAX_MTU); + /* Set the current MTU to the maximum supported */ + hw->mtu = hw->max_mtu; + hw->flbufsz = DEFAULT_FLBUF_SIZE; + + /* read the Rx offset configured from firmware */ + if (NFD_CFG_MAJOR_VERSION_of(hw->ver) < 2) + hw->rx_offset = NFP_NET_RX_OFFSET; + else + hw->rx_offset = nn_cfg_readl(hw, NFP_NET_CFG_RX_OFFSET_ADDR); + + hw->ctrl = 0; + hw->stride_rx = stride; + hw->stride_tx = stride; + + /* Reuse cfg queue setup function */ + nfp_net_cfg_queue_setup(hw); + + PMD_INIT_LOG(INFO, "%s vNIC max_rx_queues: %u, max_tx_queues: %u", + vnic_type, hw->max_rx_queues, hw->max_tx_queues); + + /* Initializing spinlock for reconfigs */ + rte_spinlock_init(&hw->reconfig_lock); + + return 0; +} + int nfp_init_app_fw_flower(struct nfp_pf_dev *pf_dev) { + int ret; unsigned int numa_node; + struct nfp_net_hw *pf_hw; struct nfp_app_fw_flower *app_fw_flower; numa_node = rte_socket_id(); @@ -34,12 +118,65 @@ pf_dev->app_fw_priv = app_fw_flower; + /* Allocate memory for the PF AND ctrl vNIC here (hence the * 2) */ + pf_hw = rte_zmalloc_socket("nfp_pf_vnic", 2 * sizeof(struct nfp_net_adapter), + RTE_CACHE_LINE_SIZE, numa_node); + if (pf_hw == NULL) { + PMD_INIT_LOG(ERR, "Could not malloc nfp pf vnic"); + ret = -ENOMEM; + goto app_cleanup; + } + + /* Map the PF ctrl bar */ + pf_dev->ctrl_bar = nfp_rtsym_map(pf_dev->sym_tbl, "_pf0_net_bar0", + 32768, &pf_dev->ctrl_area); + if (pf_dev->ctrl_bar == NULL) { + PMD_INIT_LOG(ERR, "Cloud not map the PF vNIC ctrl bar"); + ret = -ENODEV; + goto vnic_cleanup; + } + + /* Fill in the PF vNIC and populate app struct */ + app_fw_flower->pf_hw = pf_hw; + pf_hw->ctrl_bar = pf_dev->ctrl_bar; + pf_hw->pf_dev = pf_dev; + pf_hw->cpp = pf_dev->cpp; + + ret = nfp_flower_init_vnic_common(app_fw_flower->pf_hw, "pf_vnic"); + if (ret != 0) { + PMD_INIT_LOG(ERR, "Could not initialize flower PF vNIC"); + goto pf_cpp_area_cleanup; + } + return 0; + +pf_cpp_area_cleanup: + nfp_cpp_area_free(pf_dev->ctrl_area); +vnic_cleanup: + rte_free(pf_hw); +app_cleanup: + rte_free(app_fw_flower); + + return ret; } int -nfp_secondary_init_app_fw_flower(__rte_unused struct nfp_cpp *cpp) +nfp_secondary_init_app_fw_flower(struct nfp_cpp *cpp) { - PMD_INIT_LOG(ERR, "Flower firmware not supported"); - return -ENOTSUP; + struct rte_eth_dev *eth_dev; + const char *port_name = "pf_vnic_eth_dev"; + + PMD_INIT_LOG(DEBUG, "Secondary attaching to port %s", port_name); + + eth_dev = rte_eth_dev_attach_secondary(port_name); + if (eth_dev == NULL) { + PMD_INIT_LOG(ERR, "Secondary process attach to port %s failed", port_name); + return -ENODEV; + } + + eth_dev->process_private = cpp; + eth_dev->dev_ops = &nfp_flower_pf_vnic_ops; + rte_eth_dev_probing_finish(eth_dev); + + return 0; } diff --git a/drivers/net/nfp/flower/nfp_flower.h b/drivers/net/nfp/flower/nfp_flower.h index 8b9ef95..51e05e8 100644 --- a/drivers/net/nfp/flower/nfp_flower.h +++ b/drivers/net/nfp/flower/nfp_flower.h @@ -8,6 +8,8 @@ /* The flower application's private structure */ struct nfp_app_fw_flower { + /* Pointer to the PF vNIC */ + struct nfp_net_hw *pf_hw; }; int nfp_init_app_fw_flower(struct nfp_pf_dev *pf_dev); diff --git a/drivers/net/nfp/nfp_common.h b/drivers/net/nfp/nfp_common.h index 52dc3ca..27ea9c2 100644 --- a/drivers/net/nfp/nfp_common.h +++ b/drivers/net/nfp/nfp_common.h @@ -448,6 +448,9 @@ int nfp_net_rss_hash_conf_get(struct rte_eth_dev *dev, #define NFP_PRIV_TO_APP_FW_NIC(app_fw_priv)\ ((struct nfp_app_fw_nic *)app_fw_priv) +#define NFP_PRIV_TO_APP_FW_FLOWER(app_fw_priv)\ + ((struct nfp_app_fw_flower *)app_fw_priv) + #endif /* _NFP_COMMON_H_ */ /* * Local variables: From patchwork Mon Sep 26 06:59:50 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Chaoyong He X-Patchwork-Id: 116847 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id D626DA00C2; Mon, 26 Sep 2022 09:01:08 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id A40B742847; Mon, 26 Sep 2022 09:00:33 +0200 (CEST) Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2097.outbound.protection.outlook.com [40.107.244.97]) by mails.dpdk.org (Postfix) with ESMTP id 023D242847 for ; Mon, 26 Sep 2022 09:00:31 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=DdTGVWZCrd8tJAWth2XDoEGYl8iyG3iAkv0wPb54J8DUrWiqJjF6zZAwiLsQXLbWOEmndy0R4UguX1lc/Hj+kV+Mrtu/dJ+8EkU+wn+hsPjnWYtvHza4GUF+w6ezcgXeXWkFyjhDLya2MiFvLUF8d6OCsw4+jgiQebwPk30TD3krxgLxx9pt1cB9wjcAJ/EnyO9Xmdpb3Ro1lmJHP7ayFG1DUrYQtLss5CbrKrmwO5B7DC9RKUQT6iKJ01oWcGxGup6NUrYUvIphoTHAto3n1JMmBxAR+8/SqdmpyKrtNLCrgOvswqAQKGNdB8RrtBZQ92I0CbEbLZ5k2nOSNDBy/Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=l9j5remVnqr5T106xyHBBxXvEfir+nn2BW6pNeM1viA=; b=nzInftWJlqI8Qweg0nJ0JxcriYaFXe8bFf/4QmXCNUal8Pqy+X4miA4sVKOOvQlvIqh4TSD9RSN31b+RsOEK5TQEadkqws5nGb3a8/iDm42ZJj7K7w7xeeidETN4lo8YHDpoU0/Veaoa9opK9/571tBBqU+JMwVwQhRYEppOJfyIz7ZK0bgDim/6VMA6URwT30Dmw0Iql6AnZwt30ZFQ8GW/iMPJW35DnhdW38KIP0O4Prl/qF063pFCNJ5dkv8LBihCSfPR/XZrq1YWu+rALsru+u3Iho4FEMJQCDGrYFooNraCOvWGemtT3TXT+WgFXGrxADBztQCoT78q8qtewg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=corigine.com; dmarc=pass action=none header.from=corigine.com; dkim=pass header.d=corigine.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=corigine.onmicrosoft.com; s=selector2-corigine-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=l9j5remVnqr5T106xyHBBxXvEfir+nn2BW6pNeM1viA=; b=cn2cnDPvW7GnZvIC2Crk3csB/LJrZGuJ4NpD6KA8ME9u4NuHxbsmXaX4IsnNId6vnprEOD9K4oJP9j0zLlAhBmuCM7bS/XQRoHAgu9wP3xRtw/amyJa1IMNfiIRZ8FC82kvTa6u+y1YNfVupR2LAWEg0evodkrBhkwCXAhEQgDY= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=corigine.com; Received: from SJ0PR13MB5545.namprd13.prod.outlook.com (2603:10b6:a03:424::5) by SA1PR13MB4893.namprd13.prod.outlook.com (2603:10b6:806:18a::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5676.14; Mon, 26 Sep 2022 07:00:30 +0000 Received: from SJ0PR13MB5545.namprd13.prod.outlook.com ([fe80::779:2722:a8e5:503b]) by SJ0PR13MB5545.namprd13.prod.outlook.com ([fe80::779:2722:a8e5:503b%7]) with mapi id 15.20.5676.014; Mon, 26 Sep 2022 07:00:30 +0000 From: Chaoyong He To: dev@dpdk.org Cc: oss-drivers@corigine.com, niklas.soderlund@corigine.com, Chaoyong He Subject: [PATCH v10 06/13] net/nfp: add flower ctrl VNIC related logics Date: Mon, 26 Sep 2022 14:59:50 +0800 Message-Id: <1664175597-37248-7-git-send-email-chaoyong.he@corigine.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1664175597-37248-1-git-send-email-chaoyong.he@corigine.com> References: <1664175597-37248-1-git-send-email-chaoyong.he@corigine.com> X-ClientProxiedBy: SG2P153CA0005.APCP153.PROD.OUTLOOK.COM (2603:1096::15) To SJ0PR13MB5545.namprd13.prod.outlook.com (2603:10b6:a03:424::5) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SJ0PR13MB5545:EE_|SA1PR13MB4893:EE_ X-MS-Office365-Filtering-Correlation-Id: 3be667f6-0ad2-4dff-8ccd-08da9f8ccc44 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: EdUxgO/1A4fxcYFBl49rMPDPwo/Sb3D0qNP2iPDP6c91gQmNYB0UGhYkrU5I/cVFLrl3oprBI+8umXHKGKqrHxDLVxjJcbhFHCk3jk26TFnXO4Vz3VtpXWIYSLZPowpdbNdqshEMUV8o8i3r23a4Sz9DBQvYuWtjzGt+ntkE9xw3n0VXmzpGQlk1m1g9w2tPfk4RcUKqUc8xBUSKKc7y6FOj3ipSLgOn5ZKQcfJlWits4XiXJnafVek459UhEeShVqNOBXXquXO11jDJWuU2RE8UM2lPuHhB4rt/H2hHF9/mCOFjYZJX9w0wJ65EMwzG0MNSK8WoMuvGJIMjcLGrdWr3SZg5T0vXdzKLN8Zt84ZAm/6OvnUPJYEF7l3I5DtYVBtveYh+ag9VvRyytmuAy1VZBqaSWXKRYSzckv6NTyHA3rBdo5q4A9cKSgUhq+D0BJzpJKzJUUBpo+iDRALJWlfOODA5u1Nf2ncgPeXn7CbEthjnTkrHLMMMP9nAEbtgGX2u4LuDCzhfsKt9qqjNhD+J7IsNS4zsxLvE1LfEmmuhSQDF8n8XNt2UN6ISSbachDigHR++7gnxge0Ceb0NIx+7wLHtyq3ZeiUJR0fcV+Vbbp0DbB+YU8l4Ei/e/O+Rw/5lf3QpYualDorcLF2KyDwlfvnFlzUtc30mR2QGimpg7FXNdT3HeV3134bfnNg/JpyjKgODYysc1ssTGSoRzqa3TOyFwzgqZnMux39+2zH2VRnFn3RVupDeV+qDIFXXbuvHglX/CC7M/AZFisTxvA== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SJ0PR13MB5545.namprd13.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230022)(4636009)(366004)(396003)(39830400003)(136003)(376002)(346002)(451199015)(36756003)(38100700002)(38350700002)(2906002)(86362001)(44832011)(30864003)(66946007)(8936002)(66556008)(66476007)(5660300002)(8676002)(4326008)(83380400001)(186003)(66574015)(6512007)(6916009)(316002)(52116002)(6666004)(2616005)(26005)(6506007)(478600001)(41300700001)(107886003)(6486002); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?S9zN8HY0dYrAsWd2hmbqJ8o3gHep?= =?utf-8?q?5XIGvChGj/YhjqZBcU92Q/L3inC7y5Sj4eS16/JyIg7+LYjF/h2MS01RK8o9HABhg?= =?utf-8?q?6iVkNls8L8xZKAKh75fHzua0nGFM2p/zN8Iadc5NIaOC9tHo6p1SsUcTrP3oyKSve?= =?utf-8?q?YsjQLl3Zs3scZJQyKf/sIir/HbUJbID3NDBDv56P6iMHYuctj7S/PUphFbzoqtZDy?= =?utf-8?q?MlxvNqX5eQLET/ruODfy4CIx+LEr6RTX5YzQhE//eY/iC5IkD+buM/Jbajb3dDPaO?= =?utf-8?q?UrlDplRSOxCn7H5bt4xOQLrhkEErH58dNqikfvbUUafo6Rx0Hpa4bOgYU7ylYUtQR?= =?utf-8?q?QcUbNN7Qxu7y6e0omSppF55+KWs8Sv4byn3ozxbnosv0hTApdQm8FwiVE6xNTSU3Y?= =?utf-8?q?0gCZ9f/FpiqHMx/F1KScIuDzlTJ2GjbvtCPNZUVkUP79dhXbo6IMRaFIbBEInGwza?= =?utf-8?q?lcIFYgsqB8Y3OfzuV7L5d/D46lvb3hd++gdUOrPT/aLTPZ+SPT7DEeQyc7ldFn+wg?= =?utf-8?q?jpAU7/FJ6qpNiFL5OmQMZTM4bvYThI+kkVCMZdVX7euz7lMkiUmfdo9Htetwky6OH?= =?utf-8?q?NbtpTdJ74riENtFGPkY1sUnfMkkKHG9AbjqOn/i32TVh0uXwPZ+kKKIbLp99ND5r1?= =?utf-8?q?7ARr1ttxWptD0dbW84pBU+FwkW4Du7tLa4vbMiP8K6v3YYHKh8smNysM3kMFrCBEE?= =?utf-8?q?QFxcVWUmzJZcz+xPDkImYmlGmMOd9sud9Wg5rnlpFXincTAwja3oTZkYWCa7ke/UU?= =?utf-8?q?8xq3pWTVSROQ6Qg2qLr0xNb0+xGPqEZnwTTsOCPN/IjhlySjnsrbNNwZSkYc7je6I?= =?utf-8?q?cy5OfeLhDg34XvCJG1rBl9nR1R9wqRoceja5GD+jukXibh6iJRxzPufP27GV17rHl?= =?utf-8?q?qI/CUQwkI5FV+K1KM0L+f1T/FS3moDgbeCQNnIo97j/cRNQFAhVwzyIgpF9JSlwuz?= =?utf-8?q?yQPqht1ubgFBckV1UHjeAGYMY6kGPlF/VwCnQ5bheh08gCndmVkfk1mQ6BjMAqxjk?= =?utf-8?q?Mnovny9odRn7osZTO5ogkkByay+ndRSvgRQ3UmjMGtlfLfs1ePCLhITvz/7cBIJZa?= =?utf-8?q?aMLwxBN4CTyXGtiQIf/vw+0dRhRj0oWDnZMO2E0hqAUeTUTclbHZmvnS8HlKDqfII?= =?utf-8?q?Im0CbunrTZQK1jSiLuvcVG4O/RZcFUbFMDR8VRLVPr973zLhmpKplksJYNNK8a69p?= =?utf-8?q?myLsWRIpc6d1VU9dI8ZU0SG9sGZnWf2QWAcqHjBnrXPcxnnTDIr28lqF0COxB7rLo?= =?utf-8?q?drj3SsxjjaCh2pojOLPE2xBJ9M6D3ZlywKAEP37kBUd/ygV/bXNCPKloGd4MJ6Hev?= =?utf-8?q?FS0s9MWJHvvdr3Oba+6EcK5IYS1YwB/U1emIolLKn2SLWvJYeraKvTS3SxVnBCzFO?= =?utf-8?q?B8oHRdjSeET25CB5jfOLbsUCX6CI/TrzB6UI2pYIgOjk8O+oEmhioaf7/1D4HWkKi?= =?utf-8?q?uNSmLQZIP6kZ9qysOQ0ndCPO4TTS09iBBQwdTBGfKus0is8fJ8TBaiZtuqAGiVb5j?= =?utf-8?q?erktlRI41ERL3ApmzYrTTh5ugCQHm3jPvg=3D=3D?= X-OriginatorOrg: corigine.com X-MS-Exchange-CrossTenant-Network-Message-Id: 3be667f6-0ad2-4dff-8ccd-08da9f8ccc44 X-MS-Exchange-CrossTenant-AuthSource: SJ0PR13MB5545.namprd13.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 26 Sep 2022 07:00:30.0454 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: fe128f2c-073b-4c20-818e-7246a585940c X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: v2deLtRXhgVriLkZJx0afkMNkkXA0XckBWXcqFwm87s/uH/LKiXGMUNKJbSKr0HYtAbFk6731b1biCQIIwhNzdpJgyBr1JwoVbEUxHCrHuI= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR13MB4893 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Adds the setup/start logic for the ctrl vNIC. This vNIC is used by the PMD and flower firmware application as a communication channel between driver and firmware. In the case of OVS it is also used to communicate flow statistics from hardware to the driver. A rte_eth device is not exposed to DPDK for this vNIC as it is strictly used internally by flower logic. Because of the add of ctrl vNIC, a new PCItoCPPBar is needed. Modify the related logics. Signed-off-by: Chaoyong He Reviewed-by: Niklas Söderlund --- drivers/net/nfp/flower/nfp_flower.c | 376 +++++++++++++++++++++++++++++ drivers/net/nfp/flower/nfp_flower.h | 6 + drivers/net/nfp/nfpcore/nfp_cpp_pcie_ops.c | 31 ++- 3 files changed, 401 insertions(+), 12 deletions(-) diff --git a/drivers/net/nfp/flower/nfp_flower.c b/drivers/net/nfp/flower/nfp_flower.c index aec9e28..efa8c6b 100644 --- a/drivers/net/nfp/flower/nfp_flower.c +++ b/drivers/net/nfp/flower/nfp_flower.c @@ -20,6 +20,7 @@ #include "../nfpcore/nfp_nsp.h" #include "nfp_flower.h" +#define CTRL_VNIC_NB_DESC 512 #define DEFAULT_FLBUF_SIZE 9216 static const struct eth_dev_ops nfp_flower_pf_vnic_ops = { @@ -98,12 +99,353 @@ return 0; } +static int +nfp_flower_init_ctrl_vnic(struct nfp_net_hw *hw) +{ + uint32_t i; + int ret = 0; + uint16_t n_txq; + uint16_t n_rxq; + unsigned int numa_node; + struct rte_mempool *mp; + struct nfp_net_rxq *rxq; + struct nfp_net_txq *txq; + struct nfp_pf_dev *pf_dev; + struct rte_eth_dev *eth_dev; + const struct rte_memzone *tz; + struct nfp_app_fw_flower *app_fw_flower; + + /* Set up some pointers here for ease of use */ + pf_dev = hw->pf_dev; + app_fw_flower = NFP_PRIV_TO_APP_FW_FLOWER(pf_dev->app_fw_priv); + + ret = nfp_flower_init_vnic_common(hw, "ctrl_vnic"); + if (ret != 0) { + PMD_INIT_LOG(ERR, "Could not init pf vnic"); + return -EINVAL; + } + + /* Allocate memory for the eth_dev of the vNIC */ + hw->eth_dev = rte_zmalloc("nfp_ctrl_vnic", + sizeof(struct rte_eth_dev), RTE_CACHE_LINE_SIZE); + if (hw->eth_dev == NULL) { + PMD_INIT_LOG(ERR, "Could not allocate ctrl vnic"); + return -ENOMEM; + } + + /* Grab the pointer to the newly created rte_eth_dev here */ + eth_dev = hw->eth_dev; + + /* Also allocate memory for the data part of the eth_dev */ + eth_dev->data = rte_zmalloc("nfp_ctrl_vnic_data", + sizeof(struct rte_eth_dev_data), RTE_CACHE_LINE_SIZE); + if (eth_dev->data == NULL) { + PMD_INIT_LOG(ERR, "Could not allocate ctrl vnic data"); + ret = -ENOMEM; + goto eth_dev_cleanup; + } + + /* Create a mbuf pool for the ctrl vNIC */ + numa_node = rte_socket_id(); + app_fw_flower->ctrl_pktmbuf_pool = rte_pktmbuf_pool_create("ctrl_mbuf_pool", + 4 * CTRL_VNIC_NB_DESC, 64, 0, 9216, numa_node); + if (app_fw_flower->ctrl_pktmbuf_pool == NULL) { + PMD_INIT_LOG(ERR, "Create mbuf pool for ctrl vnic failed"); + ret = -ENOMEM; + goto dev_data_cleanup; + } + + mp = app_fw_flower->ctrl_pktmbuf_pool; + + /* Configure the ctrl vNIC device */ + n_rxq = hw->max_rx_queues; + n_txq = hw->max_tx_queues; + eth_dev->data->rx_queues = rte_zmalloc("ethdev->rx_queues", + sizeof(eth_dev->data->rx_queues[0]) * n_rxq, + RTE_CACHE_LINE_SIZE); + if (eth_dev->data->rx_queues == NULL) { + PMD_INIT_LOG(ERR, "rte_zmalloc failed for ctrl vNIC rx queues"); + ret = -ENOMEM; + goto mempool_cleanup; + } + + eth_dev->data->tx_queues = rte_zmalloc("ethdev->tx_queues", + sizeof(eth_dev->data->tx_queues[0]) * n_txq, + RTE_CACHE_LINE_SIZE); + if (eth_dev->data->tx_queues == NULL) { + PMD_INIT_LOG(ERR, "rte_zmalloc failed for ctrl vNIC tx queues"); + ret = -ENOMEM; + goto rx_queue_free; + } + + /* Fill in some of the eth_dev fields */ + eth_dev->device = &pf_dev->pci_dev->device; + eth_dev->data->nb_tx_queues = n_rxq; + eth_dev->data->nb_rx_queues = n_txq; + eth_dev->data->dev_private = hw; + + /* Set up the Rx queues */ + for (i = 0; i < n_rxq; i++) { + rxq = rte_zmalloc_socket("ethdev RX queue", + sizeof(struct nfp_net_rxq), RTE_CACHE_LINE_SIZE, + numa_node); + if (rxq == NULL) { + PMD_DRV_LOG(ERR, "Error allocating rxq"); + ret = -ENOMEM; + goto rx_queue_setup_cleanup; + } + + eth_dev->data->rx_queues[i] = rxq; + + /* Hw queues mapping based on firmware configuration */ + rxq->qidx = i; + rxq->fl_qcidx = i * hw->stride_rx; + rxq->rx_qcidx = rxq->fl_qcidx + (hw->stride_rx - 1); + rxq->qcp_fl = hw->rx_bar + NFP_QCP_QUEUE_OFF(rxq->fl_qcidx); + rxq->qcp_rx = hw->rx_bar + NFP_QCP_QUEUE_OFF(rxq->rx_qcidx); + + /* + * Tracking mbuf size for detecting a potential mbuf overflow due to + * RX offset + */ + rxq->mem_pool = mp; + rxq->mbuf_size = rxq->mem_pool->elt_size; + rxq->mbuf_size -= (sizeof(struct rte_mbuf) + RTE_PKTMBUF_HEADROOM); + hw->flbufsz = rxq->mbuf_size; + + rxq->rx_count = CTRL_VNIC_NB_DESC; + rxq->rx_free_thresh = DEFAULT_RX_FREE_THRESH; + rxq->drop_en = 1; + + /* + * Allocate RX ring hardware descriptors. A memzone large enough to + * handle the maximum ring size is allocated in order to allow for + * resizing in later calls to the queue setup function. + */ + tz = rte_eth_dma_zone_reserve(eth_dev, "ctrl_rx_ring", i, + sizeof(struct nfp_net_rx_desc) * NFP_NET_MAX_RX_DESC, + NFP_MEMZONE_ALIGN, numa_node); + if (tz == NULL) { + PMD_DRV_LOG(ERR, "Error allocating rx dma"); + rte_free(rxq); + ret = -ENOMEM; + goto rx_queue_setup_cleanup; + } + + /* Saving physical and virtual addresses for the RX ring */ + rxq->dma = (uint64_t)tz->iova; + rxq->rxds = (struct nfp_net_rx_desc *)tz->addr; + + /* Mbuf pointers array for referencing mbufs linked to RX descriptors */ + rxq->rxbufs = rte_zmalloc_socket("rxq->rxbufs", + sizeof(*rxq->rxbufs) * CTRL_VNIC_NB_DESC, + RTE_CACHE_LINE_SIZE, numa_node); + if (rxq->rxbufs == NULL) { + rte_eth_dma_zone_free(eth_dev, "ctrl_rx_ring", i); + rte_free(rxq); + ret = -ENOMEM; + goto rx_queue_setup_cleanup; + } + + nfp_net_reset_rx_queue(rxq); + + rxq->hw = hw; + + /* + * Telling the HW about the physical address of the RX ring and number + * of descriptors in log2 format + */ + nn_cfg_writeq(hw, NFP_NET_CFG_RXR_ADDR(i), rxq->dma); + nn_cfg_writeb(hw, NFP_NET_CFG_RXR_SZ(i), rte_log2_u32(CTRL_VNIC_NB_DESC)); + } + + /* Set up the Tx queues */ + for (i = 0; i < n_txq; i++) { + txq = rte_zmalloc_socket("ethdev TX queue", + sizeof(struct nfp_net_txq), RTE_CACHE_LINE_SIZE, + numa_node); + if (txq == NULL) { + PMD_DRV_LOG(ERR, "Error allocating txq"); + ret = -ENOMEM; + goto tx_queue_setup_cleanup; + } + + eth_dev->data->tx_queues[i] = txq; + + /* + * Allocate TX ring hardware descriptors. A memzone large enough to + * handle the maximum ring size is allocated in order to allow for + * resizing in later calls to the queue setup function. + */ + tz = rte_eth_dma_zone_reserve(eth_dev, "ctrl_tx_ring", i, + sizeof(struct nfp_net_nfd3_tx_desc) * NFP_NET_MAX_TX_DESC, + NFP_MEMZONE_ALIGN, numa_node); + if (tz == NULL) { + PMD_DRV_LOG(ERR, "Error allocating tx dma"); + rte_free(txq); + ret = -ENOMEM; + goto tx_queue_setup_cleanup; + } + + txq->tx_count = CTRL_VNIC_NB_DESC; + txq->tx_free_thresh = DEFAULT_RX_FREE_THRESH; + txq->tx_pthresh = DEFAULT_TX_PTHRESH; + txq->tx_hthresh = DEFAULT_TX_HTHRESH; + txq->tx_wthresh = DEFAULT_TX_WTHRESH; + + /* Queue mapping based on firmware configuration */ + txq->qidx = i; + txq->tx_qcidx = i * hw->stride_tx; + txq->qcp_q = hw->tx_bar + NFP_QCP_QUEUE_OFF(txq->tx_qcidx); + + /* Saving physical and virtual addresses for the TX ring */ + txq->dma = (uint64_t)tz->iova; + txq->txds = (struct nfp_net_nfd3_tx_desc *)tz->addr; + + /* Mbuf pointers array for referencing mbufs linked to TX descriptors */ + txq->txbufs = rte_zmalloc_socket("txq->txbufs", + sizeof(*txq->txbufs) * CTRL_VNIC_NB_DESC, + RTE_CACHE_LINE_SIZE, numa_node); + if (txq->txbufs == NULL) { + rte_eth_dma_zone_free(eth_dev, "ctrl_tx_ring", i); + rte_free(txq); + ret = -ENOMEM; + goto tx_queue_setup_cleanup; + } + + nfp_net_reset_tx_queue(txq); + + txq->hw = hw; + + /* + * Telling the HW about the physical address of the TX ring and number + * of descriptors in log2 format + */ + nn_cfg_writeq(hw, NFP_NET_CFG_TXR_ADDR(i), txq->dma); + nn_cfg_writeb(hw, NFP_NET_CFG_TXR_SZ(i), rte_log2_u32(CTRL_VNIC_NB_DESC)); + } + + return 0; + +tx_queue_setup_cleanup: + for (i = 0; i < hw->max_tx_queues; i++) { + txq = eth_dev->data->tx_queues[i]; + if (txq != NULL) { + rte_free(txq->txbufs); + rte_eth_dma_zone_free(eth_dev, "ctrl_tx_ring", i); + rte_free(txq); + } + } +rx_queue_setup_cleanup: + for (i = 0; i < hw->max_rx_queues; i++) { + rxq = eth_dev->data->rx_queues[i]; + if (rxq != NULL) { + rte_free(rxq->rxbufs); + rte_eth_dma_zone_free(eth_dev, "ctrl_rx_ring", i); + rte_free(rxq); + } + } + rte_free(eth_dev->data->tx_queues); +rx_queue_free: + rte_free(eth_dev->data->rx_queues); +mempool_cleanup: + rte_mempool_free(mp); +dev_data_cleanup: + rte_free(eth_dev->data); +eth_dev_cleanup: + rte_free(eth_dev); + + return ret; +} + +static void +nfp_flower_cleanup_ctrl_vnic(struct nfp_net_hw *hw) +{ + uint32_t i; + struct nfp_net_rxq *rxq; + struct nfp_net_txq *txq; + struct rte_eth_dev *eth_dev; + struct nfp_app_fw_flower *app_fw_flower; + + eth_dev = hw->eth_dev; + app_fw_flower = NFP_PRIV_TO_APP_FW_FLOWER(hw->pf_dev->app_fw_priv); + + for (i = 0; i < hw->max_tx_queues; i++) { + txq = eth_dev->data->tx_queues[i]; + if (txq != NULL) { + rte_free(txq->txbufs); + rte_eth_dma_zone_free(eth_dev, "ctrl_tx_ring", i); + rte_free(txq); + } + } + + for (i = 0; i < hw->max_rx_queues; i++) { + rxq = eth_dev->data->rx_queues[i]; + if (rxq != NULL) { + rte_free(rxq->rxbufs); + rte_eth_dma_zone_free(eth_dev, "ctrl_rx_ring", i); + rte_free(rxq); + } + } + + rte_free(eth_dev->data->tx_queues); + rte_free(eth_dev->data->rx_queues); + rte_mempool_free(app_fw_flower->ctrl_pktmbuf_pool); + rte_free(eth_dev->data); + rte_free(eth_dev); +} + +static int +nfp_flower_start_ctrl_vnic(struct nfp_net_hw *hw) +{ + int ret; + uint32_t update; + uint32_t new_ctrl; + struct rte_eth_dev *dev; + + dev = hw->eth_dev; + + /* Disabling queues just in case... */ + nfp_net_disable_queues(dev); + + /* Enabling the required queues in the device */ + nfp_net_enable_queues(dev); + + /* Writing configuration parameters in the device */ + nfp_net_params_setup(hw); + + new_ctrl = NFP_NET_CFG_CTRL_ENABLE; + update = NFP_NET_CFG_UPDATE_GEN | NFP_NET_CFG_UPDATE_RING | + NFP_NET_CFG_UPDATE_MSIX; + + rte_wmb(); + + /* If an error when reconfig we avoid to change hw state */ + ret = nfp_net_reconfig(hw, new_ctrl, update); + if (ret != 0) { + PMD_INIT_LOG(ERR, "Failed to reconfig ctrl vnic"); + return -EIO; + } + + hw->ctrl = new_ctrl; + + /* Setup the freelist ring */ + ret = nfp_net_rx_freelist_setup(dev); + if (ret != 0) { + PMD_INIT_LOG(ERR, "Error with flower ctrl vNIC freelist setup"); + return -EIO; + } + + return 0; +} + int nfp_init_app_fw_flower(struct nfp_pf_dev *pf_dev) { int ret; unsigned int numa_node; struct nfp_net_hw *pf_hw; + struct nfp_net_hw *ctrl_hw; struct nfp_app_fw_flower *app_fw_flower; numa_node = rte_socket_id(); @@ -148,8 +490,42 @@ goto pf_cpp_area_cleanup; } + /* The ctrl vNIC struct comes directly after the PF one */ + app_fw_flower->ctrl_hw = pf_hw + 1; + ctrl_hw = app_fw_flower->ctrl_hw; + + /* Map the ctrl vNIC ctrl bar */ + ctrl_hw->ctrl_bar = nfp_rtsym_map(pf_dev->sym_tbl, "_pf0_net_ctrl_bar", + 32768, &ctrl_hw->ctrl_area); + if (ctrl_hw->ctrl_bar == NULL) { + PMD_INIT_LOG(ERR, "Cloud not map the ctrl vNIC ctrl bar"); + ret = -ENODEV; + goto pf_cpp_area_cleanup; + } + + /* Now populate the ctrl vNIC */ + ctrl_hw->pf_dev = pf_dev; + ctrl_hw->cpp = pf_dev->cpp; + + ret = nfp_flower_init_ctrl_vnic(app_fw_flower->ctrl_hw); + if (ret != 0) { + PMD_INIT_LOG(ERR, "Could not initialize flower ctrl vNIC"); + goto ctrl_cpp_area_cleanup; + } + + /* Start the ctrl vNIC */ + ret = nfp_flower_start_ctrl_vnic(app_fw_flower->ctrl_hw); + if (ret != 0) { + PMD_INIT_LOG(ERR, "Could not start flower ctrl vNIC"); + goto ctrl_vnic_cleanup; + } + return 0; +ctrl_vnic_cleanup: + nfp_flower_cleanup_ctrl_vnic(app_fw_flower->ctrl_hw); +ctrl_cpp_area_cleanup: + nfp_cpp_area_free(ctrl_hw->ctrl_area); pf_cpp_area_cleanup: nfp_cpp_area_free(pf_dev->ctrl_area); vnic_cleanup: diff --git a/drivers/net/nfp/flower/nfp_flower.h b/drivers/net/nfp/flower/nfp_flower.h index 51e05e8..7b138d8 100644 --- a/drivers/net/nfp/flower/nfp_flower.h +++ b/drivers/net/nfp/flower/nfp_flower.h @@ -10,6 +10,12 @@ struct nfp_app_fw_flower { /* Pointer to the PF vNIC */ struct nfp_net_hw *pf_hw; + + /* Pointer to a mempool for the ctrlvNIC */ + struct rte_mempool *ctrl_pktmbuf_pool; + + /* Pointer to the ctrl vNIC */ + struct nfp_net_hw *ctrl_hw; }; int nfp_init_app_fw_flower(struct nfp_pf_dev *pf_dev); diff --git a/drivers/net/nfp/nfpcore/nfp_cpp_pcie_ops.c b/drivers/net/nfp/nfpcore/nfp_cpp_pcie_ops.c index 08bc4e8..22c8bc4 100644 --- a/drivers/net/nfp/nfpcore/nfp_cpp_pcie_ops.c +++ b/drivers/net/nfp/nfpcore/nfp_cpp_pcie_ops.c @@ -91,7 +91,10 @@ * @refcnt: number of current users * @iomem: mapped IO memory */ +#define NFP_BAR_MIN 1 +#define NFP_BAR_MID 5 #define NFP_BAR_MAX 7 + struct nfp_bar { struct nfp_pcie_user *nfp; uint32_t barcfg; @@ -292,6 +295,7 @@ struct nfp_pcie_user { * BAR0.0: Reserved for General Mapping (for MSI-X access to PCIe SRAM) * * Halving PCItoCPPBars for primary and secondary processes. + * For CoreNIC firmware: * NFP PMD just requires two fixed slots, one for configuration BAR, * and another for accessing the hw queues. Another slot is needed * for setting the link up or down. Secondary processes do not need @@ -301,6 +305,9 @@ struct nfp_pcie_user { * supported. Due to this requirement and future extensions requiring * new slots per process, only one secondary process is supported by * now. + * For Flower firmware: + * NFP PMD need another fixed slots, used as the configureation BAR + * for ctrl vNIC. */ static int nfp_enable_bars(struct nfp_pcie_user *nfp) @@ -309,11 +316,11 @@ struct nfp_pcie_user { int x, start, end; if (rte_eal_process_type() == RTE_PROC_PRIMARY) { - start = 4; - end = 1; + start = NFP_BAR_MID; + end = NFP_BAR_MIN; } else { - start = 7; - end = 4; + start = NFP_BAR_MAX; + end = NFP_BAR_MID; } for (x = start; x > end; x--) { bar = &nfp->bar[x - 1]; @@ -341,11 +348,11 @@ struct nfp_pcie_user { int x, start, end; if (rte_eal_process_type() == RTE_PROC_PRIMARY) { - start = 4; - end = 1; + start = NFP_BAR_MID; + end = NFP_BAR_MIN; } else { - start = 7; - end = 4; + start = NFP_BAR_MAX; + end = NFP_BAR_MID; } for (x = start; x > end; x--) { bar = &nfp->bar[x - 1]; @@ -364,11 +371,11 @@ struct nfp_pcie_user { int x, start, end; if (rte_eal_process_type() == RTE_PROC_PRIMARY) { - start = 4; - end = 1; + start = NFP_BAR_MID; + end = NFP_BAR_MIN; } else { - start = 7; - end = 4; + start = NFP_BAR_MAX; + end = NFP_BAR_MID; } for (x = start; x > end; x--) { From patchwork Mon Sep 26 06:59:51 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Chaoyong He X-Patchwork-Id: 116848 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 7C9A8A00C2; Mon, 26 Sep 2022 09:01:19 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id A084142B80; Mon, 26 Sep 2022 09:00:36 +0200 (CEST) Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2130.outbound.protection.outlook.com [40.107.244.130]) by mails.dpdk.org (Postfix) with ESMTP id BC1C742B75 for ; Mon, 26 Sep 2022 09:00:33 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=mj6QifDCwpMfIqy/i0TL5I7Hr7+bJ7gXrKpEU1PNztbFH27y+paK321FC/qgzMzW/hZh2WzG75VgzhbpZOKiKUo9sUYUG3B6yAYAIPxDz7ECq2MeEVQugP3L6QGOD8LqoeNPCkc6xygM+M5DJp1hGYK0h0yXSDgnqMfzI1RB9qL8NQRmmSYVMY/jblei2yWCZMAQZMVFRO1CdiPY1fxAfl5auYmyzgnwIbA1IXZ+S7AdjxwVFDdqUXyiQ3NzeIwjWHi5TxOrJ7r6ZYXccTFpDPr+Ug9hysLc6aZyhQmrGtRjMnv5Tq0gy38Xk1KcaNFXzWIQr8yMspZs27A/UFdV+Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=HVKuP7tUbIZLw6xgolmmPujd42xFro8GShmm2ItScIM=; b=W1H1sZinDHmht5QyaUL2AUEqVcn4j1fFH4egIOwwGnh4+m5ixO44FDi0nsjsja5fh2K2blAbguO7xSkKI/Ov3cjOtN98yHkj+4JJpe4OzLUdAP2+2FyVG+vCSSWC3/Yo8VBwdAXxQwSFwqX1rij5UKs3iaPNTwZ2sHIm0xhey8acwGRxkymIYp189w2vP78tmYKbQ4LRKoohY3VkLbTCunhSGpntL2PWYaP+Tuudq1OVKJtHwWXVOYgY4zYecVm59VKUEOh6zIaA4/mSx/KLycX7glhZ599nkILdr+zF8j1gC8z0CW0c6JaOmqxeIMecxlXS/4uXzNmhgbgfhJhFxg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=corigine.com; dmarc=pass action=none header.from=corigine.com; dkim=pass header.d=corigine.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=corigine.onmicrosoft.com; s=selector2-corigine-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=HVKuP7tUbIZLw6xgolmmPujd42xFro8GShmm2ItScIM=; b=pSm2d8UbvUqjOmovlEntAjVCmxdhtWncCHZOYpHOxWvcmpq5Jzx+n4Q8zUvmo4+PZdcnpLE7B0ADzwHz35+kEc8+QC/gZePj1QdGt4ODbINbelwf8hn434i/ZwUkYKBH8AyZr/rQrRW6RfR7q2XQhp1P8O0MpQwkztPp6+/IEao= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=corigine.com; Received: from SJ0PR13MB5545.namprd13.prod.outlook.com (2603:10b6:a03:424::5) by SA1PR13MB4893.namprd13.prod.outlook.com (2603:10b6:806:18a::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5676.14; Mon, 26 Sep 2022 07:00:32 +0000 Received: from SJ0PR13MB5545.namprd13.prod.outlook.com ([fe80::779:2722:a8e5:503b]) by SJ0PR13MB5545.namprd13.prod.outlook.com ([fe80::779:2722:a8e5:503b%7]) with mapi id 15.20.5676.014; Mon, 26 Sep 2022 07:00:32 +0000 From: Chaoyong He To: dev@dpdk.org Cc: oss-drivers@corigine.com, niklas.soderlund@corigine.com, Chaoyong He , Heinrich Kuhn Subject: [PATCH v10 07/13] net/nfp: move common rxtx function for flower use Date: Mon, 26 Sep 2022 14:59:51 +0800 Message-Id: <1664175597-37248-8-git-send-email-chaoyong.he@corigine.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1664175597-37248-1-git-send-email-chaoyong.he@corigine.com> References: <1664175597-37248-1-git-send-email-chaoyong.he@corigine.com> X-ClientProxiedBy: SG2P153CA0005.APCP153.PROD.OUTLOOK.COM (2603:1096::15) To SJ0PR13MB5545.namprd13.prod.outlook.com (2603:10b6:a03:424::5) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SJ0PR13MB5545:EE_|SA1PR13MB4893:EE_ X-MS-Office365-Filtering-Correlation-Id: 0808a3e4-223b-4979-9640-08da9f8ccd9e X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Yx7rpEae/BQOkYcq398pgnVYOothoAWziIblJpo3RM/3291Wtc4xjTjoQiGnwYh3fzBue6JFriJ26d4bDggvoYrox9ntgGHjJEDyBnc6lEjtnXjxQXJU+8Vtnyi+x2wdoMI+kzsR1uL2QmwqwkgZ4CT3kuBDvMYP8KKIaXFIf+/TJL+1DVsByFyfDoPwkUhUock5AxIQmDxmg3yIIWWlFoGdKMX0qFQcbHfC+N0c8Qql9VlYofrbFwa6gdWNxGyzoMnAdM2Va8nuCqUdyI8aRSbqz6JUgIfi5KvPNcIsC+yG1xMUMHP8CZmgYesX9+ubVtKOW66XbSqnNT+To+YS7rWGezzq1j4e/AUxzDdymgZNdsRZFYRrcJmXUuwjfs5qwIjOEemDpUZJzLFtYWAnQwcF1xALmErCEDYGNHVBk8Sz/qraKD+3it3C/0pDLknACnVGT5WCxa2kiJ7tn/dN063/Xs0T5lovAlUaUuBBJgLprVHhAk+/Ra5MtNgzS8cS0InE7bjGvNRLopyWZV9obuQR++56LlkNqJgj0+BqLQvi8HAAg7j5FUDKgyxc8/iQc9MQeZeAwy3jLPv+ci4/gWr65Plo8Xnor/eTifIvvOe4b/TUy7D+JQ1PmsnlpCOktZ7V9euTK3Uay0zBmoBWpJscXEM8lnjzBgpQOfSOtrAoXR9JfYVSHv67IgCcT+i9c50YizZBFzCAybaR1MZpQpEEKLKFEg3Yg/BNk6g/hS3XNVJH39ynEdr82VYGvnK7wQciM8vHvC1LO7rkjH9BXQ== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SJ0PR13MB5545.namprd13.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230022)(4636009)(366004)(396003)(39830400003)(136003)(376002)(346002)(451199015)(36756003)(38100700002)(38350700002)(2906002)(86362001)(44832011)(66946007)(8936002)(66556008)(66476007)(5660300002)(8676002)(4326008)(83380400001)(186003)(54906003)(6512007)(6916009)(316002)(52116002)(6666004)(2616005)(26005)(6506007)(478600001)(41300700001)(107886003)(6486002); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?rEnW+ClJk0VOpA0CtsfNORTTbB+3?= =?utf-8?q?DQlb3AK6gtTzEIzjCMBOOyOI9bSMEwE97IBy0RsJBwdn1jdEy20jO0RezZDSrx0Lt?= =?utf-8?q?n61GX9oyJ7vIYNyCZWE9/A4LO0rEJ6DI8bl7PXzvCW2f/0GUo9yd54sPMy5MNzigw?= =?utf-8?q?Cc2aS4eLGV/qI0zRRb5WcsHDy6McsoSU8vSalqzkaU6XBY8Y+Y0QqyPC7M0Ks1N1d?= =?utf-8?q?ybnWlwr4WhxlgL5Q+DR3v+rWzg8UAYtso9uLZBIsHDSTcixfMXBDfkTlCrYKTd2Dd?= =?utf-8?q?otHh3EXuJMheKn4AY2o9PJy7yCJta5/P5zE3ChlM33/bpvJF9tJJZx1ZwNNJOBJVI?= =?utf-8?q?vU1xaT5YoQEakv4oogV70W6+J5CZoqhY3OEMD9qa7nFXyl/G1wzpusXYTg4D3Y3EK?= =?utf-8?q?sH4F+w0dYVGY1FcWOM2elmBISQK7aS2ceruY/QHgtfPUDfjcJWAJ/Kmb2lOKKeFTJ?= =?utf-8?q?aCNWobaqFHnnRBgyeZZYTw/4zmGArrIJtBUWab3B1KGkF2aWYS3SGZvSOr+D6p2Iw?= =?utf-8?q?CQbOfY3srTLEVsYyCBYZtOYrfcpDjRLaZkI0OzHuPh86LIzCgpr947Z26/ad3OD0k?= =?utf-8?q?K24TX8KPr69v/wRbPVPcWGcOKXAWYzUc80aNCLcNXE3rYZChV/7uXNjCb7ggnqjkb?= =?utf-8?q?o/LPaeqJ4QK8+7No5xGKF/dZnsNi34l5YxcMbHQSo2VDERkzVOKPK8WQ1UfQ+ge9m?= =?utf-8?q?x35QJYo8VW+rQWDcxfhYeplB+y71s85XYXcefd7NLJhG4VX2jYwj+w4Ls4bbd7Nwq?= =?utf-8?q?RS4XtfnwHsrqHj1V4Fcr2wjdWKwY9UvtlhZpgcbAvSS1r3k9xTU0W0hEDF/rXDYw2?= =?utf-8?q?aeOetCw0xnI74H3zofnwsFzN6vpJBAEJOq5WglbE6fNKHqBWEmMO6OLFnSoRwMFNR?= =?utf-8?q?A5jNZz1CFcOa8qRDAlfkYEimm6m94OoHKVvf6w28tsdebDR+aRYjw/psP0CTZX9tb?= =?utf-8?q?4lQrFBSY7XD2cXInvR0D5NJrEGoBjhXvdsPRR8+4imrF6XPzbE/RaAgJm8bHY7rmp?= =?utf-8?q?No2y/75ssDIh/IXv/O+hNiULkSDUtq9uiytG9NiJ4pt9K/PybXLJe3TJsce33AhdH?= =?utf-8?q?1FuQGHobHirmQ03yUiFXHKuPnbSUtWNApLLZhLGEJYzVGiQkNseJnYTSw1PBH1Twl?= =?utf-8?q?HedvQiiwQaEKpibNbufdzRk1CsF36Nj8/eO9pu6tv/wIDYuU/vW8V13CoapxQoGzC?= =?utf-8?q?3kTvRlfFDwcT5LtqEpgXkoXY398SSXvrBui2Jq9W61a1eXMuf/NjJMJ906wLPGrdb?= =?utf-8?q?JSnOen8SDHUJbfJTlljz30dvDQGmsjPwMNWhUrLCyq03xpRQYEuyUYZvn9X1yrYJI?= =?utf-8?q?2JlnkHf9feRfVTmjQ7zgYmaCvdRyytuwGHhz8ypStHjFm/JiitObvSzAXUWpusjsj?= =?utf-8?q?pv7rMNiNblRaaSsTokTlVWgBjhFIWfHwxCkellB8RLptphRff6y4mvOi5N38teU2k?= =?utf-8?q?RxJOTHObPrDljSSEC6ggYcaw6tsaxN/uUFBJqIHIGEs62lwQFTxNSzmNHzgjEQcvQ?= =?utf-8?q?e9u5BYcxkl0YH/aAC+ouEyU7ZCfa1VW2AQ=3D=3D?= X-OriginatorOrg: corigine.com X-MS-Exchange-CrossTenant-Network-Message-Id: 0808a3e4-223b-4979-9640-08da9f8ccd9e X-MS-Exchange-CrossTenant-AuthSource: SJ0PR13MB5545.namprd13.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 26 Sep 2022 07:00:32.3119 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: fe128f2c-073b-4c20-818e-7246a585940c X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: LJbS/rzG5FigK2HCYBbROS9s5eOu4BeXz+YySe8inUyO9TSpMKVOdoBQ82RKhf9gmli9J5y1AqkT1ylFEBk1URcWvU8Oqgj42AgQ6gFX3g4= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR13MB4893 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Move some common Rx and Tx logic to the header file so that they can be re-used by flower Tx and Rx logic. Signed-off-by: Chaoyong He Signed-off-by: Heinrich Kuhn Reviewed-by: Niklas Söderlund --- drivers/net/nfp/nfp_rxtx.c | 32 +------------------------------- drivers/net/nfp/nfp_rxtx.h | 31 +++++++++++++++++++++++++++++++ 2 files changed, 32 insertions(+), 31 deletions(-) diff --git a/drivers/net/nfp/nfp_rxtx.c b/drivers/net/nfp/nfp_rxtx.c index 8429b44..8d63a7b 100644 --- a/drivers/net/nfp/nfp_rxtx.c +++ b/drivers/net/nfp/nfp_rxtx.c @@ -116,12 +116,6 @@ return count; } -static inline void -nfp_net_mbuf_alloc_failed(struct nfp_net_rxq *rxq) -{ - rte_eth_devices[rxq->port_id].data->rx_mbuf_alloc_failed++; -} - /* * nfp_net_set_hash - Set mbuf hash data * @@ -583,7 +577,7 @@ * @txq: TX queue to work with * Returns number of descriptors freed */ -static int +int nfp_net_tx_free_bufs(struct nfp_net_txq *txq) { uint32_t qcp_rd_p; @@ -774,30 +768,6 @@ return 0; } -/* Leaving always free descriptors for avoiding wrapping confusion */ -static inline -uint32_t nfp_net_nfd3_free_tx_desc(struct nfp_net_txq *txq) -{ - if (txq->wr_p >= txq->rd_p) - return txq->tx_count - (txq->wr_p - txq->rd_p) - 8; - else - return txq->rd_p - txq->wr_p - 8; -} - -/* - * nfp_net_txq_full - Check if the TX queue free descriptors - * is below tx_free_threshold - * - * @txq: TX queue to check - * - * This function uses the host copy* of read/write pointers - */ -static inline -uint32_t nfp_net_nfd3_txq_full(struct nfp_net_txq *txq) -{ - return (nfp_net_nfd3_free_tx_desc(txq) < txq->tx_free_thresh); -} - /* nfp_net_tx_tso - Set TX descriptor for TSO */ static inline void nfp_net_nfd3_tx_tso(struct nfp_net_txq *txq, struct nfp_net_nfd3_tx_desc *txd, diff --git a/drivers/net/nfp/nfp_rxtx.h b/drivers/net/nfp/nfp_rxtx.h index 5c005d7..a30171f 100644 --- a/drivers/net/nfp/nfp_rxtx.h +++ b/drivers/net/nfp/nfp_rxtx.h @@ -330,6 +330,36 @@ struct nfp_net_rxq { int rx_qcidx; } __rte_aligned(64); +static inline void +nfp_net_mbuf_alloc_failed(struct nfp_net_rxq *rxq) +{ + rte_eth_devices[rxq->port_id].data->rx_mbuf_alloc_failed++; +} + +/* Leaving always free descriptors for avoiding wrapping confusion */ +static inline uint32_t +nfp_net_nfd3_free_tx_desc(struct nfp_net_txq *txq) +{ + if (txq->wr_p >= txq->rd_p) + return txq->tx_count - (txq->wr_p - txq->rd_p) - 8; + else + return txq->rd_p - txq->wr_p - 8; +} + +/* + * nfp_net_nfd3_txq_full - Check if the TX queue free descriptors + * is below tx_free_threshold + * + * @txq: TX queue to check + * + * This function uses the host copy* of read/write pointers + */ +static inline uint32_t +nfp_net_nfd3_txq_full(struct nfp_net_txq *txq) +{ + return (nfp_net_nfd3_free_tx_desc(txq) < txq->tx_free_thresh); +} + int nfp_net_rx_freelist_setup(struct rte_eth_dev *dev); uint32_t nfp_net_rx_queue_count(void *rx_queue); uint16_t nfp_net_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, @@ -355,6 +385,7 @@ int nfp_net_nfdk_tx_queue_setup(struct rte_eth_dev *dev, uint16_t nfp_net_nfdk_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts); +int nfp_net_tx_free_bufs(struct nfp_net_txq *txq); #endif /* _NFP_RXTX_H_ */ /* From patchwork Mon Sep 26 06:59:52 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Chaoyong He X-Patchwork-Id: 116849 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id F122DA00C2; Mon, 26 Sep 2022 09:01:26 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 9E0DA42B78; Mon, 26 Sep 2022 09:00:38 +0200 (CEST) Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2110.outbound.protection.outlook.com [40.107.244.110]) by mails.dpdk.org (Postfix) with ESMTP id 3DF7442B6F for ; Mon, 26 Sep 2022 09:00:36 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=j2MDK0TzBPqJEXb5Kjau+hH15NvUPWZ33SEwsXRFPAbECvwRx2g5VcCBrUciF++YQjUSmEe0yFcYmEhZmiOfhhW2tj31gQ0Jd4j1kkfhppZbZl7lwsAuzXnsZ0HBbkezZwBYUxKOzZJhMoIFsyrHwJVNtbpww1mVGRiFJZZWNTX3Sl6Y/y1KfkJonEB7YwYEwN0FhR8fkaAei2aBe8m7VUgljm+wBnNAgdRNOwwjiU5YaQ+jN+XJcHILMLtN21rjifwsDHu8ZF/a8/+lr3L8UZUjpo1fdnrH2jQnS8g1dZXzzQZtFoXUQ/2vdXdUD2OCIJIG8Lm+zFhyy08dU/p6ag== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=+2lrBuBZ3S25L4N6U9IJ+RfWyzutl047mlVJKRG0kgc=; b=OwdK4P5hFLT55XGIdJgDkXeQGid1+9R21p83Ju7P0AfrhEtdW6KrnjWQ1eD696fwIRMLF7mOdPwi1lz41ttrbntl4xjITR8MkExNQzRGOjmjeTpgHqJueyOYQfgl6WTzG7ocWE8buDs6cY1hRW+W8ervIn2u9QpyzrRj5gmZf5I1EvejCQcA7uIh0KRgRVreBATwsLRu6nqFJoJduSJqobLeSY7w5su9tO7aPVofHPSFKI3i9dqxG4zpVozMXb/Tu/nRixH6iorKub0IMJpUBkvg4Z8JEwb7PM+5k4LxtZT8N46AGzHCDd1thk4ZtOVOmPpqn6hJLogu/ZwaQ5l2LQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=corigine.com; dmarc=pass action=none header.from=corigine.com; dkim=pass header.d=corigine.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=corigine.onmicrosoft.com; s=selector2-corigine-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=+2lrBuBZ3S25L4N6U9IJ+RfWyzutl047mlVJKRG0kgc=; b=doC5uu/soDvoK+KTnE7eBH68j0hMQWySOAoPoPO4jSQR/DdS+dymr7/EtFrzqkMfPprSfr+IpO1cIk0CMtFtUVla3XDw39ZPjAzqjJ3SVHWcq+nqwfUJqEIqzJwcV47ybUZeFsXL1vV5stJQsRrggmhW+XjBFvEVtfPvFWR4v4Q= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=corigine.com; Received: from SJ0PR13MB5545.namprd13.prod.outlook.com (2603:10b6:a03:424::5) by SA1PR13MB4893.namprd13.prod.outlook.com (2603:10b6:806:18a::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5676.14; Mon, 26 Sep 2022 07:00:34 +0000 Received: from SJ0PR13MB5545.namprd13.prod.outlook.com ([fe80::779:2722:a8e5:503b]) by SJ0PR13MB5545.namprd13.prod.outlook.com ([fe80::779:2722:a8e5:503b%7]) with mapi id 15.20.5676.014; Mon, 26 Sep 2022 07:00:34 +0000 From: Chaoyong He To: dev@dpdk.org Cc: oss-drivers@corigine.com, niklas.soderlund@corigine.com, Chaoyong He , Heinrich Kuhn Subject: [PATCH v10 08/13] net/nfp: add flower ctrl VNIC rxtx logic Date: Mon, 26 Sep 2022 14:59:52 +0800 Message-Id: <1664175597-37248-9-git-send-email-chaoyong.he@corigine.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1664175597-37248-1-git-send-email-chaoyong.he@corigine.com> References: <1664175597-37248-1-git-send-email-chaoyong.he@corigine.com> X-ClientProxiedBy: SG2P153CA0005.APCP153.PROD.OUTLOOK.COM (2603:1096::15) To SJ0PR13MB5545.namprd13.prod.outlook.com (2603:10b6:a03:424::5) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SJ0PR13MB5545:EE_|SA1PR13MB4893:EE_ X-MS-Office365-Filtering-Correlation-Id: 2684871f-f43a-426b-327b-08da9f8ccf04 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Kd7nzv3Ifbky/dwlHxvwmpP2WhMD4VoYSkTThNd5qnc4VmsLzeMcAOC/YUvJmTdzZurm3tLoCfvu3cNzqU4UA8R3s+X80qe8JlLVZRyEz+wD2gW2SMcSB5LFbWphduny1cEhvS5213UMxFgZwB6SmXjFPbBRRkZ4wBbDtVihiZfRxKQafRJXT2KF5dBh6LxpZ4eq6RjGYZDVqiPXZ2mYfgHJJSrcBRQF/YPernegilPhbh4hX/Pf1BOCXB8g2d6Hy94lfVfd7UvhWGm+R6yX36ohdAX8t0xf6b/ukXb80YoQXveOuZFhoStNzLSPptJBVLfJAB9gtYFucFide6GaadS7x+1w/zxSmxyllwmrcw/kqlY0EBx++Zbm76fAwQGj4Az1dWBQtRL9p0fN3V1DA64FhJN8IeS7oozF0Q+wBuHHeEBMaQTRYtp4UZrUveh6sKCcDD3jft4M6rcw720zKAyDVQJlpkAkFfxh/+k6jSlF0Ql49Lrb+kowMFj2fWegveBUgXdD0BZbEiQymZb5a88epcNKwBdUNr4gs6zjJW+dc/jRLI8IYbUZVAXuvxYoidyp+N/UU1VNGFZPePp/hLqaKtqy2RWMVO8v6XIfkBH3bpSou3m/IzRsRmjT80/OpNmH9tCCuUVXY8V5nxKe17PZqaet2vjzwBR3PN9w9Cl73nbC2ekk+ZXYVc8JJKvv/aQHEWiLpRKOlfMtpVX7QdyEEoVn6caFm39R0NtwPlaZgmOgnVgTFwvp3qalpoSP5K1BOkJ1urvLuUX4DWsl2A== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SJ0PR13MB5545.namprd13.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230022)(4636009)(366004)(396003)(39830400003)(136003)(376002)(346002)(451199015)(36756003)(38100700002)(38350700002)(2906002)(86362001)(44832011)(30864003)(66946007)(8936002)(66556008)(66476007)(5660300002)(8676002)(4326008)(83380400001)(186003)(54906003)(6512007)(6916009)(316002)(52116002)(6666004)(2616005)(26005)(6506007)(478600001)(41300700001)(107886003)(6486002); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?BAe6R5NRTjnriG8j2B7aS+1Oj1WW?= =?utf-8?q?jsc3uG67ClR2TTDqwbBCezOnYNoRlJ9IMUeg9u1dVnUIKoGtXKtZZNjBWG5p4g5p/?= =?utf-8?q?+UTh2mSElM94vPle9n29Sq6ChqceCvI34O5BxWQfdJfnszbSlJbJZLTHKaixiTIWQ?= =?utf-8?q?SGlyJBlekuBeGqarbRbIN6LwGyMiY8PBDyWI3tCRu/1PMqYlv/Pi7OJFsQ/B9oF/Q?= =?utf-8?q?zCASNF2DW9iu/r1x+cTCmA7gpD4EnWgIx7kwwTRN9FXC8odQ9ke2BteKYGi1pKgU+?= =?utf-8?q?09utEcnPv0lK0iYGSAXGA8tOmDlM9PiMZDqgljyQlcB3k7oC/Fa8ThDIk4TpdtQa1?= =?utf-8?q?kciq+u2PJvI7uutiYS3XPTnPPK46kYRGMgcWjvFfpJ7gsVLzYgskre9+LnvenEY1g?= =?utf-8?q?coNzp0TGHPlWLX+cA5fKxasXqKSzWVzQS2H+TjULg9s66A5FYqWyG7UBnk7t7HrM/?= =?utf-8?q?JN7o186GZe3WPaOdR9szcRFh1/KFCIDLJHRvUWfcILK3/cowjq/itq8uSUasFa4Lj?= =?utf-8?q?aHGHvmLt6If/Z3mZpXayXJQw/2+K2hLvB+pok51iGpBsg0mJNs16+qmmOzLZxnpNv?= =?utf-8?q?ABpdPFeiDC/IHxKJndtD8Pno+v9JgcWgYKS8bBbreXiOXXTTLoFxFpsyV5IEehDrK?= =?utf-8?q?9pKh0I6jNPIeOUZhymkDPKKqv+4NhG/atLad59CPA61DIRGiTPqmdIxggpZZm1WpV?= =?utf-8?q?bTBhmIhTidDZtvYsn6dZoGFEhwvZ6hhpIkIpCdpBNxIkwCPjH5j5dE689ss9HNT3v?= =?utf-8?q?Xeia0w4z9FjE2mWKa2YwkXcrD24Hc2bpq++2fNkoJZ2HH07zkIhvUdm2E583bsxPI?= =?utf-8?q?aauLfPSFnVwIS0E7LBBtzoAq/itupMr+2VC2oCQZCuj1/B2Gf6J4RDMIswwGjgig8?= =?utf-8?q?ZtQ6ZAAu4FUMxVzTEqcjpcJaCs71rn5Fq2zd7cvxMpubCTZLhEDWILMhAQSapuFfq?= =?utf-8?q?DKGVlIW08T7mLtTWpVyuEIT8a6TSuqGwcsoaSfLl6/40pND/35hgcazbALVdPzjwu?= =?utf-8?q?PEB8Wnzvk6VYirhNUbnHQEHN9G7OW2+jMlIkKqrK84jgzyNBCUJafHzu3bovxEPGK?= =?utf-8?q?KXywAvimyadDDRGHljAk603R3cQYDn2ZMDX0gyAKT1YuBsz/8wkCieLQyXCQ5mHUN?= =?utf-8?q?Er+XpHlxCSyrwKL5jZjSMtOS99uZJQkNFZcRhIyldLwL83baBaKOt9E3U1dhY0uwh?= =?utf-8?q?KBMRwCUXhXoE2VCEjpzM1wXY4qEjPMzUVBXFG8Xd+pvnuaJir9xnnDNAHiq2Fyhe/?= =?utf-8?q?fGgUs2w5oIhgXNlJ5e9PylbaBocGGlfBWWCWy8BFyclnxgwC8gssXfbOStGrNgbhX?= =?utf-8?q?nv78oQYUFmFc7PhCSQLFJXhNgKXp8sP7HfEL2vlPnCjq7KXcMceg0v8CvJKCdWBhd?= =?utf-8?q?lPhNCI9poAAclgQjHM2l70VgLLFgMepO5uCb3UTMvNdFnMha0GYHPkzjjkfUtmUUa?= =?utf-8?q?e+hZPvGVbIDD/6uwbkz34EZDPTov/A1Y1u/8baKBRxsY56FJbXHz1x4CDev2AirHn?= =?utf-8?q?wiljJeBDl3umSFDRkgI4DIeUduVTXSowRQ=3D=3D?= X-OriginatorOrg: corigine.com X-MS-Exchange-CrossTenant-Network-Message-Id: 2684871f-f43a-426b-327b-08da9f8ccf04 X-MS-Exchange-CrossTenant-AuthSource: SJ0PR13MB5545.namprd13.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 26 Sep 2022 07:00:34.4992 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: fe128f2c-073b-4c20-818e-7246a585940c X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: Zivpd2b/xJJYjPOntTMvdSyeEbZgpGdN7dVJDsGcTIqJOXCacUOpWn4P6dk5xteQ0pcGtff6IGiFmbYwo6acyt8RLRbZvkRz08Vm+eYO+Hs= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR13MB4893 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Adds the Rx and Tx function for the ctrl VNIC. The logic is mostly identical to the normal Rx and Tx functionality of the NFP PMD. Make use of the ctrl VNIC service logic to service the ctrl vNIC Rx path. Signed-off-by: Chaoyong He Signed-off-by: Heinrich Kuhn Reviewed-by: Niklas Söderlund --- doc/guides/rel_notes/release_22_11.rst | 1 + drivers/net/nfp/flower/nfp_flower.c | 49 ++++++ drivers/net/nfp/flower/nfp_flower.h | 14 ++ drivers/net/nfp/flower/nfp_flower_ctrl.c | 250 +++++++++++++++++++++++++++++++ drivers/net/nfp/flower/nfp_flower_ctrl.h | 13 ++ drivers/net/nfp/meson.build | 1 + 6 files changed, 328 insertions(+) create mode 100644 drivers/net/nfp/flower/nfp_flower_ctrl.c create mode 100644 drivers/net/nfp/flower/nfp_flower_ctrl.h diff --git a/doc/guides/rel_notes/release_22_11.rst b/doc/guides/rel_notes/release_22_11.rst index ad438b2..689f20b 100644 --- a/doc/guides/rel_notes/release_22_11.rst +++ b/doc/guides/rel_notes/release_22_11.rst @@ -61,6 +61,7 @@ New Features * Added the support of flower firmware. * Added the flower service infrastructure. + * Added the control message interactive channels between PMD and firmware. * **Updated Wangxun ngbe driver.** diff --git a/drivers/net/nfp/flower/nfp_flower.c b/drivers/net/nfp/flower/nfp_flower.c index efa8c6b..405364a 100644 --- a/drivers/net/nfp/flower/nfp_flower.c +++ b/drivers/net/nfp/flower/nfp_flower.c @@ -19,6 +19,7 @@ #include "../nfpcore/nfp_rtsym.h" #include "../nfpcore/nfp_nsp.h" #include "nfp_flower.h" +#include "nfp_flower_ctrl.h" #define CTRL_VNIC_NB_DESC 512 #define DEFAULT_FLBUF_SIZE 9216 @@ -439,6 +440,46 @@ return 0; } +static int +nfp_flower_ctrl_vnic_service(void *arg) +{ + struct nfp_app_fw_flower *app_fw_flower = arg; + + nfp_flower_ctrl_vnic_poll(app_fw_flower); + + return 0; +} + +static int +nfp_flower_enable_services(struct nfp_app_fw_flower *app_fw_flower) +{ + int ret; + uint32_t service_id; + const struct rte_service_spec flower_service = { + .name = "flower_ctrl_vnic_service", + .callback = nfp_flower_ctrl_vnic_service, + .callback_userdata = (void *)app_fw_flower, + }; + + /* Register the flower services */ + ret = rte_service_component_register(&flower_service, &service_id); + if (ret != 0) { + PMD_INIT_LOG(ERR, "Could not register %s", flower_service.name); + return -EINVAL; + } + + PMD_INIT_LOG(INFO, "%s registered", flower_service.name); + + /* Map them to available service cores*/ + ret = nfp_map_service(service_id); + if (ret != 0) { + PMD_INIT_LOG(ERR, "Could not map %s", flower_service.name); + return -EINVAL; + } + + return 0; +} + int nfp_init_app_fw_flower(struct nfp_pf_dev *pf_dev) { @@ -520,6 +561,14 @@ goto ctrl_vnic_cleanup; } + /* Start up flower services */ + ret = nfp_flower_enable_services(app_fw_flower); + if (ret != 0) { + PMD_INIT_LOG(ERR, "Could not enable flower services"); + ret = -ESRCH; + goto ctrl_vnic_cleanup; + } + return 0; ctrl_vnic_cleanup: diff --git a/drivers/net/nfp/flower/nfp_flower.h b/drivers/net/nfp/flower/nfp_flower.h index 7b138d8..4a80d52 100644 --- a/drivers/net/nfp/flower/nfp_flower.h +++ b/drivers/net/nfp/flower/nfp_flower.h @@ -6,6 +6,14 @@ #ifndef _NFP_FLOWER_H_ #define _NFP_FLOWER_H_ +/* + * Flower fallback and ctrl path always adds and removes + * 8 bytes of prepended data. Tx descriptors must point + * to the correct packet data offset after metadata has + * been added + */ +#define FLOWER_PKT_DATA_OFFSET 8 + /* The flower application's private structure */ struct nfp_app_fw_flower { /* Pointer to the PF vNIC */ @@ -16,6 +24,12 @@ struct nfp_app_fw_flower { /* Pointer to the ctrl vNIC */ struct nfp_net_hw *ctrl_hw; + + /* Ctrl vNIC Rx counter */ + uint64_t ctrl_vnic_rx_count; + + /* Ctrl vNIC Tx counter */ + uint64_t ctrl_vnic_tx_count; }; int nfp_init_app_fw_flower(struct nfp_pf_dev *pf_dev); diff --git a/drivers/net/nfp/flower/nfp_flower_ctrl.c b/drivers/net/nfp/flower/nfp_flower_ctrl.c new file mode 100644 index 0000000..df908ef --- /dev/null +++ b/drivers/net/nfp/flower/nfp_flower_ctrl.c @@ -0,0 +1,250 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (c) 2022 Corigine, Inc. + * All rights reserved. + */ + +#include +#include + +#include "../nfp_common.h" +#include "../nfp_logs.h" +#include "../nfp_ctrl.h" +#include "../nfp_rxtx.h" +#include "nfp_flower.h" +#include "nfp_flower_ctrl.h" + +#define MAX_PKT_BURST 32 + +static uint16_t +nfp_flower_ctrl_vnic_recv(void *rx_queue, + struct rte_mbuf **rx_pkts, + uint16_t nb_pkts) +{ + uint64_t dma_addr; + uint16_t avail = 0; + struct rte_mbuf *mb; + uint16_t nb_hold = 0; + struct nfp_net_hw *hw; + struct nfp_net_rxq *rxq; + struct rte_mbuf *new_mb; + struct nfp_net_rx_buff *rxb; + struct nfp_net_rx_desc *rxds; + + rxq = rx_queue; + if (unlikely(rxq == NULL)) { + /* + * DPDK just checks the queue is lower than max queues + * enabled. But the queue needs to be configured + */ + PMD_RX_LOG(ERR, "RX Bad queue"); + return 0; + } + + hw = rxq->hw; + while (avail < nb_pkts) { + rxb = &rxq->rxbufs[rxq->rd_p]; + if (unlikely(rxb == NULL)) { + PMD_RX_LOG(ERR, "rxb does not exist!"); + break; + } + + rxds = &rxq->rxds[rxq->rd_p]; + if ((rxds->rxd.meta_len_dd & PCIE_DESC_RX_DD) == 0) + break; + + /* + * Memory barrier to ensure that we won't do other + * reads before the DD bit. + */ + rte_rmb(); + + /* + * We got a packet. Let's alloc a new mbuf for refilling the + * free descriptor ring as soon as possible + */ + new_mb = rte_pktmbuf_alloc(rxq->mem_pool); + if (unlikely(new_mb == NULL)) { + PMD_RX_LOG(ERR, + "RX mbuf alloc failed port_id=%u queue_id=%u", + rxq->port_id, (unsigned int)rxq->qidx); + nfp_net_mbuf_alloc_failed(rxq); + break; + } + + nb_hold++; + + /* + * Grab the mbuf and refill the descriptor with the + * previously allocated mbuf + */ + mb = rxb->mbuf; + rxb->mbuf = new_mb; + + /* Size of this segment */ + mb->data_len = rxds->rxd.data_len - NFP_DESC_META_LEN(rxds); + /* Size of the whole packet. We just support 1 segment */ + mb->pkt_len = mb->data_len; + + if (unlikely((mb->data_len + hw->rx_offset) > rxq->mbuf_size)) { + /* + * This should not happen and the user has the + * responsibility of avoiding it. But we have + * to give some info about the error + */ + RTE_LOG_DP(ERR, PMD, + "mbuf overflow likely due to the RX offset.\n" + "\t\tYour mbuf size should have extra space for" + " RX offset=%u bytes.\n" + "\t\tCurrently you just have %u bytes available" + " but the received packet is %u bytes long", + hw->rx_offset, + rxq->mbuf_size - hw->rx_offset, + mb->data_len); + rte_pktmbuf_free(mb); + break; + } + + /* Filling the received mbuf with packet info */ + if (hw->rx_offset) + mb->data_off = RTE_PKTMBUF_HEADROOM + hw->rx_offset; + else + mb->data_off = RTE_PKTMBUF_HEADROOM + NFP_DESC_META_LEN(rxds); + + /* No scatter mode supported */ + mb->nb_segs = 1; + mb->next = NULL; + mb->port = rxq->port_id; + + rx_pkts[avail++] = mb; + + /* Now resetting and updating the descriptor */ + rxds->vals[0] = 0; + rxds->vals[1] = 0; + dma_addr = rte_cpu_to_le_64(RTE_MBUF_DMA_ADDR_DEFAULT(new_mb)); + rxds->fld.dd = 0; + rxds->fld.dma_addr_hi = (dma_addr >> 32) & 0xff; + rxds->fld.dma_addr_lo = dma_addr & 0xffffffff; + + rxq->rd_p++; + if (unlikely(rxq->rd_p == rxq->rx_count)) /* wrapping?*/ + rxq->rd_p = 0; + } + + if (nb_hold == 0) + return 0; + + nb_hold += rxq->nb_rx_hold; + + /* + * FL descriptors needs to be written before incrementing the + * FL queue WR pointer + */ + rte_wmb(); + if (nb_hold >= rxq->rx_free_thresh) { + PMD_RX_LOG(DEBUG, "port=%hu queue=%d nb_hold=%hu avail=%hu", + rxq->port_id, rxq->qidx, nb_hold, avail); + nfp_qcp_ptr_add(rxq->qcp_fl, NFP_QCP_WRITE_PTR, nb_hold); + nb_hold = 0; + } + + rxq->nb_rx_hold = nb_hold; + + return avail; +} + +uint16_t +nfp_flower_ctrl_vnic_xmit(struct nfp_app_fw_flower *app_fw_flower, + struct rte_mbuf *mbuf) +{ + uint16_t cnt = 0; + uint64_t dma_addr; + uint32_t free_descs; + struct rte_mbuf **lmbuf; + struct nfp_net_txq *txq; + struct nfp_net_hw *ctrl_hw; + struct rte_eth_dev *ctrl_dev; + struct nfp_net_nfd3_tx_desc *txds; + + ctrl_hw = app_fw_flower->ctrl_hw; + ctrl_dev = ctrl_hw->eth_dev; + + /* Flower ctrl vNIC only has a single tx queue */ + txq = ctrl_dev->data->tx_queues[0]; + if (unlikely(txq == NULL)) { + /* + * DPDK just checks the queue is lower than max queues + * enabled. But the queue needs to be configured + */ + PMD_TX_LOG(ERR, "ctrl dev TX Bad queue"); + goto xmit_end; + } + + txds = &txq->txds[txq->wr_p]; + txds->vals[0] = 0; + txds->vals[1] = 0; + txds->vals[2] = 0; + txds->vals[3] = 0; + + if (nfp_net_nfd3_txq_full(txq)) + nfp_net_tx_free_bufs(txq); + + free_descs = nfp_net_nfd3_free_tx_desc(txq); + if (unlikely(free_descs == 0)) { + PMD_TX_LOG(ERR, "ctrl dev no free descs"); + goto xmit_end; + } + + lmbuf = &txq->txbufs[txq->wr_p].mbuf; + RTE_MBUF_PREFETCH_TO_FREE(*lmbuf); + if (*lmbuf) + rte_pktmbuf_free_seg(*lmbuf); + + *lmbuf = mbuf; + dma_addr = rte_mbuf_data_iova(mbuf); + + txds->data_len = mbuf->pkt_len; + txds->dma_len = txds->data_len; + txds->dma_addr_hi = (dma_addr >> 32) & 0xff; + txds->dma_addr_lo = (dma_addr & 0xffffffff); + txds->offset_eop = FLOWER_PKT_DATA_OFFSET | PCIE_DESC_TX_EOP; + + txq->wr_p++; + if (unlikely(txq->wr_p == txq->tx_count)) /* wrapping?*/ + txq->wr_p = 0; + + cnt++; + app_fw_flower->ctrl_vnic_tx_count++; + +xmit_end: + rte_wmb(); + nfp_qcp_ptr_add(txq->qcp_q, NFP_QCP_WRITE_PTR, 1); + + return cnt; +} + +void +nfp_flower_ctrl_vnic_poll(struct nfp_app_fw_flower *app_fw_flower) +{ + uint16_t i; + uint16_t count; + struct nfp_net_rxq *rxq; + struct nfp_net_hw *ctrl_hw; + struct rte_eth_dev *ctrl_eth_dev; + struct rte_mbuf *pkts_burst[MAX_PKT_BURST]; + + ctrl_hw = app_fw_flower->ctrl_hw; + ctrl_eth_dev = ctrl_hw->eth_dev; + + /* ctrl vNIC only has a single Rx queue */ + rxq = ctrl_eth_dev->data->rx_queues[0]; + + while (true) { + count = nfp_flower_ctrl_vnic_recv(rxq, pkts_burst, MAX_PKT_BURST); + if (count != 0) { + app_fw_flower->ctrl_vnic_rx_count += count; + /* Process cmsgs here, only free for now */ + for (i = 0; i < count; i++) + rte_pktmbuf_free(pkts_burst[i]); + } + } +} diff --git a/drivers/net/nfp/flower/nfp_flower_ctrl.h b/drivers/net/nfp/flower/nfp_flower_ctrl.h new file mode 100644 index 0000000..1e38578 --- /dev/null +++ b/drivers/net/nfp/flower/nfp_flower_ctrl.h @@ -0,0 +1,13 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (c) 2022 Corigine, Inc. + * All rights reserved. + */ + +#ifndef _NFP_FLOWER_CTRL_H_ +#define _NFP_FLOWER_CTRL_H_ + +void nfp_flower_ctrl_vnic_poll(struct nfp_app_fw_flower *app_fw_flower); +uint16_t nfp_flower_ctrl_vnic_xmit(struct nfp_app_fw_flower *app_fw_flower, + struct rte_mbuf *mbuf); + +#endif /* _NFP_FLOWER_CTRL_H_ */ diff --git a/drivers/net/nfp/meson.build b/drivers/net/nfp/meson.build index 7ae3115..8710213 100644 --- a/drivers/net/nfp/meson.build +++ b/drivers/net/nfp/meson.build @@ -7,6 +7,7 @@ if not is_linux or not dpdk_conf.get('RTE_ARCH_64') endif sources = files( 'flower/nfp_flower.c', + 'flower/nfp_flower_ctrl.c', 'nfpcore/nfp_cpp_pcie_ops.c', 'nfpcore/nfp_nsp.c', 'nfpcore/nfp_cppcore.c', From patchwork Mon Sep 26 06:59:53 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Chaoyong He X-Patchwork-Id: 116850 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 4287FA00C2; Mon, 26 Sep 2022 09:01:38 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 1E94D42B86; Mon, 26 Sep 2022 09:00:41 +0200 (CEST) Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2111.outbound.protection.outlook.com [40.107.244.111]) by mails.dpdk.org (Postfix) with ESMTP id 5800F42B7B for ; Mon, 26 Sep 2022 09:00:39 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=AwQVWh2zTKhw/4VZoVd92Th2D1Hc/EOPyWZsWgx/vUACJwjcrq/oM0T1Ivs49Nyi94bmz56ydlWeISB7F1FgGdKpiMpKTYhzCJuIK2w4IOUyfTiI87FCsA5aefo0RxXkUusb9JUOtuiFrGHKtYgFYMZF8D0NeBNkunGSXlJ74WvLHJqJy4dNSFrsPPCL9z35zDsIs1hb5Ts61Q9HFMGXK9M/NGQlD1ifBIrEyUxVvvsTLyfiQh8tAQufr9mXZLtK3PHoMBOC64O/zRuIrCfplEXLVuUy5q91yZmTV7P+ZJtSnA4sGlHoDvShT8Uj62ly8KbHzgHgP5qwZN/gq6tXoA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=65X8IrBfz1AHRvf2srOsOb5uiue+t7C5u5br9t2gkiw=; b=FWu6b96aeSBNeUKQUz9zdxuiEbKQy4KUvbdoWOY/X1qTvsy7d9hMDFsEDpdDqmZjQ57SRdHjBjM+n4ozSgMd3Z2V+BpDPpSuCH75DY0Y7Hh6+sn5lLZYHHLFthf+bhPUD3zoPCnIK8wWD9BVDI3WIRWQhnyW3EmFDXepVJNqujZ0mWBQqFavRGBkuZ2gIKk2DKfBsA5IheoHfMq4s5CmOnWRrxdGPTKZBH4t/yOGcAlGQKsQTb5bSvb+0xpYcMk5Mp89D8jzdT/PWx9qYztNXesBJqgQnVDiqTz84wq15Tv3l0uYRkAiVXcdi69VV2quDnfuNfBaVGSYhrYl4wX2ew== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=corigine.com; dmarc=pass action=none header.from=corigine.com; dkim=pass header.d=corigine.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=corigine.onmicrosoft.com; s=selector2-corigine-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=65X8IrBfz1AHRvf2srOsOb5uiue+t7C5u5br9t2gkiw=; b=qmaMUop/cpTSs9RZK/jIPjculGNicjYvzwEcpyyQmKQj+afnYEpNaZo2ANnYtByTLWSUSjJEWehthRlCYh2mBwHThM5wgSVLaqtKq8ir9tbAyofws9ClDDnUxt2JGz0pmxF8QxbWz9ub48uT5wPlJAGJoWzrlcq0IFWg4cYNFOg= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=corigine.com; Received: from SJ0PR13MB5545.namprd13.prod.outlook.com (2603:10b6:a03:424::5) by SA1PR13MB4893.namprd13.prod.outlook.com (2603:10b6:806:18a::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5676.14; Mon, 26 Sep 2022 07:00:36 +0000 Received: from SJ0PR13MB5545.namprd13.prod.outlook.com ([fe80::779:2722:a8e5:503b]) by SJ0PR13MB5545.namprd13.prod.outlook.com ([fe80::779:2722:a8e5:503b%7]) with mapi id 15.20.5676.014; Mon, 26 Sep 2022 07:00:36 +0000 From: Chaoyong He To: dev@dpdk.org Cc: oss-drivers@corigine.com, niklas.soderlund@corigine.com, Chaoyong He Subject: [PATCH v10 09/13] net/nfp: add flower representor framework Date: Mon, 26 Sep 2022 14:59:53 +0800 Message-Id: <1664175597-37248-10-git-send-email-chaoyong.he@corigine.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1664175597-37248-1-git-send-email-chaoyong.he@corigine.com> References: <1664175597-37248-1-git-send-email-chaoyong.he@corigine.com> X-ClientProxiedBy: SG2P153CA0005.APCP153.PROD.OUTLOOK.COM (2603:1096::15) To SJ0PR13MB5545.namprd13.prod.outlook.com (2603:10b6:a03:424::5) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SJ0PR13MB5545:EE_|SA1PR13MB4893:EE_ X-MS-Office365-Filtering-Correlation-Id: 1f628796-07da-422b-2e9a-08da9f8cd02c X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: TuP4s2VyHqAQqi0Ut4g0MmwBKXd+W2SERkB+rFlZO4a5BQSu3JPWuM+HuWFNacT3xYku+aTb8ECMle3l7DgfFGAZu4UdKHGuL5hkkLhd2wTpI1vzxzlp/9K0wgxgrOqqqCbPzEYqFoYUorhp09N6P5lhIhFB1D7jgw59mlYMtjXJS89tHSm+pewxfxw5YC+aL1pMWKTdQBdfZV6J3EzCJN5mlFMLRkuiGneb5TF9Cku3vX3Sa355LiwAmK6pP5CdASfn+h+7OredDuI6h6xpJmsTEt/wyp+nnU6IlRPpyyH+pe4/lgo9m7zjOrBIYH2aBceTZ0YsfN+Dgr9Vn5EC9Em1uFXjD4ZPXV/1sKMf1qFb+SuqmGxQtAfXJ0TD+Z6iqK4fqOO1JAS8O4/J1sWmqEVdgWOcsjrdruO7Rz+YpWsWB5Cq7Oe1zygY8avnhCdI5fnZ90Hl2XnxYF3Gy2I8tCzKz8/2LLPK2JQSXQ02nPF2DrQFOfLbNWUv9l6FAX8qrJrp+/xK8NxS83VB0K3maDcLvlyVKct0eTxlgr+0lsh42XetH5nuf9dKzc0HR7uhpSjn05VBhQmgsgaFb/6Vx1DvF61rX9RdiYuqRLFsvTri2RILENEBHd7vil1xmvU5q38L2wjBQKBwu74yJyEsldIpcm8PquS1GxzHJ/eCQ7WUuhKFKmz4sMgRiAMMNwc5RVkcN01COjCoyW7xAeGOS04CdzUOVL4MfV0KGma8VDJGMMfQiQ5vNquxLSDuND5NB9gHuQrD38yK6YF6xjjBbQ== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SJ0PR13MB5545.namprd13.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230022)(4636009)(366004)(396003)(39830400003)(136003)(376002)(346002)(451199015)(36756003)(38100700002)(38350700002)(2906002)(86362001)(44832011)(30864003)(66946007)(8936002)(66556008)(66476007)(5660300002)(8676002)(4326008)(83380400001)(186003)(66574015)(6512007)(6916009)(316002)(52116002)(6666004)(2616005)(26005)(6506007)(478600001)(41300700001)(107886003)(6486002)(579004); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?gKn2hWUOm134owiUk6m98WFbPhVe?= =?utf-8?q?lGABmTvlv9111G2tflH8rHUX2h3RbKw0U04C93OabhUap3zCMfSvgZz6gYHKkD7Lf?= =?utf-8?q?vPYEroTZyjZC6wiMc+wrbQ2hXD8Sd/PVh2WSdxpJ9wXxdXcbHZNwcZwAVYZcQ4ZBm?= =?utf-8?q?OTzYGOUUOadTyXkqDUpvEJmHohpJo3OnJWw6ifD4ftAr8TMUXiovXynloyeXkohKJ?= =?utf-8?q?VR7urwEet/bqg3Anm0i2LmD+Oo3TlbtaPMKZ9cn458Gtno9P/hCoRWqvm/Vhf6SxF?= =?utf-8?q?hYlagqEy+gtGTDlaRrsX8b8xqsnxLvKCom9PnE+THcj81HwjuYr+9wWOYIQF0kbEX?= =?utf-8?q?RGs+kgQD23G/0LaIe3v/HOPsT9b1o7XiFkN0KyANNMZZvcbZ3o/11Sj/H+ljzXJoU?= =?utf-8?q?mufsEjdMb6ntiCvsW0w5MBX3YKea2qZ8jWGcD5LRoMc5wLkB3MYii2ytqw7AO8VnX?= =?utf-8?q?MBIZNAssPUnjje8FWrRJAy7JAoDUI8WLJ6VqDu41vVXh7gpiEc5K+qf70vxKs4TvO?= =?utf-8?q?3Xdx8O5vsLwTNEKHMeZkIJPg+MLvHoPn7qlxQ0OPuIP7ukHsa9gJ5p0G1ucxqXK9v?= =?utf-8?q?P4Uqw3O5iF91XNwzLbKgWTGLKKYwj7VM6AA0fLIdB9j0S8p0gIVhVZv24gUN4+An7?= =?utf-8?q?13IS5NY9NDKn6V6tnU8UUJonIItk1B8T3/Ro+1GD3Zty6WcZmEW05+cYCWfgSU+Ro?= =?utf-8?q?JhgUANiX71awtKnSaYrWpbtPN3i8eOE/9XvvsJzvbfZPaSFYzkqZYZqyOp/Awfx3F?= =?utf-8?q?94fuQLGUMp+KZclYky6WebB8kTJhO7xCFoA2qooRYKWdTz2eEiR3nnHy97IsjIvwc?= =?utf-8?q?YTArl14YTXPcIXZGSJMWyIhAyOyPQwuzXU8kF86wmbYyQGr9rLvZ89yu46DP3uV2e?= =?utf-8?q?IZSHTXFgl1mTPS77KQZFxEgpib+UhEk34D9Q+wo0rKvz0MpBsc4Fa8V8PzunJJz8n?= =?utf-8?q?T6tdZ60zzdaMV2Nv92vDJ8koFIwsfaou8+sM340H2tUm8LtgRTR31iXFmjzNlaUjD?= =?utf-8?q?C7LW+CJu3pJIUpb5SDukOIm+h1n5cBpU4RRLnrwu88BWON2JAMpq0f7V3aeW9azKC?= =?utf-8?q?hUHB1DxO+m7HMiD4T7IU0Ajb9u2++oap/PkE1ixR6wRUjYK//LatWefo8z15bzBcv?= =?utf-8?q?Oa1QNL7F7AOWhWwwLFl6Y8+b1Yt7R2ZfGLKv7ccb6NFUnGSR3+/xY0Kk8WoCkBZ6y?= =?utf-8?q?UBEw2m9zjWHs4oY4hHNCcYD/rsqDF1KS/XXducNxYlB5WJGCrDMqVml+NPd2ypc00?= =?utf-8?q?u2w9FLEADjZKPUPiMzycVdAXy12vlYVNZJuf0ClkSvr9Oq7P36Hdy1Zm4VFYSM4rO?= =?utf-8?q?4weB5+kr25CCKzs8MHDr5lpXJS4JsUhefPPwkOsyH0p3eRQcb8/QvrkfBGhbq1JbB?= =?utf-8?q?ZLo+lPW+tE+TnWEt6yThtek/D6O/7k91R3g3xP6g0ksz4UFZ+6G5DjjBf6qzTm2lX?= =?utf-8?q?h1y7jYgvq53VnxGgH8ibZVB+KYGJefpvE2peP9wc7xcMfwpdX5iB0T5FYkB1MQm80?= =?utf-8?q?EfZ6Vmqojo3w5p40wH2HAUaZRc2Xqukk5A=3D=3D?= X-OriginatorOrg: corigine.com X-MS-Exchange-CrossTenant-Network-Message-Id: 1f628796-07da-422b-2e9a-08da9f8cd02c X-MS-Exchange-CrossTenant-AuthSource: SJ0PR13MB5545.namprd13.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 26 Sep 2022 07:00:36.7358 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: fe128f2c-073b-4c20-818e-7246a585940c X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 3YzYEBBiwZ+itEeSatWh1/zQi/FNz14RWqzRwVDMyVljJiYnMmbXhDbrgdt8XOPku8Dhsa9z72BJ9LRzcm5cVjL2qO7u3JEXhQJT2QAyl10= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR13MB4893 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Adds the framework to support flower representors. The number of VF representors are parsed from the command line. For physical port representors the current logic aims to create a representor for each physical port present on the hardware. An eth_dev is created for each physical port and VF, and flower firmware requires a MAC repr cmsg to be transmitted to firmware with info about the number of physical ports configured. Reify messages are sent to hardware for each physical port representor. An rte_ring is also created per representor so that traffic can be pushed and pulled to this interface. To up and down the real device represented by a flower representor port a port mod message is used to convey that info to the firmware. This message will be used in the dev_ops callbacks of flower representors. Each cmsg generated by the driver is prepended with a cmsg header. This commit also adds the logic to fill in the header of cmsgs. Also add the Rx and Tx path for flower representors. For Rx packets are dequeued from the representor ring and passed to the eth_dev. For Tx the first queue of the PF vNIC is used. Metadata about the representor is added before the packet is sent down to firmware. Signed-off-by: Chaoyong He Reviewed-by: Niklas Söderlund --- doc/guides/nics/nfp.rst | 6 + doc/guides/rel_notes/release_22_11.rst | 1 + drivers/net/nfp/flower/nfp_flower.c | 7 + drivers/net/nfp/flower/nfp_flower.h | 21 + drivers/net/nfp/flower/nfp_flower_cmsg.c | 179 +++++ drivers/net/nfp/flower/nfp_flower_cmsg.h | 173 +++++ drivers/net/nfp/flower/nfp_flower_representor.c | 833 ++++++++++++++++++++++++ drivers/net/nfp/flower/nfp_flower_representor.h | 39 ++ drivers/net/nfp/meson.build | 2 + 9 files changed, 1261 insertions(+) create mode 100644 drivers/net/nfp/flower/nfp_flower_cmsg.c create mode 100644 drivers/net/nfp/flower/nfp_flower_cmsg.h create mode 100644 drivers/net/nfp/flower/nfp_flower_representor.c create mode 100644 drivers/net/nfp/flower/nfp_flower_representor.h diff --git a/doc/guides/nics/nfp.rst b/doc/guides/nics/nfp.rst index 4faab39..c62b1fb 100644 --- a/doc/guides/nics/nfp.rst +++ b/doc/guides/nics/nfp.rst @@ -194,3 +194,9 @@ The flower firmware application requires the PMD running two services: * PF vNIC service: handling the feedback traffic. * ctrl vNIC service: communicate between PMD and firmware through control message. + +To achieve the offload of flow, the representor ports are exposed to OVS. +The flower firmware application support representor port for VF and physical +port. There will always exist a representor port for each physical port, +and the number of the representor port for VF is specified by the user through +parameter. diff --git a/doc/guides/rel_notes/release_22_11.rst b/doc/guides/rel_notes/release_22_11.rst index 689f20b..302144e 100644 --- a/doc/guides/rel_notes/release_22_11.rst +++ b/doc/guides/rel_notes/release_22_11.rst @@ -62,6 +62,7 @@ New Features * Added the support of flower firmware. * Added the flower service infrastructure. * Added the control message interactive channels between PMD and firmware. + * Added the support of representor port. * **Updated Wangxun ngbe driver.** diff --git a/drivers/net/nfp/flower/nfp_flower.c b/drivers/net/nfp/flower/nfp_flower.c index 405364a..c79b4a5 100644 --- a/drivers/net/nfp/flower/nfp_flower.c +++ b/drivers/net/nfp/flower/nfp_flower.c @@ -20,6 +20,7 @@ #include "../nfpcore/nfp_nsp.h" #include "nfp_flower.h" #include "nfp_flower_ctrl.h" +#include "nfp_flower_representor.h" #define CTRL_VNIC_NB_DESC 512 #define DEFAULT_FLBUF_SIZE 9216 @@ -569,6 +570,12 @@ goto ctrl_vnic_cleanup; } + ret = nfp_flower_repr_create(app_fw_flower); + if (ret != 0) { + PMD_INIT_LOG(ERR, "Could not create representor ports"); + goto ctrl_vnic_cleanup; + } + return 0; ctrl_vnic_cleanup: diff --git a/drivers/net/nfp/flower/nfp_flower.h b/drivers/net/nfp/flower/nfp_flower.h index 4a80d52..e1cd0a3 100644 --- a/drivers/net/nfp/flower/nfp_flower.h +++ b/drivers/net/nfp/flower/nfp_flower.h @@ -14,8 +14,20 @@ */ #define FLOWER_PKT_DATA_OFFSET 8 +#define MAX_FLOWER_PHYPORTS 8 +#define MAX_FLOWER_VFS 64 + /* The flower application's private structure */ struct nfp_app_fw_flower { + /* switch domain for this app */ + uint16_t switch_domain_id; + + /* Number of VF representors */ + uint8_t num_vf_reprs; + + /* Number of phyport representors */ + uint8_t num_phyport_reprs; + /* Pointer to the PF vNIC */ struct nfp_net_hw *pf_hw; @@ -30,6 +42,15 @@ struct nfp_app_fw_flower { /* Ctrl vNIC Tx counter */ uint64_t ctrl_vnic_tx_count; + + /* Array of phyport representors */ + struct nfp_flower_representor *phy_reprs[MAX_FLOWER_PHYPORTS]; + + /* Array of VF representors */ + struct nfp_flower_representor *vf_reprs[MAX_FLOWER_VFS]; + + /* PF representor */ + struct nfp_flower_representor *pf_repr; }; int nfp_init_app_fw_flower(struct nfp_pf_dev *pf_dev); diff --git a/drivers/net/nfp/flower/nfp_flower_cmsg.c b/drivers/net/nfp/flower/nfp_flower_cmsg.c new file mode 100644 index 0000000..750a629 --- /dev/null +++ b/drivers/net/nfp/flower/nfp_flower_cmsg.c @@ -0,0 +1,179 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2022 Corigine, Inc. + * All rights reserved. + */ + +#include "../nfpcore/nfp_nsp.h" +#include "../nfp_logs.h" +#include "../nfp_common.h" +#include "nfp_flower.h" +#include "nfp_flower_cmsg.h" +#include "nfp_flower_ctrl.h" +#include "nfp_flower_representor.h" + +static void * +nfp_flower_cmsg_init(struct rte_mbuf *m, + enum nfp_flower_cmsg_type type, + uint32_t size) +{ + char *pkt; + uint32_t data; + uint32_t new_size = size; + struct nfp_flower_cmsg_hdr *hdr; + + pkt = rte_pktmbuf_mtod(m, char *); + PMD_DRV_LOG(DEBUG, "flower_cmsg_init using pkt at %p", pkt); + + data = rte_cpu_to_be_32(NFP_NET_META_PORTID); + rte_memcpy(pkt, &data, 4); + pkt += 4; + new_size += 4; + + /* First the metadata as flower requires it */ + data = rte_cpu_to_be_32(NFP_META_PORT_ID_CTRL); + rte_memcpy(pkt, &data, 4); + pkt += 4; + new_size += 4; + + /* Now the ctrl header */ + hdr = (struct nfp_flower_cmsg_hdr *)pkt; + hdr->pad = 0; + hdr->type = type; + hdr->version = NFP_FLOWER_CMSG_VER1; + + pkt = (char *)hdr + NFP_FLOWER_CMSG_HLEN; + new_size += NFP_FLOWER_CMSG_HLEN; + + m->pkt_len = new_size; + m->data_len = m->pkt_len; + + return pkt; +} + +static void +nfp_flower_cmsg_mac_repr_init(struct rte_mbuf *mbuf, int num_ports) +{ + uint32_t size; + struct nfp_flower_cmsg_mac_repr *msg; + enum nfp_flower_cmsg_type type = NFP_FLOWER_CMSG_TYPE_MAC_REPR; + + size = sizeof(*msg) + (num_ports * sizeof(msg->ports[0])); + msg = nfp_flower_cmsg_init(mbuf, type, size); + memset(msg->reserved, 0, sizeof(msg->reserved)); + msg->num_ports = num_ports; +} + +static void +nfp_flower_cmsg_mac_repr_fill(struct rte_mbuf *m, + unsigned int idx, + unsigned int nbi, + unsigned int nbi_port, + unsigned int phys_port) +{ + struct nfp_flower_cmsg_mac_repr *msg; + + msg = (struct nfp_flower_cmsg_mac_repr *)nfp_flower_cmsg_get_data(m); + msg->ports[idx].idx = idx; + msg->ports[idx].info = nbi & NFP_FLOWER_CMSG_MAC_REPR_NBI; + msg->ports[idx].nbi_port = nbi_port; + msg->ports[idx].phys_port = phys_port; +} + +int +nfp_flower_cmsg_mac_repr(struct nfp_app_fw_flower *app_fw_flower) +{ + int i; + uint16_t cnt; + unsigned int nbi; + unsigned int nbi_port; + unsigned int phys_port; + struct rte_mbuf *mbuf; + struct nfp_eth_table *nfp_eth_table; + + mbuf = rte_pktmbuf_alloc(app_fw_flower->ctrl_pktmbuf_pool); + if (mbuf == NULL) { + PMD_DRV_LOG(ERR, "Could not allocate mac repr cmsg"); + return -ENOMEM; + } + + nfp_flower_cmsg_mac_repr_init(mbuf, app_fw_flower->num_phyport_reprs); + + /* Fill in the mac repr cmsg */ + nfp_eth_table = app_fw_flower->pf_hw->pf_dev->nfp_eth_table; + for (i = 0; i < app_fw_flower->num_phyport_reprs; i++) { + nbi = nfp_eth_table->ports[i].nbi; + nbi_port = nfp_eth_table->ports[i].base; + phys_port = nfp_eth_table->ports[i].index; + + nfp_flower_cmsg_mac_repr_fill(mbuf, i, nbi, nbi_port, phys_port); + } + + /* Send the cmsg via the ctrl vNIC */ + cnt = nfp_flower_ctrl_vnic_xmit(app_fw_flower, mbuf); + if (cnt == 0) { + PMD_DRV_LOG(ERR, "Send cmsg through ctrl vnic failed."); + rte_pktmbuf_free(mbuf); + return -EIO; + } + + return 0; +} + +int +nfp_flower_cmsg_repr_reify(struct nfp_app_fw_flower *app_fw_flower, + struct nfp_flower_representor *repr) +{ + uint16_t cnt; + struct rte_mbuf *mbuf; + struct nfp_flower_cmsg_port_reify *msg; + + mbuf = rte_pktmbuf_alloc(app_fw_flower->ctrl_pktmbuf_pool); + if (mbuf == NULL) { + PMD_DRV_LOG(DEBUG, "alloc mbuf for repr reify failed"); + return -ENOMEM; + } + + msg = nfp_flower_cmsg_init(mbuf, NFP_FLOWER_CMSG_TYPE_PORT_REIFY, sizeof(*msg)); + msg->portnum = rte_cpu_to_be_32(repr->port_id); + msg->reserved = 0; + msg->info = rte_cpu_to_be_16(1); + + cnt = nfp_flower_ctrl_vnic_xmit(app_fw_flower, mbuf); + if (cnt == 0) { + PMD_DRV_LOG(ERR, "Send cmsg through ctrl vnic failed."); + rte_pktmbuf_free(mbuf); + return -EIO; + } + + return 0; +} + +int +nfp_flower_cmsg_port_mod(struct nfp_app_fw_flower *app_fw_flower, + uint32_t port_id, bool carrier_ok) +{ + uint16_t cnt; + struct rte_mbuf *mbuf; + struct nfp_flower_cmsg_port_mod *msg; + + mbuf = rte_pktmbuf_alloc(app_fw_flower->ctrl_pktmbuf_pool); + if (mbuf == NULL) { + PMD_DRV_LOG(DEBUG, "alloc mbuf for repr portmod failed"); + return -ENOMEM; + } + + msg = nfp_flower_cmsg_init(mbuf, NFP_FLOWER_CMSG_TYPE_PORT_MOD, sizeof(*msg)); + msg->portnum = rte_cpu_to_be_32(port_id); + msg->reserved = 0; + msg->info = carrier_ok; + msg->mtu = 9000; + + cnt = nfp_flower_ctrl_vnic_xmit(app_fw_flower, mbuf); + if (cnt == 0) { + PMD_DRV_LOG(ERR, "Send cmsg through ctrl vnic failed."); + rte_pktmbuf_free(mbuf); + return -EIO; + } + + return 0; +} diff --git a/drivers/net/nfp/flower/nfp_flower_cmsg.h b/drivers/net/nfp/flower/nfp_flower_cmsg.h new file mode 100644 index 0000000..0bf8fc8 --- /dev/null +++ b/drivers/net/nfp/flower/nfp_flower_cmsg.h @@ -0,0 +1,173 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2022 Corigine, Inc. + * All rights reserved. + */ + +#ifndef _NFP_CMSG_H_ +#define _NFP_CMSG_H_ + +#include +#include + +struct nfp_flower_cmsg_hdr { + rte_be16_t pad; + uint8_t type; + uint8_t version; +}; + +/* Types defined for control messages */ +enum nfp_flower_cmsg_type { + NFP_FLOWER_CMSG_TYPE_FLOW_ADD = 0, + NFP_FLOWER_CMSG_TYPE_FLOW_MOD = 1, + NFP_FLOWER_CMSG_TYPE_FLOW_DEL = 2, + NFP_FLOWER_CMSG_TYPE_LAG_CONFIG = 4, + NFP_FLOWER_CMSG_TYPE_PORT_REIFY = 6, + NFP_FLOWER_CMSG_TYPE_MAC_REPR = 7, + NFP_FLOWER_CMSG_TYPE_PORT_MOD = 8, + NFP_FLOWER_CMSG_TYPE_MERGE_HINT = 9, + NFP_FLOWER_CMSG_TYPE_NO_NEIGH = 10, + NFP_FLOWER_CMSG_TYPE_TUN_MAC = 11, + NFP_FLOWER_CMSG_TYPE_ACTIVE_TUNS = 12, + NFP_FLOWER_CMSG_TYPE_TUN_NEIGH = 13, + NFP_FLOWER_CMSG_TYPE_TUN_IPS = 14, + NFP_FLOWER_CMSG_TYPE_FLOW_STATS = 15, + NFP_FLOWER_CMSG_TYPE_PORT_ECHO = 16, + NFP_FLOWER_CMSG_TYPE_QOS_MOD = 18, + NFP_FLOWER_CMSG_TYPE_QOS_DEL = 19, + NFP_FLOWER_CMSG_TYPE_QOS_STATS = 20, + NFP_FLOWER_CMSG_TYPE_PRE_TUN_RULE = 21, + NFP_FLOWER_CMSG_TYPE_TUN_IPS_V6 = 22, + NFP_FLOWER_CMSG_TYPE_NO_NEIGH_V6 = 23, + NFP_FLOWER_CMSG_TYPE_TUN_NEIGH_V6 = 24, + NFP_FLOWER_CMSG_TYPE_ACTIVE_TUNS_V6 = 25, + NFP_FLOWER_CMSG_TYPE_MAX = 32, +}; + +/* + * NFP_FLOWER_CMSG_TYPE_MAC_REPR + * Bit 3 3 2 2 2 2 2 2 2 2 2 2 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 + * -----\ 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0 + * Word +---------------+-----------+---+---------------+---------------+ + * 0 | spare |Number of ports| + * +---------------+-----------+---+---------------+---------------+ + * 1 | Index | spare |NBI| Port on NBI | Chip-wide port| + * +---------------+-----------+---+---------------+---------------+ + * .... + * +---------------+-----------+---+---------------+---------------+ + * N-1 | Index | spare |NBI| Port on NBI | Chip-wide port| + * +---------------+-----------+---+---------------+---------------+ + * N | Index | spare |NBI| Port on NBI | Chip-wide port| + * +---------------+-----------+---+---------------+---------------+ + * + * Index: index into the eth table + * NBI (bits 17-16): NBI number (0-3) + * Port on NBI (bits 15-8): “base” in the driver + * this forms NBIX.PortY notation as the NSP eth table. + * "Chip-wide" port (bits 7-0): + */ +struct nfp_flower_cmsg_mac_repr { + uint8_t reserved[3]; + uint8_t num_ports; + struct { + uint8_t idx; + uint8_t info; + uint8_t nbi_port; + uint8_t phys_port; + } ports[0]; +}; + +/* + * NFP_FLOWER_CMSG_TYPE_PORT_REIFY + * Bit 3 3 2 2 2 2 2 2 2 2 2 2 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 + * -----\ 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0 + * Word +-------+-------+---+---+-------+---+---+-----------+-----------+ + * 0 |Port Ty|Sys ID |NIC|Rsv| Spare |PCI|typ| vNIC | queue | + * +-------+-----+-+---+---+-------+---+---+-----------+---------+-+ + * 1 | Spare |E| + * +-------------------------------------------------------------+-+ + * E: 1 = Representor exists, 0 = Representor does not exist + */ +struct nfp_flower_cmsg_port_reify { + rte_be32_t portnum; + rte_be16_t reserved; + rte_be16_t info; +}; + +/* + * NFP_FLOWER_CMSG_TYPE_PORT_MOD + * Bit 3 3 2 2 2 2 2 2 2 2 2 2 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 + * -----\ 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0 + * Word +-------+-------+---+---+-------+---+---+-------+---+-----------+ + * 0 |Port Ty|Sys ID |NIC|Rsv| Reserved | Port | + * +-------+-------+---+---+-----+-+---+---+-------+---+-----------+ + * 1 | Spare |L| MTU | + * +-----------------------------+-+-------------------------------+ + * L: Link or Admin state bit. When message is generated by host, this + * bit indicates the admin state (0=down, 1=up). When generated by + * NFP, it indicates the link state (0=down, 1=up) + * + * Port Type (word 1, bits 31 to 28) = 1 (Physical Network) + * Port: “Chip-wide number” as assigned by BSP + * + * Bit 3 3 2 2 2 2 2 2 2 2 2 2 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 + * -----\ 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0 + * Word +-------+-------+---+---+-------+---+---+-------+---+-----------+ + * 0 |Port Ty|Sys ID |NIC|Rsv| Spare |PCI|typ| vNIC | queue | + * +-------+-----+-+---+---+---+-+-+---+---+-------+---+-----------+ + * 1 | Spare |L| MTU | + * +-----------------------------+-+-------------------------------+ + * L: Link or Admin state bit. When message is generated by host, this + * bit indicates the admin state (0=down, 1=up). When generated by + * NFP, it indicates the link state (0=down, 1=up) + * + * Port Type (word 1, bits 31 to 28) = 2 (PCIE) + */ +struct nfp_flower_cmsg_port_mod { + rte_be32_t portnum; + uint8_t reserved; + uint8_t info; + rte_be16_t mtu; +}; + +enum nfp_flower_cmsg_port_type { + NFP_FLOWER_CMSG_PORT_TYPE_UNSPEC, + NFP_FLOWER_CMSG_PORT_TYPE_PHYS_PORT, + NFP_FLOWER_CMSG_PORT_TYPE_PCIE_PORT, + NFP_FLOWER_CMSG_PORT_TYPE_OTHER_PORT, +}; + +enum nfp_flower_cmsg_port_vnic_type { + NFP_FLOWER_CMSG_PORT_VNIC_TYPE_VF, + NFP_FLOWER_CMSG_PORT_VNIC_TYPE_PF, + NFP_FLOWER_CMSG_PORT_VNIC_TYPE_CTRL, +}; + +#define NFP_FLOWER_CMSG_MAC_REPR_NBI (0x3) + +#define NFP_FLOWER_CMSG_HLEN sizeof(struct nfp_flower_cmsg_hdr) +#define NFP_FLOWER_CMSG_VER1 1 +#define NFP_NET_META_PORTID 5 +#define NFP_META_PORT_ID_CTRL ~0U + +#define NFP_FLOWER_CMSG_PORT_TYPE(x) (((x) >> 28) & 0xf) /* [31,28] */ +#define NFP_FLOWER_CMSG_PORT_SYS_ID(x) (((x) >> 24) & 0xf) /* [24,27] */ +#define NFP_FLOWER_CMSG_PORT_NFP_ID(x) (((x) >> 22) & 0x3) /* [22,23] */ +#define NFP_FLOWER_CMSG_PORT_PCI(x) (((x) >> 14) & 0x3) /* [14,15] */ +#define NFP_FLOWER_CMSG_PORT_VNIC_TYPE(x) (((x) >> 12) & 0x3) /* [12,13] */ +#define NFP_FLOWER_CMSG_PORT_VNIC(x) (((x) >> 6) & 0x3f) /* [6,11] */ +#define NFP_FLOWER_CMSG_PORT_PCIE_Q(x) ((x) & 0x3f) /* [0,5] */ +#define NFP_FLOWER_CMSG_PORT_PHYS_PORT_NUM(x) ((x) & 0xff) /* [0,7] */ + +static inline char* +nfp_flower_cmsg_get_data(struct rte_mbuf *m) +{ + return rte_pktmbuf_mtod(m, char *) + 4 + 4 + NFP_FLOWER_CMSG_HLEN; +} + +int nfp_flower_cmsg_mac_repr(struct nfp_app_fw_flower *app_fw_flower); +int nfp_flower_cmsg_repr_reify(struct nfp_app_fw_flower *app_fw_flower, + struct nfp_flower_representor *repr); +int nfp_flower_cmsg_port_mod(struct nfp_app_fw_flower *app_fw_flower, + uint32_t port_id, bool carrier_ok); + +#endif /* _NFP_CMSG_H_ */ diff --git a/drivers/net/nfp/flower/nfp_flower_representor.c b/drivers/net/nfp/flower/nfp_flower_representor.c new file mode 100644 index 0000000..19b1fed --- /dev/null +++ b/drivers/net/nfp/flower/nfp_flower_representor.c @@ -0,0 +1,833 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (c) 2022 Corigine, Inc. + * All rights reserved. + */ + +#include +#include + +#include "../nfp_common.h" +#include "../nfp_logs.h" +#include "../nfp_ctrl.h" +#include "../nfp_rxtx.h" +#include "../nfpcore/nfp_mip.h" +#include "../nfpcore/nfp_rtsym.h" +#include "../nfpcore/nfp_nsp.h" +#include "nfp_flower.h" +#include "nfp_flower_representor.h" +#include "nfp_flower_ctrl.h" +#include "nfp_flower_cmsg.h" + +static int +nfp_pf_repr_rx_queue_setup(struct rte_eth_dev *dev, + uint16_t queue_idx, + uint16_t nb_desc, + unsigned int socket_id, + const struct rte_eth_rxconf *rx_conf, + struct rte_mempool *mp) +{ + struct nfp_net_hw *hw; + struct nfp_net_rxq *rxq; + const struct rte_memzone *tz; + struct nfp_flower_representor *repr; + + repr = (struct nfp_flower_representor *)dev->data->dev_private; + hw = repr->app_fw_flower->pf_hw; + + /* Allocating rx queue data structure */ + rxq = rte_zmalloc_socket("ethdev RX queue", sizeof(struct nfp_net_rxq), + RTE_CACHE_LINE_SIZE, socket_id); + if (rxq == NULL) + return -ENOMEM; + + dev->data->rx_queues[queue_idx] = rxq; + + /* Hw queues mapping based on firmware configuration */ + rxq->qidx = queue_idx; + rxq->fl_qcidx = queue_idx * hw->stride_rx; + rxq->rx_qcidx = rxq->fl_qcidx + (hw->stride_rx - 1); + rxq->qcp_fl = hw->rx_bar + NFP_QCP_QUEUE_OFF(rxq->fl_qcidx); + rxq->qcp_rx = hw->rx_bar + NFP_QCP_QUEUE_OFF(rxq->rx_qcidx); + + /* + * Tracking mbuf size for detecting a potential mbuf overflow due to + * RX offset + */ + rxq->mem_pool = mp; + rxq->mbuf_size = rxq->mem_pool->elt_size; + rxq->mbuf_size -= (sizeof(struct rte_mbuf) + RTE_PKTMBUF_HEADROOM); + hw->flbufsz = rxq->mbuf_size; + + rxq->rx_count = nb_desc; + rxq->port_id = dev->data->port_id; + rxq->rx_free_thresh = rx_conf->rx_free_thresh; + rxq->drop_en = rx_conf->rx_drop_en; + + /* + * Allocate RX ring hardware descriptors. A memzone large enough to + * handle the maximum ring size is allocated in order to allow for + * resizing in later calls to the queue setup function. + */ + tz = rte_eth_dma_zone_reserve(dev, "rx_ring", queue_idx, + sizeof(struct nfp_net_rx_desc) * NFP_NET_MAX_RX_DESC, + NFP_MEMZONE_ALIGN, socket_id); + if (tz == NULL) { + PMD_DRV_LOG(ERR, "Error allocating rx dma"); + nfp_net_rx_queue_release(dev, queue_idx); + dev->data->rx_queues[queue_idx] = NULL; + return -ENOMEM; + } + + /* Saving physical and virtual addresses for the RX ring */ + rxq->dma = (uint64_t)tz->iova; + rxq->rxds = (struct nfp_net_rx_desc *)tz->addr; + + /* mbuf pointers array for referencing mbufs linked to RX descriptors */ + rxq->rxbufs = rte_zmalloc_socket("rxq->rxbufs", + sizeof(*rxq->rxbufs) * nb_desc, + RTE_CACHE_LINE_SIZE, socket_id); + if (rxq->rxbufs == NULL) { + nfp_net_rx_queue_release(dev, queue_idx); + dev->data->rx_queues[queue_idx] = NULL; + return -ENOMEM; + } + + nfp_net_reset_rx_queue(rxq); + rxq->hw = hw; + + /* + * Telling the HW about the physical address of the RX ring and number + * of descriptors in log2 format + */ + nn_cfg_writeq(hw, NFP_NET_CFG_RXR_ADDR(queue_idx), rxq->dma); + nn_cfg_writeb(hw, NFP_NET_CFG_RXR_SZ(queue_idx), rte_log2_u32(nb_desc)); + + return 0; +} + +static int +nfp_pf_repr_tx_queue_setup(struct rte_eth_dev *dev, + uint16_t queue_idx, + uint16_t nb_desc, + unsigned int socket_id, + const struct rte_eth_txconf *tx_conf) +{ + struct nfp_net_hw *hw; + struct nfp_net_txq *txq; + uint16_t tx_free_thresh; + const struct rte_memzone *tz; + struct nfp_flower_representor *repr; + + repr = (struct nfp_flower_representor *)dev->data->dev_private; + hw = repr->app_fw_flower->pf_hw; + + tx_free_thresh = (tx_conf->tx_free_thresh) ? tx_conf->tx_free_thresh : + DEFAULT_TX_FREE_THRESH; + if (tx_free_thresh > nb_desc) + return -EINVAL; + + /* Allocating tx queue data structure */ + txq = rte_zmalloc_socket("ethdev TX queue", sizeof(struct nfp_net_txq), + RTE_CACHE_LINE_SIZE, socket_id); + if (txq == NULL) { + PMD_DRV_LOG(ERR, "Error allocating tx dma"); + return -ENOMEM; + } + + dev->data->tx_queues[queue_idx] = txq; + + /* + * Allocate TX ring hardware descriptors. A memzone large enough to + * handle the maximum ring size is allocated in order to allow for + * resizing in later calls to the queue setup function. + */ + tz = rte_eth_dma_zone_reserve(dev, "tx_ring", queue_idx, + sizeof(struct nfp_net_nfd3_tx_desc) * NFP_NET_MAX_TX_DESC, + NFP_MEMZONE_ALIGN, socket_id); + if (tz == NULL) { + PMD_DRV_LOG(ERR, "Error allocating tx dma"); + nfp_net_tx_queue_release(dev, queue_idx); + dev->data->tx_queues[queue_idx] = NULL; + return -ENOMEM; + } + + txq->tx_count = nb_desc; + txq->tx_free_thresh = tx_free_thresh; + txq->tx_pthresh = tx_conf->tx_thresh.pthresh; + txq->tx_hthresh = tx_conf->tx_thresh.hthresh; + txq->tx_wthresh = tx_conf->tx_thresh.wthresh; + + /* queue mapping based on firmware configuration */ + txq->qidx = queue_idx; + txq->tx_qcidx = queue_idx * hw->stride_tx; + txq->qcp_q = hw->tx_bar + NFP_QCP_QUEUE_OFF(txq->tx_qcidx); + + txq->port_id = dev->data->port_id; + + /* Saving physical and virtual addresses for the TX ring */ + txq->dma = (uint64_t)tz->iova; + txq->txds = (struct nfp_net_nfd3_tx_desc *)tz->addr; + + /* mbuf pointers array for referencing mbufs linked to TX descriptors */ + txq->txbufs = rte_zmalloc_socket("txq->txbufs", + sizeof(*txq->txbufs) * nb_desc, + RTE_CACHE_LINE_SIZE, socket_id); + if (txq->txbufs == NULL) { + nfp_net_tx_queue_release(dev, queue_idx); + dev->data->tx_queues[queue_idx] = NULL; + return -ENOMEM; + } + + nfp_net_reset_tx_queue(txq); + txq->hw = hw; + + /* + * Telling the HW about the physical address of the TX ring and number + * of descriptors in log2 format + */ + nn_cfg_writeq(hw, NFP_NET_CFG_TXR_ADDR(queue_idx), txq->dma); + nn_cfg_writeb(hw, NFP_NET_CFG_TXR_SZ(queue_idx), rte_log2_u32(nb_desc)); + + return 0; +} + +static int +nfp_flower_repr_link_update(struct rte_eth_dev *dev, + __rte_unused int wait_to_complete) +{ + int ret; + uint32_t nn_link_status; + struct nfp_net_hw *pf_hw; + struct rte_eth_link *link; + struct nfp_flower_representor *repr; + + static const uint32_t ls_to_ethtool[] = { + [NFP_NET_CFG_STS_LINK_RATE_UNSUPPORTED] = RTE_ETH_SPEED_NUM_NONE, + [NFP_NET_CFG_STS_LINK_RATE_UNKNOWN] = RTE_ETH_SPEED_NUM_NONE, + [NFP_NET_CFG_STS_LINK_RATE_1G] = RTE_ETH_SPEED_NUM_1G, + [NFP_NET_CFG_STS_LINK_RATE_10G] = RTE_ETH_SPEED_NUM_10G, + [NFP_NET_CFG_STS_LINK_RATE_25G] = RTE_ETH_SPEED_NUM_25G, + [NFP_NET_CFG_STS_LINK_RATE_40G] = RTE_ETH_SPEED_NUM_40G, + [NFP_NET_CFG_STS_LINK_RATE_50G] = RTE_ETH_SPEED_NUM_50G, + [NFP_NET_CFG_STS_LINK_RATE_100G] = RTE_ETH_SPEED_NUM_100G, + }; + + repr = (struct nfp_flower_representor *)dev->data->dev_private; + link = &repr->link; + pf_hw = repr->app_fw_flower->pf_hw; + + memset(link, 0, sizeof(struct rte_eth_link)); + nn_link_status = nn_cfg_readl(pf_hw, NFP_NET_CFG_STS); + + if (nn_link_status & NFP_NET_CFG_STS_LINK) + link->link_status = RTE_ETH_LINK_UP; + + link->link_duplex = RTE_ETH_LINK_FULL_DUPLEX; + + nn_link_status = (nn_link_status >> NFP_NET_CFG_STS_LINK_RATE_SHIFT) & + NFP_NET_CFG_STS_LINK_RATE_MASK; + + if (nn_link_status >= RTE_DIM(ls_to_ethtool)) + link->link_speed = RTE_ETH_SPEED_NUM_NONE; + else + link->link_speed = ls_to_ethtool[nn_link_status]; + + ret = rte_eth_linkstatus_set(dev, link); + if (ret == 0) { + if (link->link_status) + PMD_DRV_LOG(INFO, "NIC Link is Up"); + else + PMD_DRV_LOG(INFO, "NIC Link is Down"); + } + + return ret; +} + +static int +nfp_flower_repr_dev_infos_get(__rte_unused struct rte_eth_dev *dev, + struct rte_eth_dev_info *dev_info) +{ + /* Hardcoded pktlen and queues for now */ + dev_info->max_rx_queues = 1; + dev_info->max_tx_queues = 1; + dev_info->min_rx_bufsize = RTE_ETHER_MIN_MTU; + dev_info->max_rx_pktlen = 9000; + + dev_info->rx_offload_capa = RTE_ETH_RX_OFFLOAD_VLAN_STRIP; + dev_info->rx_offload_capa |= RTE_ETH_RX_OFFLOAD_IPV4_CKSUM | + RTE_ETH_RX_OFFLOAD_UDP_CKSUM | + RTE_ETH_RX_OFFLOAD_TCP_CKSUM; + + dev_info->tx_offload_capa = RTE_ETH_TX_OFFLOAD_VLAN_INSERT; + dev_info->tx_offload_capa |= RTE_ETH_TX_OFFLOAD_IPV4_CKSUM | + RTE_ETH_TX_OFFLOAD_UDP_CKSUM | + RTE_ETH_TX_OFFLOAD_TCP_CKSUM; + dev_info->tx_offload_capa |= RTE_ETH_TX_OFFLOAD_TCP_TSO; + dev_info->tx_offload_capa |= RTE_ETH_TX_OFFLOAD_MULTI_SEGS; + + dev_info->max_mac_addrs = 1; + + return 0; +} + +static int +nfp_flower_repr_dev_configure(struct rte_eth_dev *dev) +{ + struct nfp_net_hw *pf_hw; + struct rte_eth_conf *dev_conf; + struct rte_eth_rxmode *rxmode; + struct nfp_flower_representor *repr; + + repr = (struct nfp_flower_representor *)dev->data->dev_private; + pf_hw = repr->app_fw_flower->pf_hw; + + dev_conf = &dev->data->dev_conf; + rxmode = &dev_conf->rxmode; + + /* Checking MTU set */ + if (rxmode->mtu > pf_hw->flbufsz) { + PMD_DRV_LOG(INFO, "MTU (%u) larger then current mbufsize (%u) not supported", + rxmode->mtu, pf_hw->flbufsz); + return -ERANGE; + } + + return 0; +} + +static int +nfp_flower_repr_dev_start(struct rte_eth_dev *dev) +{ + struct nfp_flower_representor *repr; + struct nfp_app_fw_flower *app_fw_flower; + + repr = (struct nfp_flower_representor *)dev->data->dev_private; + app_fw_flower = repr->app_fw_flower; + + if (repr->repr_type == NFP_REPR_TYPE_PHYS_PORT) { + nfp_eth_set_configured(app_fw_flower->pf_hw->pf_dev->cpp, + repr->nfp_idx, 1); + } + + nfp_flower_cmsg_port_mod(app_fw_flower, repr->port_id, true); + + return 0; +} + +static int +nfp_flower_repr_dev_stop(struct rte_eth_dev *dev) +{ + struct nfp_flower_representor *repr; + struct nfp_app_fw_flower *app_fw_flower; + + repr = (struct nfp_flower_representor *)dev->data->dev_private; + app_fw_flower = repr->app_fw_flower; + + nfp_flower_cmsg_port_mod(app_fw_flower, repr->port_id, false); + + if (repr->repr_type == NFP_REPR_TYPE_PHYS_PORT) { + nfp_eth_set_configured(app_fw_flower->pf_hw->pf_dev->cpp, + repr->nfp_idx, 0); + } + + return 0; +} + +static int +nfp_flower_repr_rx_queue_setup(struct rte_eth_dev *dev, + uint16_t rx_queue_id, + __rte_unused uint16_t nb_rx_desc, + unsigned int socket_id, + __rte_unused const struct rte_eth_rxconf *rx_conf, + __rte_unused struct rte_mempool *mb_pool) +{ + struct nfp_net_rxq *rxq; + struct nfp_net_hw *pf_hw; + struct nfp_flower_representor *repr; + + repr = (struct nfp_flower_representor *)dev->data->dev_private; + pf_hw = repr->app_fw_flower->pf_hw; + + /* Allocating rx queue data structure */ + rxq = rte_zmalloc_socket("ethdev RX queue", sizeof(struct nfp_net_rxq), + RTE_CACHE_LINE_SIZE, socket_id); + if (rxq == NULL) + return -ENOMEM; + + rxq->hw = pf_hw; + rxq->qidx = rx_queue_id; + rxq->port_id = dev->data->port_id; + dev->data->rx_queues[rx_queue_id] = rxq; + + return 0; +} + +static int +nfp_flower_repr_tx_queue_setup(struct rte_eth_dev *dev, + uint16_t tx_queue_id, + __rte_unused uint16_t nb_tx_desc, + unsigned int socket_id, + __rte_unused const struct rte_eth_txconf *tx_conf) +{ + struct nfp_net_txq *txq; + struct nfp_net_hw *pf_hw; + struct nfp_flower_representor *repr; + + repr = (struct nfp_flower_representor *)dev->data->dev_private; + pf_hw = repr->app_fw_flower->pf_hw; + + /* Allocating tx queue data structure */ + txq = rte_zmalloc_socket("ethdev TX queue", sizeof(struct nfp_net_txq), + RTE_CACHE_LINE_SIZE, socket_id); + if (txq == NULL) + return -ENOMEM; + + txq->hw = pf_hw; + txq->qidx = tx_queue_id; + txq->port_id = dev->data->port_id; + dev->data->tx_queues[tx_queue_id] = txq; + + return 0; +} + +static int +nfp_flower_repr_stats_get(struct rte_eth_dev *ethdev, + struct rte_eth_stats *stats) +{ + struct nfp_flower_representor *repr; + + repr = (struct nfp_flower_representor *)ethdev->data->dev_private; + rte_memcpy(stats, &repr->repr_stats, sizeof(struct rte_eth_stats)); + + return 0; +} + +static int +nfp_flower_repr_stats_reset(struct rte_eth_dev *ethdev) +{ + struct nfp_flower_representor *repr; + + repr = (struct nfp_flower_representor *)ethdev->data->dev_private; + memset(&repr->repr_stats, 0, sizeof(struct rte_eth_stats)); + + return 0; +} + +static int +nfp_flower_repr_promiscuous_enable(struct rte_eth_dev *dev) +{ + struct nfp_net_hw *pf_hw; + struct nfp_flower_representor *repr; + + repr = (struct nfp_flower_representor *)dev->data->dev_private; + pf_hw = repr->app_fw_flower->pf_hw; + + if (!(pf_hw->cap & NFP_NET_CFG_CTRL_PROMISC)) { + PMD_DRV_LOG(INFO, "Promiscuous mode not supported"); + return -ENOTSUP; + } + + if (pf_hw->ctrl & NFP_NET_CFG_CTRL_PROMISC) { + PMD_DRV_LOG(INFO, "Promiscuous mode already enabled"); + return 0; + } + + return nfp_net_promisc_enable(pf_hw->eth_dev); +} + +static int +nfp_flower_repr_promiscuous_disable(struct rte_eth_dev *dev) +{ + struct nfp_net_hw *pf_hw; + struct nfp_flower_representor *repr; + + repr = (struct nfp_flower_representor *)dev->data->dev_private; + pf_hw = repr->app_fw_flower->pf_hw; + + if ((pf_hw->ctrl & NFP_NET_CFG_CTRL_PROMISC) == 0) { + PMD_DRV_LOG(INFO, "Promiscuous mode already disabled"); + return 0; + } + + return nfp_net_promisc_disable(pf_hw->eth_dev); +} + +static int +nfp_flower_repr_mac_addr_set(struct rte_eth_dev *ethdev, + struct rte_ether_addr *mac_addr) +{ + struct nfp_flower_representor *repr; + + repr = (struct nfp_flower_representor *)ethdev->data->dev_private; + rte_ether_addr_copy(mac_addr, &repr->mac_addr); + rte_ether_addr_copy(mac_addr, ethdev->data->mac_addrs); + + return 0; +} + +static const struct eth_dev_ops nfp_flower_pf_repr_dev_ops = { + .dev_infos_get = nfp_flower_repr_dev_infos_get, + + .dev_configure = nfp_flower_repr_dev_configure, + + .rx_queue_setup = nfp_pf_repr_rx_queue_setup, + .tx_queue_setup = nfp_pf_repr_tx_queue_setup, + + .link_update = nfp_flower_repr_link_update, + + .stats_get = nfp_flower_repr_stats_get, + .stats_reset = nfp_flower_repr_stats_reset, + + .promiscuous_enable = nfp_flower_repr_promiscuous_enable, + .promiscuous_disable = nfp_flower_repr_promiscuous_disable, + + .mac_addr_set = nfp_flower_repr_mac_addr_set, +}; + +static const struct eth_dev_ops nfp_flower_repr_dev_ops = { + .dev_infos_get = nfp_flower_repr_dev_infos_get, + + .dev_start = nfp_flower_repr_dev_start, + .dev_configure = nfp_flower_repr_dev_configure, + .dev_stop = nfp_flower_repr_dev_stop, + + .rx_queue_setup = nfp_flower_repr_rx_queue_setup, + .tx_queue_setup = nfp_flower_repr_tx_queue_setup, + + .link_update = nfp_flower_repr_link_update, + + .stats_get = nfp_flower_repr_stats_get, + .stats_reset = nfp_flower_repr_stats_reset, + + .promiscuous_enable = nfp_flower_repr_promiscuous_enable, + .promiscuous_disable = nfp_flower_repr_promiscuous_disable, + + .mac_addr_set = nfp_flower_repr_mac_addr_set, +}; + +static uint32_t +nfp_flower_get_phys_port_id(uint8_t port) +{ + return (NFP_FLOWER_CMSG_PORT_TYPE_PHYS_PORT << 28) | port; +} + +static uint32_t +nfp_get_pcie_port_id(struct nfp_cpp *cpp, + int type, + uint8_t vnic, + uint8_t queue) +{ + uint8_t nfp_pcie; + uint32_t port_id; + + nfp_pcie = NFP_CPP_INTERFACE_UNIT_of(nfp_cpp_interface(cpp)); + port_id = ((nfp_pcie & 0x3) << 14) | + ((type & 0x3) << 12) | + ((vnic & 0x3f) << 6) | + (queue & 0x3f) | + ((NFP_FLOWER_CMSG_PORT_TYPE_PCIE_PORT & 0xf) << 28); + + return port_id; +} + +static int +nfp_flower_pf_repr_init(struct rte_eth_dev *eth_dev, + void *init_params) +{ + struct nfp_flower_representor *repr; + struct nfp_flower_representor *init_repr_data; + + /* Cast the input representor data to the correct struct here */ + init_repr_data = (struct nfp_flower_representor *)init_params; + + /* Memory has been allocated in the eth_dev_create() function */ + repr = eth_dev->data->dev_private; + + /* Copy data here from the input representor template*/ + repr->vf_id = init_repr_data->vf_id; + repr->switch_domain_id = init_repr_data->switch_domain_id; + repr->repr_type = init_repr_data->repr_type; + repr->app_fw_flower = init_repr_data->app_fw_flower; + + snprintf(repr->name, sizeof(repr->name), "%s", init_repr_data->name); + + eth_dev->dev_ops = &nfp_flower_pf_repr_dev_ops; + eth_dev->data->dev_flags |= RTE_ETH_DEV_REPRESENTOR; + + eth_dev->data->representor_id = 0; + + /* This backer port is that of the eth_device created for the PF vNIC */ + eth_dev->data->backer_port_id = 0; + + /* Only single queues for representor devices */ + eth_dev->data->nb_rx_queues = 1; + eth_dev->data->nb_tx_queues = 1; + + /* Allocating memory for mac addr */ + eth_dev->data->mac_addrs = rte_zmalloc("mac_addr", RTE_ETHER_ADDR_LEN, 0); + if (eth_dev->data->mac_addrs == NULL) { + PMD_INIT_LOG(ERR, "Failed to allocate memory for repr MAC"); + return -ENOMEM; + } + + rte_ether_addr_copy(&init_repr_data->mac_addr, &repr->mac_addr); + rte_ether_addr_copy(&init_repr_data->mac_addr, eth_dev->data->mac_addrs); + + repr->app_fw_flower->pf_repr = repr; + repr->app_fw_flower->pf_hw->eth_dev = eth_dev; + + return 0; +} + +static int +nfp_flower_repr_init(struct rte_eth_dev *eth_dev, + void *init_params) +{ + int ret; + unsigned int numa_node; + char ring_name[RTE_ETH_NAME_MAX_LEN]; + struct nfp_app_fw_flower *app_fw_flower; + struct nfp_flower_representor *repr; + struct nfp_flower_representor *init_repr_data; + + /* Cast the input representor data to the correct struct here */ + init_repr_data = (struct nfp_flower_representor *)init_params; + app_fw_flower = init_repr_data->app_fw_flower; + + /* Memory has been allocated in the eth_dev_create() function */ + repr = eth_dev->data->dev_private; + + /* + * We need multiproduce rings as we can have multiple PF ports. + * On the other hand, we need single consumer rings, as just one + * representor PMD will try to read from the ring. + */ + snprintf(ring_name, sizeof(ring_name), "%s_%s", init_repr_data->name, "ring"); + numa_node = rte_socket_id(); + repr->ring = rte_ring_create(ring_name, 256, numa_node, RING_F_SC_DEQ); + if (repr->ring == NULL) { + PMD_DRV_LOG(ERR, "rte_ring_create failed for %s", ring_name); + return -ENOMEM; + } + + /* Copy data here from the input representor template*/ + repr->vf_id = init_repr_data->vf_id; + repr->switch_domain_id = init_repr_data->switch_domain_id; + repr->port_id = init_repr_data->port_id; + repr->nfp_idx = init_repr_data->nfp_idx; + repr->repr_type = init_repr_data->repr_type; + repr->app_fw_flower = init_repr_data->app_fw_flower; + + snprintf(repr->name, sizeof(repr->name), "%s", init_repr_data->name); + + eth_dev->dev_ops = &nfp_flower_repr_dev_ops; + eth_dev->data->dev_flags |= RTE_ETH_DEV_REPRESENTOR; + + if (repr->repr_type == NFP_REPR_TYPE_PHYS_PORT) + eth_dev->data->representor_id = repr->vf_id; + else + eth_dev->data->representor_id = repr->vf_id + + app_fw_flower->num_phyport_reprs + 1; + + /* This backer port is that of the eth_device created for the PF vNIC */ + eth_dev->data->backer_port_id = 0; + + /* Only single queues for representor devices */ + eth_dev->data->nb_rx_queues = 1; + eth_dev->data->nb_tx_queues = 1; + + /* Allocating memory for mac addr */ + eth_dev->data->mac_addrs = rte_zmalloc("mac_addr", RTE_ETHER_ADDR_LEN, 0); + if (eth_dev->data->mac_addrs == NULL) { + PMD_INIT_LOG(ERR, "Failed to allocate memory for repr MAC"); + ret = -ENOMEM; + goto ring_cleanup; + } + + rte_ether_addr_copy(&init_repr_data->mac_addr, &repr->mac_addr); + rte_ether_addr_copy(&init_repr_data->mac_addr, eth_dev->data->mac_addrs); + + /* Send reify message to hardware to inform it about the new repr */ + ret = nfp_flower_cmsg_repr_reify(app_fw_flower, repr); + if (ret != 0) { + PMD_INIT_LOG(WARNING, "Failed to send repr reify message"); + goto mac_cleanup; + } + + /* Add repr to correct array */ + if (repr->repr_type == NFP_REPR_TYPE_PHYS_PORT) + app_fw_flower->phy_reprs[repr->nfp_idx] = repr; + else + app_fw_flower->vf_reprs[repr->vf_id] = repr; + + return 0; + +mac_cleanup: + rte_free(eth_dev->data->mac_addrs); +ring_cleanup: + rte_ring_free(repr->ring); + + return ret; +} + +static int +nfp_flower_repr_alloc(struct nfp_app_fw_flower *app_fw_flower) +{ + int i; + int ret; + struct rte_eth_dev *eth_dev; + struct nfp_eth_table *nfp_eth_table; + struct nfp_eth_table_port *eth_port; + struct nfp_flower_representor flower_repr = { + .switch_domain_id = app_fw_flower->switch_domain_id, + .app_fw_flower = app_fw_flower, + }; + + nfp_eth_table = app_fw_flower->pf_hw->pf_dev->nfp_eth_table; + eth_dev = app_fw_flower->ctrl_hw->eth_dev; + + /* Send a NFP_FLOWER_CMSG_TYPE_MAC_REPR cmsg to hardware*/ + ret = nfp_flower_cmsg_mac_repr(app_fw_flower); + if (ret != 0) { + PMD_INIT_LOG(ERR, "Cloud not send mac repr cmsgs"); + return ret; + } + + /* Create a rte_eth_dev for PF vNIC representor */ + flower_repr.repr_type = NFP_REPR_TYPE_PF; + + /* PF vNIC reprs get a random MAC address */ + rte_eth_random_addr(flower_repr.mac_addr.addr_bytes); + sprintf(flower_repr.name, "flower_repr_pf"); + + /* Create a eth_dev for this representor */ + ret = rte_eth_dev_create(eth_dev->device, flower_repr.name, + sizeof(struct nfp_flower_representor), + NULL, NULL, nfp_flower_pf_repr_init, &flower_repr); + if (ret != 0) { + PMD_INIT_LOG(ERR, "Failed to init the pf repr"); + return -EINVAL; + } + + /* Create a rte_eth_dev for every phyport representor */ + for (i = 0; i < app_fw_flower->num_phyport_reprs; i++) { + eth_port = &nfp_eth_table->ports[i]; + flower_repr.repr_type = NFP_REPR_TYPE_PHYS_PORT; + flower_repr.port_id = nfp_flower_get_phys_port_id(eth_port->index); + flower_repr.nfp_idx = eth_port->eth_index; + flower_repr.vf_id = i + 1; + + /* Copy the real mac of the interface to the representor struct */ + rte_ether_addr_copy((struct rte_ether_addr *)eth_port->mac_addr, + &flower_repr.mac_addr); + sprintf(flower_repr.name, "flower_repr_p%d", i); + + /* + * Create a eth_dev for this representor + * This will also allocate private memory for the device + */ + ret = rte_eth_dev_create(eth_dev->device, flower_repr.name, + sizeof(struct nfp_flower_representor), + NULL, NULL, nfp_flower_repr_init, &flower_repr); + if (ret != 0) { + PMD_INIT_LOG(ERR, "Cloud not create eth_dev for repr"); + break; + } + } + + if (i < app_fw_flower->num_phyport_reprs) + return ret; + + /* + * Now allocate eth_dev's for VF representors. + * Also send reify messages + */ + for (i = 0; i < app_fw_flower->num_vf_reprs; i++) { + flower_repr.repr_type = NFP_REPR_TYPE_VF; + flower_repr.port_id = nfp_get_pcie_port_id(app_fw_flower->pf_hw->cpp, + NFP_FLOWER_CMSG_PORT_VNIC_TYPE_VF, i, 0); + flower_repr.nfp_idx = 0; + flower_repr.vf_id = i; + + /* VF reprs get a random MAC address */ + rte_eth_random_addr(flower_repr.mac_addr.addr_bytes); + sprintf(flower_repr.name, "flower_repr_vf%d", i); + + /* This will also allocate private memory for the device*/ + ret = rte_eth_dev_create(eth_dev->device, flower_repr.name, + sizeof(struct nfp_flower_representor), + NULL, NULL, nfp_flower_repr_init, &flower_repr); + if (ret != 0) { + PMD_INIT_LOG(ERR, "Cloud not create eth_dev for repr"); + break; + } + } + + if (i < app_fw_flower->num_vf_reprs) + return ret; + + return 0; +} + +int +nfp_flower_repr_create(struct nfp_app_fw_flower *app_fw_flower) +{ + int ret; + struct nfp_pf_dev *pf_dev; + struct rte_pci_device *pci_dev; + struct nfp_eth_table *nfp_eth_table; + struct rte_eth_devargs eth_da = { + .nb_representor_ports = 0 + }; + + pf_dev = app_fw_flower->pf_hw->pf_dev; + pci_dev = pf_dev->pci_dev; + + /* Allocate a switch domain for the flower app */ + if (app_fw_flower->switch_domain_id == RTE_ETH_DEV_SWITCH_DOMAIN_ID_INVALID && + rte_eth_switch_domain_alloc(&app_fw_flower->switch_domain_id)) { + PMD_INIT_LOG(WARNING, "failed to allocate switch domain for device"); + } + + /* Now parse PCI device args passed for representor info */ + if (pci_dev->device.devargs != NULL) { + ret = rte_eth_devargs_parse(pci_dev->device.devargs->args, ð_da); + if (ret != 0) { + PMD_INIT_LOG(ERR, "devarg parse failed"); + return -EINVAL; + } + } + + if (eth_da.nb_representor_ports == 0) { + PMD_INIT_LOG(DEBUG, "No representor port need to create."); + return 0; + } + + /* There always exist phy repr */ + nfp_eth_table = pf_dev->nfp_eth_table; + if (eth_da.nb_representor_ports < nfp_eth_table->count + 1) { + PMD_INIT_LOG(ERR, "Should also create repr port for phy port and PF vNIC."); + return -ERANGE; + } + + /* Only support VF representor creation via the command line */ + if (eth_da.type != RTE_ETH_REPRESENTOR_VF) { + PMD_INIT_LOG(ERR, "Unsupported representor type: %d", eth_da.type); + return -ENOTSUP; + } + + /* Fill in flower app with repr counts */ + app_fw_flower->num_phyport_reprs = (uint8_t)nfp_eth_table->count; + app_fw_flower->num_vf_reprs = eth_da.nb_representor_ports - + nfp_eth_table->count - 1; + + PMD_INIT_LOG(INFO, "%d number of VF reprs", app_fw_flower->num_vf_reprs); + PMD_INIT_LOG(INFO, "%d number of phyport reprs", app_fw_flower->num_phyport_reprs); + + ret = nfp_flower_repr_alloc(app_fw_flower); + if (ret != 0) { + PMD_INIT_LOG(ERR, "representors allocation failed"); + return -EINVAL; + } + + return 0; +} diff --git a/drivers/net/nfp/flower/nfp_flower_representor.h b/drivers/net/nfp/flower/nfp_flower_representor.h new file mode 100644 index 0000000..af44ef3 --- /dev/null +++ b/drivers/net/nfp/flower/nfp_flower_representor.h @@ -0,0 +1,39 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2022 Corigine, Inc. + * All rights reserved. + */ + +#ifndef _NFP_FLOWER_REPRESENTOR_H_ +#define _NFP_FLOWER_REPRESENTOR_H_ + +/* + * enum nfp_repr_type - type of representor + * @NFP_REPR_TYPE_PHYS_PORT: external NIC port + * @NFP_REPR_TYPE_PF: physical function + * @NFP_REPR_TYPE_VF: virtual function + * @NFP_REPR_TYPE_MAX: number of representor types + */ +enum nfp_repr_type { + NFP_REPR_TYPE_PHYS_PORT = 0, + NFP_REPR_TYPE_PF, + NFP_REPR_TYPE_VF, + NFP_REPR_TYPE_MAX, +}; + +struct nfp_flower_representor { + uint16_t vf_id; + uint16_t switch_domain_id; + uint32_t repr_type; + uint32_t port_id; + uint32_t nfp_idx; /* only valid for the repr of physical port */ + char name[RTE_ETH_NAME_MAX_LEN]; + struct rte_ether_addr mac_addr; + struct nfp_app_fw_flower *app_fw_flower; + struct rte_ring *ring; + struct rte_eth_link link; + struct rte_eth_stats repr_stats; +}; + +int nfp_flower_repr_create(struct nfp_app_fw_flower *app_fw_flower); + +#endif /* _NFP_FLOWER_REPRESENTOR_H_ */ diff --git a/drivers/net/nfp/meson.build b/drivers/net/nfp/meson.build index 8710213..8a63979 100644 --- a/drivers/net/nfp/meson.build +++ b/drivers/net/nfp/meson.build @@ -7,7 +7,9 @@ if not is_linux or not dpdk_conf.get('RTE_ARCH_64') endif sources = files( 'flower/nfp_flower.c', + 'flower/nfp_flower_cmsg.c', 'flower/nfp_flower_ctrl.c', + 'flower/nfp_flower_representor.c', 'nfpcore/nfp_cpp_pcie_ops.c', 'nfpcore/nfp_nsp.c', 'nfpcore/nfp_cppcore.c', From patchwork Mon Sep 26 06:59:54 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Chaoyong He X-Patchwork-Id: 116851 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id DEB60A00C2; Mon, 26 Sep 2022 09:01:48 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 8BEE342B92; Mon, 26 Sep 2022 09:00:44 +0200 (CEST) Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2121.outbound.protection.outlook.com [40.107.92.121]) by mails.dpdk.org (Postfix) with ESMTP id 8BD4D42B89 for ; Mon, 26 Sep 2022 09:00:42 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=UE7Nw7IHZjtSz2U0PjaACfYk0smVujKtxhJGvz+duTHKDig0baK5X/DfldXyULFcbhJnwon7b/6Ler6AGFwvvx5EbvC1dhSJTFDo65w+j5j/mlwV7PHDg8bn+ZkTAR+71xltm0dBkx5Wsz3rHzPdBfWo9CK2LP5lNd5DhJohVO5+xw8O1vHcP3BBFXDZykkb6cSCERpLU2yk3ITYiJL8i2XGVzCbnYK3Sd+v6jgfJE9CTaOzcYtXUIcElnu6aVLE6GMtS44bgDXf7pRPYk/AQsrw6fucdj9jPOiiLT39Eww54n1K2eLcXCFn6V8CAe6uZ8587fSIIPhAdlb5BcQ1UQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=EWr1gfdOF1wDrXrxxsCT0+nDuKRLZExakHi4mm3hiss=; b=ZpbP+IKK7wFdFU9GiV1+DRsHnG3T2Z8Tg6KhiJYf/khY+GmY0F2bxifrAtgGPV2mUUMFRmQzsQaE5ve1ieY5yimn2bvUrdtm4PM9d71WTFaFzJTJ65bNYPY3oqQo9e0YrVLxgcw/rUv7cCT9RQtq8p6Drrxn/gglyoRqDdhPA8ubhsYkhPRIG1QkuZ8dNUi4d5II+xcYv7Bu24oqc+K7viOz18u20AtQVW+LM/x018n3yyDPnU2RlvTY+o8eBK2avWvgSqXWtXcLwnmefRyE7Y7Oh6+czbOwDhmMwVTcia/k6QDjUwsK2nfH/KKMN46fuFWuIeHUrpP/SPdZ8tgYbQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=corigine.com; dmarc=pass action=none header.from=corigine.com; dkim=pass header.d=corigine.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=corigine.onmicrosoft.com; s=selector2-corigine-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=EWr1gfdOF1wDrXrxxsCT0+nDuKRLZExakHi4mm3hiss=; b=U96onJ02L9OE2h2IpciGXG7lD9bDsqhDqjNqPTBthsEtDEfWyFCKYhEH4GqNO3hZMs/aWbWlqujL7J2eqjJk8ghTag9Ft7jfeM2YeEtVAMyJkRWT8XiEAf2Yrqe8YP1TpBBhCq5hRlnH+kgdfoN3S5i3Rvvhd7l0VAbqnySgO4I= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=corigine.com; Received: from SJ0PR13MB5545.namprd13.prod.outlook.com (2603:10b6:a03:424::5) by SA1PR13MB4941.namprd13.prod.outlook.com (2603:10b6:806:1a9::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5676.14; Mon, 26 Sep 2022 07:00:39 +0000 Received: from SJ0PR13MB5545.namprd13.prod.outlook.com ([fe80::779:2722:a8e5:503b]) by SJ0PR13MB5545.namprd13.prod.outlook.com ([fe80::779:2722:a8e5:503b%7]) with mapi id 15.20.5676.014; Mon, 26 Sep 2022 07:00:39 +0000 From: Chaoyong He To: dev@dpdk.org Cc: oss-drivers@corigine.com, niklas.soderlund@corigine.com, Chaoyong He Subject: [PATCH v10 10/13] net/nfp: add flower PF related routines Date: Mon, 26 Sep 2022 14:59:54 +0800 Message-Id: <1664175597-37248-11-git-send-email-chaoyong.he@corigine.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1664175597-37248-1-git-send-email-chaoyong.he@corigine.com> References: <1664175597-37248-1-git-send-email-chaoyong.he@corigine.com> X-ClientProxiedBy: SG2P153CA0005.APCP153.PROD.OUTLOOK.COM (2603:1096::15) To SJ0PR13MB5545.namprd13.prod.outlook.com (2603:10b6:a03:424::5) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SJ0PR13MB5545:EE_|SA1PR13MB4941:EE_ X-MS-Office365-Filtering-Correlation-Id: 930a11e5-734c-416a-1bf2-08da9f8cd1a0 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: syvssU6CfrUhVWSCcozwcffgZ3kV93U0u9Fwn410t0Kfy4TeyK15B/JieTgWzbNDTSTcnCTOS216KyzB7XMhBVQL5WJQIHoSXlZTXuVTJmHUiHoH8joj6+23ZwXC1Ige7DsR/uSuJrizAXSAD9tHqGxmMhqPcxyNhvLsx42xuVko9GZGeOHY016NvDf//S1QfLphBYmVtVhTD9pu+x1LF516HmkiCpBF2lh48fDtLQLbN09vw7r5m3/issPZxu8G7lwnbCIzFhhyVcG08zd/4edoZhsay5Qs5QoIpna+rM5tV1gmHlP/PLEQs8Jn3ATd3BPBViWNhU8bvUBw9C22Qg6e7lL4DZI1/kVTW/8f+yVq1aj9JPCI51nhVW+NrAh+GCX2DaJNHSccJPkPB1JeKC1C4Y+rBiBI6askZd13ehNiCpTbDtiTjzP8lSSRhYshSK5AAwCXX/n0biFh0WYgq8XVDH1TUrfmI4Tj98BZ4nKiU8/mHAhHhbvW3I5aoVVCuRAHjEpWZHKoR68BunkGMxkOxifInUq4nWs0KBgdkOxWGhjyXU/tILZFHwebnKvkTOa6kFdUru7XNn1O+7glUNZvNitJEig3oQbAHmuoq76gK+A5NPVhOQtLmjZyNomn9tamqbaBfCr/2mQFSfOk0YlPPvwZpjU5re1zH9YQNhWEjNLd5ZswXwDdapeubXmq8EQ3OfMNd51AqD1dXUyBmZI6zEiLxFoCn5hCYd+bfIqQVdzd4sgzlHPY/y0AmpOCnbrlkowtRC1A4mlaLHL0fQ== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SJ0PR13MB5545.namprd13.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230022)(4636009)(376002)(346002)(39830400003)(396003)(366004)(136003)(451199015)(316002)(6916009)(4326008)(66556008)(66946007)(66476007)(6486002)(8676002)(86362001)(26005)(6512007)(8936002)(83380400001)(36756003)(5660300002)(6506007)(44832011)(52116002)(41300700001)(6666004)(107886003)(2616005)(186003)(478600001)(2906002)(66574015)(38350700002)(38100700002); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?3IKUp5c5lqHLUEy0BcksIaiTbgxN?= =?utf-8?q?i5MeqmyOAbmL1dTTDwC8gwf7T7rEF69Bz8eD/qB6T1ocMHP+JJVd92mH75vBkxoY9?= =?utf-8?q?afL51q+uCLEnxKxFSizJEIRaZBGbmC6du201IYuGOuSlAA21AW4AGfaPdzKtZUC+j?= =?utf-8?q?JdKgV+zpM7ACVDBJZoIlAYnel0VPFSmlukqavJGCrRSqzQkodcV3KE+d76R+H7obr?= =?utf-8?q?fHCnm+FKK0FhdSUkBcCpZcX0wxtagZK5fqKhsPY1r9xRa+23uaPSi6GHYs+aDRvI8?= =?utf-8?q?C3sLyh6eSBC9TpjQnLeKyFqRTXMV8ZX/vkIBcu3MZQOrOX/69FTXAPCb2h4FyGGNf?= =?utf-8?q?1b3RpfS9cLe4axC3Q/YXO8rwdFgsKcBGSYSP8d6YGvdy77EEpA7HicSYx8J2M/EIz?= =?utf-8?q?7Y4y4/zXgsNB4Sx87a9U7NAT20231HiafvuUNLeW8HygUYisIQeVzXTYAsBF2JEbq?= =?utf-8?q?6R3IqxywSwFDg5lZjckggZH8IQiivQiehvk9Pvp5eqTnfap52z3yh4g4T8TW6+nqc?= =?utf-8?q?9yG7ahNOF2Bdt8VeDVQYiKSbu0MKVycDQwBiM7PuQiOrAuaFPmjrdtHBmU1WUVEde?= =?utf-8?q?k5x8tZIqm4JocynjKYG+beQvpMPMEaGvwgL/jquAfHE0nRjDzmcJu5FrsLkVG6r96?= =?utf-8?q?24BKvAJaYjQt8cZzzporVwieRqa/Nz4imxtV/B2m5oqMXwDxVzymTFgjMYx4fcUgF?= =?utf-8?q?v52aczrWqVJ6toob9+yphmulzM5hldM/AUqXyxugRmbBKWdU7mGEb1f/ezKEuCTSm?= =?utf-8?q?sS4xTLgGL7XLsstYieitqBtyALr2rfHQ+JKeSlHGkquGX8DXz7slE4fWpLQCvJN1G?= =?utf-8?q?0XZyQYPYDgyHUvdf3swZRFoqL6LfeyOr9z81Pue7zt9WcItHUI1qkgbMeZnyLIKuu?= =?utf-8?q?63AWyAoLqE5ho2Toh257obTP4r7DRYHCtcp69BKOH0AQgpqGh4AwtaAr2T65G3iLa?= =?utf-8?q?/RgonX0Md97Sh+cog+FIUX69kFgW84s1hYlfiMijgdgba7Z04bzKlKZavtFHwP8H0?= =?utf-8?q?zbEQwY+xVXyvByR+VR7YVqF+3sE3f5d6VFRFkRX40KoY/EZ33BCIW1smJtAjobDXG?= =?utf-8?q?2cUDp5lVdvwW3fB2yqDj4dRyUr43MuqPxSIkluzSdWEZY0HlouvDJGrhw5ugKHj5A?= =?utf-8?q?OneY7BKz1IRX2cs2448H3IfzcwysxK+7ttMGReAXfYgZs5Q/RdPd/bqASA4lPzhGA?= =?utf-8?q?mPpx0SAG2sNHSunv6W9TE2z8h4yvJE6sg/gO8ZjttoNARS7CMcMTTiTatn0VCIIn7?= =?utf-8?q?XZkNAr1XUlgaCcatVTJhhY6rj4j612tNnymP/j6MYsvI4qhWHkaG98pHVQZYuoVBl?= =?utf-8?q?Iy1WF73p9coT8Gj6qFaVxxRVBZ7slsDrY37co+tWNY5uRiW8MSSfRmhHirGzS3XNd?= =?utf-8?q?wMCSA3BT4f21o7rO+B60rC+QPWTr0HM4dsVumMjeJRJwg41PcLK9EasNvagvjQJXs?= =?utf-8?q?/EEAn2fz51O1IL9U+7TprDL1p6QImnGVuluUQVo0KZWTx2yPnYWXaAZsRSdVjlIUs?= =?utf-8?q?FKlo5ubq2fCLCuJADyLMAdmf+HCa25JtqA=3D=3D?= X-OriginatorOrg: corigine.com X-MS-Exchange-CrossTenant-Network-Message-Id: 930a11e5-734c-416a-1bf2-08da9f8cd1a0 X-MS-Exchange-CrossTenant-AuthSource: SJ0PR13MB5545.namprd13.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 26 Sep 2022 07:00:38.9233 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: fe128f2c-073b-4c20-818e-7246a585940c X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 0ShL9YVi1j3xPQGKCLgSXLa2Ykzr/HqyH/ppI7OqicCspDAKDfqyOMzlAw1AeGEY83kdFH8r44BhEVJYf+YQDYlhWUfDowI0J3bF6aPcpVQ= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR13MB4941 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Adds the start/stop/close routine of the flower PF vNIC. Signed-off-by: Chaoyong He Reviewed-by: Niklas Söderlund --- drivers/net/nfp/flower/nfp_flower.c | 210 ++++++++++++++++++++++++ drivers/net/nfp/flower/nfp_flower.h | 2 + drivers/net/nfp/flower/nfp_flower_representor.c | 2 + 3 files changed, 214 insertions(+) diff --git a/drivers/net/nfp/flower/nfp_flower.c b/drivers/net/nfp/flower/nfp_flower.c index c79b4a5..c287683 100644 --- a/drivers/net/nfp/flower/nfp_flower.c +++ b/drivers/net/nfp/flower/nfp_flower.c @@ -7,6 +7,7 @@ #include #include #include +#include #include #include @@ -25,10 +26,219 @@ #define CTRL_VNIC_NB_DESC 512 #define DEFAULT_FLBUF_SIZE 9216 +static void +nfp_pf_repr_enable_queues(struct rte_eth_dev *dev) +{ + struct nfp_net_hw *hw; + uint64_t enabled_queues = 0; + int i; + struct nfp_flower_representor *repr; + + repr = dev->data->dev_private; + hw = repr->app_fw_flower->pf_hw; + + /* Enabling the required TX queues in the device */ + for (i = 0; i < dev->data->nb_tx_queues; i++) + enabled_queues |= (1 << i); + + nn_cfg_writeq(hw, NFP_NET_CFG_TXRS_ENABLE, enabled_queues); + + enabled_queues = 0; + + /* Enabling the required RX queues in the device */ + for (i = 0; i < dev->data->nb_rx_queues; i++) + enabled_queues |= (1 << i); + + nn_cfg_writeq(hw, NFP_NET_CFG_RXRS_ENABLE, enabled_queues); +} + +static void +nfp_pf_repr_disable_queues(struct rte_eth_dev *dev) +{ + struct nfp_net_hw *hw; + uint32_t new_ctrl; + uint32_t update = 0; + struct nfp_flower_representor *repr; + + repr = (struct nfp_flower_representor *)dev->data->dev_private; + hw = repr->app_fw_flower->pf_hw; + + nn_cfg_writeq(hw, NFP_NET_CFG_TXRS_ENABLE, 0); + nn_cfg_writeq(hw, NFP_NET_CFG_RXRS_ENABLE, 0); + + new_ctrl = hw->ctrl & ~NFP_NET_CFG_CTRL_ENABLE; + update = NFP_NET_CFG_UPDATE_GEN | NFP_NET_CFG_UPDATE_RING | + NFP_NET_CFG_UPDATE_MSIX; + + if (hw->cap & NFP_NET_CFG_CTRL_RINGCFG) + new_ctrl &= ~NFP_NET_CFG_CTRL_RINGCFG; + + /* If an error when reconfig we avoid to change hw state */ + if (nfp_net_reconfig(hw, new_ctrl, update) < 0) + return; + + hw->ctrl = new_ctrl; +} + +int +nfp_flower_pf_start(struct rte_eth_dev *dev) +{ + int ret; + uint32_t new_ctrl; + uint32_t update = 0; + struct nfp_net_hw *hw; + struct nfp_flower_representor *repr; + + repr = (struct nfp_flower_representor *)dev->data->dev_private; + hw = repr->app_fw_flower->pf_hw; + + /* Disabling queues just in case... */ + nfp_pf_repr_disable_queues(dev); + + /* Enabling the required queues in the device */ + nfp_pf_repr_enable_queues(dev); + + new_ctrl = nfp_check_offloads(dev); + + /* Writing configuration parameters in the device */ + nfp_net_params_setup(hw); + + update |= NFP_NET_CFG_UPDATE_RSS; + + if (hw->cap & NFP_NET_CFG_CTRL_RSS2) + new_ctrl |= NFP_NET_CFG_CTRL_RSS2; + else + new_ctrl |= NFP_NET_CFG_CTRL_RSS; + + /* Enable device */ + new_ctrl |= NFP_NET_CFG_CTRL_ENABLE; + + update |= NFP_NET_CFG_UPDATE_GEN | NFP_NET_CFG_UPDATE_RING; + + if (hw->cap & NFP_NET_CFG_CTRL_RINGCFG) + new_ctrl |= NFP_NET_CFG_CTRL_RINGCFG; + + nn_cfg_writel(hw, NFP_NET_CFG_CTRL, new_ctrl); + + /* If an error when reconfig we avoid to change hw state */ + ret = nfp_net_reconfig(hw, new_ctrl, update); + if (ret != 0) { + PMD_INIT_LOG(ERR, "Failed to reconfig PF vnic"); + return -EIO; + } + + hw->ctrl = new_ctrl; + + /* Setup the freelist ring */ + ret = nfp_net_rx_freelist_setup(dev); + if (ret != 0) { + PMD_INIT_LOG(ERR, "Error with flower PF vNIC freelist setup"); + return -EIO; + } + + return 0; +} + +/* Stop device: disable rx and tx functions to allow for reconfiguring. */ +int +nfp_flower_pf_stop(struct rte_eth_dev *dev) +{ + uint16_t i; + struct nfp_net_hw *hw; + struct nfp_net_txq *this_tx_q; + struct nfp_net_rxq *this_rx_q; + struct nfp_flower_representor *repr; + + repr = (struct nfp_flower_representor *)dev->data->dev_private; + hw = repr->app_fw_flower->pf_hw; + + nfp_pf_repr_disable_queues(dev); + + /* Clear queues */ + for (i = 0; i < dev->data->nb_tx_queues; i++) { + this_tx_q = (struct nfp_net_txq *)dev->data->tx_queues[i]; + nfp_net_reset_tx_queue(this_tx_q); + } + + for (i = 0; i < dev->data->nb_rx_queues; i++) { + this_rx_q = (struct nfp_net_rxq *)dev->data->rx_queues[i]; + nfp_net_reset_rx_queue(this_rx_q); + } + + if (rte_eal_process_type() == RTE_PROC_PRIMARY) + /* Configure the physical port down */ + nfp_eth_set_configured(hw->cpp, hw->nfp_idx, 0); + else + nfp_eth_set_configured(dev->process_private, hw->nfp_idx, 0); + + return 0; +} + +/* Reset and stop device. The device can not be restarted. */ +static int +nfp_flower_pf_close(struct rte_eth_dev *dev) +{ + uint16_t i; + struct nfp_net_hw *hw; + struct nfp_pf_dev *pf_dev; + struct nfp_net_txq *this_tx_q; + struct nfp_net_rxq *this_rx_q; + struct nfp_flower_representor *repr; + struct nfp_app_fw_flower *app_fw_flower; + + if (rte_eal_process_type() != RTE_PROC_PRIMARY) + return 0; + + repr = (struct nfp_flower_representor *)dev->data->dev_private; + hw = repr->app_fw_flower->pf_hw; + pf_dev = hw->pf_dev; + app_fw_flower = NFP_PRIV_TO_APP_FW_FLOWER(pf_dev->app_fw_priv); + + /* + * We assume that the DPDK application is stopping all the + * threads/queues before calling the device close function. + */ + nfp_pf_repr_disable_queues(dev); + + /* Clear queues */ + for (i = 0; i < dev->data->nb_tx_queues; i++) { + this_tx_q = (struct nfp_net_txq *)dev->data->tx_queues[i]; + nfp_net_reset_tx_queue(this_tx_q); + } + + for (i = 0; i < dev->data->nb_rx_queues; i++) { + this_rx_q = (struct nfp_net_rxq *)dev->data->rx_queues[i]; + nfp_net_reset_rx_queue(this_rx_q); + } + + /* Cancel possible impending LSC work here before releasing the port*/ + rte_eal_alarm_cancel(nfp_net_dev_interrupt_delayed_handler, (void *)dev); + + nn_cfg_writeb(hw, NFP_NET_CFG_LSC, 0xff); + + rte_eth_dev_release_port(dev); + + /* Now it is safe to free all PF resources */ + PMD_DRV_LOG(INFO, "Freeing PF resources"); + nfp_cpp_area_free(pf_dev->ctrl_area); + nfp_cpp_area_free(pf_dev->hwqueues_area); + free(pf_dev->hwinfo); + free(pf_dev->sym_tbl); + nfp_cpp_free(pf_dev->cpp); + rte_free(app_fw_flower); + rte_free(pf_dev); + + return 0; +} + static const struct eth_dev_ops nfp_flower_pf_vnic_ops = { .dev_infos_get = nfp_net_infos_get, .link_update = nfp_net_link_update, .dev_configure = nfp_net_configure, + + .dev_start = nfp_flower_pf_start, + .dev_stop = nfp_flower_pf_stop, + .dev_close = nfp_flower_pf_close, }; static int diff --git a/drivers/net/nfp/flower/nfp_flower.h b/drivers/net/nfp/flower/nfp_flower.h index e1cd0a3..12b310f 100644 --- a/drivers/net/nfp/flower/nfp_flower.h +++ b/drivers/net/nfp/flower/nfp_flower.h @@ -55,5 +55,7 @@ struct nfp_app_fw_flower { int nfp_init_app_fw_flower(struct nfp_pf_dev *pf_dev); int nfp_secondary_init_app_fw_flower(struct nfp_cpp *cpp); +int nfp_flower_pf_start(struct rte_eth_dev *dev); +int nfp_flower_pf_stop(struct rte_eth_dev *dev); #endif /* _NFP_FLOWER_H_ */ diff --git a/drivers/net/nfp/flower/nfp_flower_representor.c b/drivers/net/nfp/flower/nfp_flower_representor.c index 19b1fed..b47b0a5 100644 --- a/drivers/net/nfp/flower/nfp_flower_representor.c +++ b/drivers/net/nfp/flower/nfp_flower_representor.c @@ -467,7 +467,9 @@ static const struct eth_dev_ops nfp_flower_pf_repr_dev_ops = { .dev_infos_get = nfp_flower_repr_dev_infos_get, + .dev_start = nfp_flower_pf_start, .dev_configure = nfp_flower_repr_dev_configure, + .dev_stop = nfp_flower_pf_stop, .rx_queue_setup = nfp_pf_repr_rx_queue_setup, .tx_queue_setup = nfp_pf_repr_tx_queue_setup, From patchwork Mon Sep 26 06:59:55 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Chaoyong He X-Patchwork-Id: 116852 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id BD60EA00C2; Mon, 26 Sep 2022 09:01:55 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 9F0C142B8E; Mon, 26 Sep 2022 09:00:46 +0200 (CEST) Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2090.outbound.protection.outlook.com [40.107.244.90]) by mails.dpdk.org (Postfix) with ESMTP id 409834069B for ; Mon, 26 Sep 2022 09:00:44 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=UIt14EfR2ZaSEhvPyNuoO/he9R8G+RbG0VOElAlbuCCuqp74rxhMn8162bbTFcIKrAwXWqu77KKkwgEwhXxN7imphxVCm88IE0pqdJkSGbhbZODtiLn09qn0i7+EbfRcHT73gKmPtNbTXoDCLRYuKOQedgSuBaXV3CUq9/OybbYqrwH/asqJQIObbs3MyI1L4ZipTe9gXVarY6fGxi4OitJAqGtnTUKjaYBCEl7ABm9WOp1LW/cP91BrUdRlnrObnvO9o+m1NFZUbfxO/udO+IBpCCNPR7FiPc0OQFNmeuARFktVUY8jzXt+ua9nbeNvA/dHtGdAozVxDIi8efwE9Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=O1/sl/qHmYymW/1KjYFvjPFSj9S3YslUJz55Kr9Ul70=; b=ll8Z35PMSZ44eWG9XfQQF7IZj8JKgx+mJIXYdW3QwskTDV6jiElJT5Dyu22cE+k3EtMF9w3QFPOKsu98hWd6Qg4HYk+d3tNS8peHjLLMl2vSFOKrQdXI4UeNwbnH48fPN9k0POGZNRcg8Lbu0dVTy96Xlucg/pHfUf3YcQeQd0piVnltRUO0QniYBDlxrr9t//wTiNqeCOfOqL7IiL9qFdjkEbXI8vhPn3dZ6dzL6RGswV1yFQSg5pIpTBepazCCD+M6NJICVEFH/3bzIfqIX6jS/80oUn1roiV6JHj8Xf+AMx6hAWMaQ8+Y+qgk//+tlfUgNvkz8cGn39ObQCf/AQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=corigine.com; dmarc=pass action=none header.from=corigine.com; dkim=pass header.d=corigine.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=corigine.onmicrosoft.com; s=selector2-corigine-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=O1/sl/qHmYymW/1KjYFvjPFSj9S3YslUJz55Kr9Ul70=; b=lZSAjuNg6MuNE7+KoR8oNjOXm4tffUPn9xa+G6OH8kCcC6XnWbpRTNvQaEgb2UoM5kmY34oTzCNt+W55djlczTZm1JoTq4B5eF9gIRsSvLfM0WRkX0pa7ebsp0EAh6uY0AklwkQKAMLN+xRZSbuCx0CFFj3uNi85G/xGBFYsDvc= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=corigine.com; Received: from SJ0PR13MB5545.namprd13.prod.outlook.com (2603:10b6:a03:424::5) by SA1PR13MB4893.namprd13.prod.outlook.com (2603:10b6:806:18a::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5676.14; Mon, 26 Sep 2022 07:00:41 +0000 Received: from SJ0PR13MB5545.namprd13.prod.outlook.com ([fe80::779:2722:a8e5:503b]) by SJ0PR13MB5545.namprd13.prod.outlook.com ([fe80::779:2722:a8e5:503b%7]) with mapi id 15.20.5676.014; Mon, 26 Sep 2022 07:00:41 +0000 From: Chaoyong He To: dev@dpdk.org Cc: oss-drivers@corigine.com, niklas.soderlund@corigine.com, Chaoyong He , Heinrich Kuhn Subject: [PATCH v10 11/13] net/nfp: move rxtx function to header file Date: Mon, 26 Sep 2022 14:59:55 +0800 Message-Id: <1664175597-37248-12-git-send-email-chaoyong.he@corigine.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1664175597-37248-1-git-send-email-chaoyong.he@corigine.com> References: <1664175597-37248-1-git-send-email-chaoyong.he@corigine.com> X-ClientProxiedBy: SG2P153CA0005.APCP153.PROD.OUTLOOK.COM (2603:1096::15) To SJ0PR13MB5545.namprd13.prod.outlook.com (2603:10b6:a03:424::5) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SJ0PR13MB5545:EE_|SA1PR13MB4893:EE_ X-MS-Office365-Filtering-Correlation-Id: 0add29c7-b48a-4352-0c6c-08da9f8cd2ee X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: PMoS8mPJ5XImSWExAk8uc4b6NLqgaYq7oC+Z1z/6BauwdfIro2jKNm6K9P2YsWgP97FB68rM3r0EegPaDNtBhKtgH70ekCWvpSg6wcX//xA/VTcMau6xgrzFAxbFM1OO2s2WnGeAw6SSxslRc4mGcNxGWMPbcbgqHeEK0xtStvLigCoc/QsVhWy6ZYNKKkKU8jvKrJN1g3aaNAqLYvDCshctZBGLY2gG/1bGxS3dNC+Q0z3AnH7sb1Q+fjlqXqVQeJWc9AqZdJP57TbVAU4Fpuw/wW05y1UzAzODldc0zkvRoFJ7fgamyqSko2vXSNgmD6c0cTqWEi6E1VVbP3c1oi34NR/legXFzEOIsT1mJGp+w7N3LmffDUGFUCFToF83AoP8W5GjDtk+BrnloMOrtnWKCkUZytLOfnQu4RHjuDFlg4a7qQTr8sqCBWHLWhv8r+esYez7y+PiB23fdGQQsXzMIxQoIs79VTct7rJDk0sjZi4ZU1mhdYzQskIfQya60buO8hwtkDGhH5SRgiHJtrTXE28dPScY3OeLqj/ujztHWaf4+B6Znc15jTHzEKTfMxAsRbI4gq+mY7eIQauLFuCbDvl944+gS5NKbptJdgfRmgunD88vhNt+0GwP3y6Y+Fozx+ZIzYpS6vsA5xPtEUedyrJnXP9SyYW5GR18u7d/Homi6rewhcw7YnI0ciReEwxIxScRTYGbcQgDp9etxhel/gv/NY5LT7zizD0hktzCAhVstZ9wLPwhKdiA8Fr3wb+QpsUw0Ooc0F0LVQqW+A== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SJ0PR13MB5545.namprd13.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230022)(4636009)(366004)(396003)(39830400003)(136003)(376002)(346002)(451199015)(36756003)(38100700002)(38350700002)(2906002)(86362001)(44832011)(66946007)(8936002)(66556008)(66476007)(5660300002)(8676002)(4326008)(83380400001)(186003)(54906003)(6512007)(6916009)(316002)(52116002)(6666004)(2616005)(26005)(6506007)(478600001)(41300700001)(107886003)(6486002); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?csb05/QUU8n9RPPmzNe867KyiV26?= =?utf-8?q?HgLBu2oh9ZiTK1YNfJnqHaMDBU5BbS2T5kE6hU64NpV3dsBU3kcqk2sJ77/nioM2h?= =?utf-8?q?p0Yc6H8k7Sc3JmQXlhpobWhrrkUq7qrPl2lS3/18BUbvxR2Lanmh4+b//6ZIDbfxK?= =?utf-8?q?abk1Yplfgwbk8ob7YfzADYzvF/QPPzmVZ2+751M82NwITBaM857fBTpzIvVFViqPM?= =?utf-8?q?7m8JKU/HVVc8E7swgjuQIr1gYjSZIftRyR4Pkn1pjYnx/QkmvSSa1AWpcf6uDBzdP?= =?utf-8?q?/FtI5GnEmxbvd68msCR8M2eeLMwCeNeWy9DAe86ISky/T8hu7SHRniCOb+cGE78vf?= =?utf-8?q?ziLNbBnXlLI0Obhiszi/l5F4+7xA/Ojw8lIob0p69L6qdNgrrljVWiNO+Cs3fNEaK?= =?utf-8?q?r6RMjeVD3vDVuI784VofNfYBK7JNpPVr6sCQunrDEeUDyqQZxCoY4lab+wut0TNwM?= =?utf-8?q?QLAWK+xUGGE+f6MVTlFYdox5seehTkBzKvqax2fPqSTWyfT8gvjKujJs597wN8wwu?= =?utf-8?q?Vv9BMRq7ahV8/B1ODTujWYEN0GMMuUXPfNzB4jOyJDBNqzC2TRHk6TqMkBOmjm35L?= =?utf-8?q?Qrorm7x3vqTqbFK2OSWi2IHnrvDEMaRMKxkCqNU2YsPMDxPy0BvHLAltTlROwo0IT?= =?utf-8?q?caHpFk0/7dLcRB4zlpcUWw5rUHErPlQinh1N6ATsGVpkVFN+ziQHkKyRjFAa8KX7d?= =?utf-8?q?EoWH0aDFVWAEGrUdBbIC7l2TxazKld7KKpqu6FFeS1/Rzz8SrV+vCfsOQ0nMwvxAA?= =?utf-8?q?E0rRHe1jMrhl+J3Gw79cu3SLZ8lTgSeOjRyxmypVMQpndj2lVizAAMv5I6UV6SR7Z?= =?utf-8?q?NcS9wzFRBuREui/MKlubqUGrIfkitOCmwdnnqo1tY3zhQjsE7cTyFXrAk3KUgAv6r?= =?utf-8?q?y7y1MSu5A5FFmKdQza14SKxGp+zqrgC8u5Sygsb74gprT6sN/1LaxDcGTxJFjfZey?= =?utf-8?q?Lpfg8q1o0bgV2prvQBeDyl5co1twAGerXhVuRULHAdUtWbQL/+NE5MlBotz0ONUpJ?= =?utf-8?q?mWOuimGqPna3pCmQebyFbnkWMp1/RPdsNZFlkPdDvFCsqUa1P4HkL+yGdqmDMtzWo?= =?utf-8?q?NEhSg+diE4zsJTsO66A9JcYhRr6n4nAQmFXxhhMLNG+BlWTdHzLBWKLFyRDWJg+Z1?= =?utf-8?q?K3jYV59V0nW4fR0trVjjUC7ENvQFkZJdCz6FQmsCNWHaPoKr/TK0s2Q6KJSV0VayR?= =?utf-8?q?0WEVnpBTQggOHsUiOYHi17yJnBiok7B2v4L0baV85sb19+cNc/9zPJu5kCNcczO5i?= =?utf-8?q?Ya+E5X/0SqrwIx0tc6pr7ZBpsNOIIWa56tEMAsvXzMSsMuiP6iA1RCVEzJ/kgjQLh?= =?utf-8?q?WEK4m08sZt6DZjni3gkEVAFLjBL5Wd3brgk96VPeg/8cD6Ny+nz+FkInMu8273GF+?= =?utf-8?q?fzm6JRo91ar1OlKN9qj2b5ta7WXzyUPkW6UJfYoc4Pq8bHW5qwY5UJMz/TIKgvLxE?= =?utf-8?q?DQeBYjDIC2KaXfn290q2H3qKaFcl67SboxRviEdHkQKpXpKBtd0TP7Xkfx9TZvWyb?= =?utf-8?q?mTa3ud/bdRkdlOUNtx1Wkaks6fJMsluMzQ=3D=3D?= X-OriginatorOrg: corigine.com X-MS-Exchange-CrossTenant-Network-Message-Id: 0add29c7-b48a-4352-0c6c-08da9f8cd2ee X-MS-Exchange-CrossTenant-AuthSource: SJ0PR13MB5545.namprd13.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 26 Sep 2022 07:00:41.0493 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: fe128f2c-073b-4c20-818e-7246a585940c X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: gjrES9X3y5sc2mj9MKBqcfLE+8k97nOFGnzovtIy9tspEskHtLBU4aZWFzfQoX7jHa8a/H8VyRm8x0nBPtdW0gXnwOm6h4WrHRV0ZWCoQYw= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR13MB4893 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org The flower firmware application makes use of the same Rx and Tx checksum logic as the normal PMD. Expose it so that flower firmware application also can make use of it. Signed-off-by: Chaoyong He Signed-off-by: Heinrich Kuhn Reviewed-by: Niklas Söderlund --- drivers/net/nfp/nfp_common.c | 2 +- drivers/net/nfp/nfp_ethdev.c | 2 +- drivers/net/nfp/nfp_ethdev_vf.c | 2 +- drivers/net/nfp/nfp_rxtx.c | 91 +---------------------------------------- drivers/net/nfp/nfp_rxtx.h | 90 ++++++++++++++++++++++++++++++++++++++++ 5 files changed, 94 insertions(+), 93 deletions(-) diff --git a/drivers/net/nfp/nfp_common.c b/drivers/net/nfp/nfp_common.c index fd46c84..511d74b 100644 --- a/drivers/net/nfp/nfp_common.c +++ b/drivers/net/nfp/nfp_common.c @@ -38,9 +38,9 @@ #include "nfpcore/nfp_nsp.h" #include "nfp_common.h" +#include "nfp_ctrl.h" #include "nfp_rxtx.h" #include "nfp_logs.h" -#include "nfp_ctrl.h" #include "nfp_cpp_bridge.h" #include diff --git a/drivers/net/nfp/nfp_ethdev.c b/drivers/net/nfp/nfp_ethdev.c index 5a9a1bd..d24d679 100644 --- a/drivers/net/nfp/nfp_ethdev.c +++ b/drivers/net/nfp/nfp_ethdev.c @@ -33,9 +33,9 @@ #include "nfpcore/nfp_nsp.h" #include "nfp_common.h" +#include "nfp_ctrl.h" #include "nfp_rxtx.h" #include "nfp_logs.h" -#include "nfp_ctrl.h" #include "nfp_cpp_bridge.h" #include "flower/nfp_flower.h" diff --git a/drivers/net/nfp/nfp_ethdev_vf.c b/drivers/net/nfp/nfp_ethdev_vf.c index affea9d..2235499 100644 --- a/drivers/net/nfp/nfp_ethdev_vf.c +++ b/drivers/net/nfp/nfp_ethdev_vf.c @@ -19,9 +19,9 @@ #include "nfpcore/nfp_rtsym.h" #include "nfp_common.h" +#include "nfp_ctrl.h" #include "nfp_rxtx.h" #include "nfp_logs.h" -#include "nfp_ctrl.h" static void nfp_netvf_read_mac(struct nfp_net_hw *hw) diff --git a/drivers/net/nfp/nfp_rxtx.c b/drivers/net/nfp/nfp_rxtx.c index 8d63a7b..95403a3 100644 --- a/drivers/net/nfp/nfp_rxtx.c +++ b/drivers/net/nfp/nfp_rxtx.c @@ -17,9 +17,9 @@ #include #include "nfp_common.h" +#include "nfp_ctrl.h" #include "nfp_rxtx.h" #include "nfp_logs.h" -#include "nfp_ctrl.h" #include "nfpcore/nfp_mip.h" #include "nfpcore/nfp_rtsym.h" #include "nfpcore/nfp-common/nfp_platform.h" @@ -208,34 +208,6 @@ } } -/* nfp_net_rx_cksum - set mbuf checksum flags based on RX descriptor flags */ -static inline void -nfp_net_rx_cksum(struct nfp_net_rxq *rxq, struct nfp_net_rx_desc *rxd, - struct rte_mbuf *mb) -{ - struct nfp_net_hw *hw = rxq->hw; - - if (!(hw->ctrl & NFP_NET_CFG_CTRL_RXCSUM)) - return; - - /* If IPv4 and IP checksum error, fail */ - if (unlikely((rxd->rxd.flags & PCIE_DESC_RX_IP4_CSUM) && - !(rxd->rxd.flags & PCIE_DESC_RX_IP4_CSUM_OK))) - mb->ol_flags |= RTE_MBUF_F_RX_IP_CKSUM_BAD; - else - mb->ol_flags |= RTE_MBUF_F_RX_IP_CKSUM_GOOD; - - /* If neither UDP nor TCP return */ - if (!(rxd->rxd.flags & PCIE_DESC_RX_TCP_CSUM) && - !(rxd->rxd.flags & PCIE_DESC_RX_UDP_CSUM)) - return; - - if (likely(rxd->rxd.flags & PCIE_DESC_RX_L4_CSUM_OK)) - mb->ol_flags |= RTE_MBUF_F_RX_L4_CKSUM_GOOD; - else - mb->ol_flags |= RTE_MBUF_F_RX_L4_CKSUM_BAD; -} - /* * RX path design: * @@ -768,67 +740,6 @@ return 0; } -/* nfp_net_tx_tso - Set TX descriptor for TSO */ -static inline void -nfp_net_nfd3_tx_tso(struct nfp_net_txq *txq, struct nfp_net_nfd3_tx_desc *txd, - struct rte_mbuf *mb) -{ - uint64_t ol_flags; - struct nfp_net_hw *hw = txq->hw; - - if (!(hw->cap & NFP_NET_CFG_CTRL_LSO_ANY)) - goto clean_txd; - - ol_flags = mb->ol_flags; - - if (!(ol_flags & RTE_MBUF_F_TX_TCP_SEG)) - goto clean_txd; - - txd->l3_offset = mb->l2_len; - txd->l4_offset = mb->l2_len + mb->l3_len; - txd->lso_hdrlen = mb->l2_len + mb->l3_len + mb->l4_len; - txd->mss = rte_cpu_to_le_16(mb->tso_segsz); - txd->flags = PCIE_DESC_TX_LSO; - return; - -clean_txd: - txd->flags = 0; - txd->l3_offset = 0; - txd->l4_offset = 0; - txd->lso_hdrlen = 0; - txd->mss = 0; -} - -/* nfp_net_tx_cksum - Set TX CSUM offload flags in TX descriptor */ -static inline void -nfp_net_nfd3_tx_cksum(struct nfp_net_txq *txq, struct nfp_net_nfd3_tx_desc *txd, - struct rte_mbuf *mb) -{ - uint64_t ol_flags; - struct nfp_net_hw *hw = txq->hw; - - if (!(hw->cap & NFP_NET_CFG_CTRL_TXCSUM)) - return; - - ol_flags = mb->ol_flags; - - /* IPv6 does not need checksum */ - if (ol_flags & RTE_MBUF_F_TX_IP_CKSUM) - txd->flags |= PCIE_DESC_TX_IP4_CSUM; - - switch (ol_flags & RTE_MBUF_F_TX_L4_MASK) { - case RTE_MBUF_F_TX_UDP_CKSUM: - txd->flags |= PCIE_DESC_TX_UDP_CSUM; - break; - case RTE_MBUF_F_TX_TCP_CKSUM: - txd->flags |= PCIE_DESC_TX_TCP_CSUM; - break; - } - - if (ol_flags & (RTE_MBUF_F_TX_IP_CKSUM | RTE_MBUF_F_TX_L4_MASK)) - txd->flags |= PCIE_DESC_TX_CSUM; -} - uint16_t nfp_net_nfd3_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts) { diff --git a/drivers/net/nfp/nfp_rxtx.h b/drivers/net/nfp/nfp_rxtx.h index a30171f..cd70bdd 100644 --- a/drivers/net/nfp/nfp_rxtx.h +++ b/drivers/net/nfp/nfp_rxtx.h @@ -360,6 +360,96 @@ struct nfp_net_rxq { return (nfp_net_nfd3_free_tx_desc(txq) < txq->tx_free_thresh); } +/* set mbuf checksum flags based on RX descriptor flags */ +static inline void +nfp_net_rx_cksum(struct nfp_net_rxq *rxq, struct nfp_net_rx_desc *rxd, + struct rte_mbuf *mb) +{ + struct nfp_net_hw *hw = rxq->hw; + + if (!(hw->ctrl & NFP_NET_CFG_CTRL_RXCSUM)) + return; + + /* If IPv4 and IP checksum error, fail */ + if (unlikely((rxd->rxd.flags & PCIE_DESC_RX_IP4_CSUM) && + !(rxd->rxd.flags & PCIE_DESC_RX_IP4_CSUM_OK))) + mb->ol_flags |= RTE_MBUF_F_RX_IP_CKSUM_BAD; + else + mb->ol_flags |= RTE_MBUF_F_RX_IP_CKSUM_GOOD; + + /* If neither UDP nor TCP return */ + if (!(rxd->rxd.flags & PCIE_DESC_RX_TCP_CSUM) && + !(rxd->rxd.flags & PCIE_DESC_RX_UDP_CSUM)) + return; + + if (likely(rxd->rxd.flags & PCIE_DESC_RX_L4_CSUM_OK)) + mb->ol_flags |= RTE_MBUF_F_RX_L4_CKSUM_GOOD; + else + mb->ol_flags |= RTE_MBUF_F_RX_L4_CKSUM_BAD; +} + +/* Set NFD3 TX descriptor for TSO */ +static inline void +nfp_net_nfd3_tx_tso(struct nfp_net_txq *txq, + struct nfp_net_nfd3_tx_desc *txd, + struct rte_mbuf *mb) +{ + uint64_t ol_flags; + struct nfp_net_hw *hw = txq->hw; + + if (!(hw->cap & NFP_NET_CFG_CTRL_LSO_ANY)) + goto clean_txd; + + ol_flags = mb->ol_flags; + + if (!(ol_flags & RTE_MBUF_F_TX_TCP_SEG)) + goto clean_txd; + + txd->l3_offset = mb->l2_len; + txd->l4_offset = mb->l2_len + mb->l3_len; + txd->lso_hdrlen = mb->l2_len + mb->l3_len + mb->l4_len; + txd->mss = rte_cpu_to_le_16(mb->tso_segsz); + txd->flags = PCIE_DESC_TX_LSO; + return; + +clean_txd: + txd->flags = 0; + txd->l3_offset = 0; + txd->l4_offset = 0; + txd->lso_hdrlen = 0; + txd->mss = 0; +} + +/* Set TX CSUM offload flags in NFD3 TX descriptor */ +static inline void +nfp_net_nfd3_tx_cksum(struct nfp_net_txq *txq, struct nfp_net_nfd3_tx_desc *txd, + struct rte_mbuf *mb) +{ + uint64_t ol_flags; + struct nfp_net_hw *hw = txq->hw; + + if (!(hw->cap & NFP_NET_CFG_CTRL_TXCSUM)) + return; + + ol_flags = mb->ol_flags; + + /* IPv6 does not need checksum */ + if (ol_flags & RTE_MBUF_F_TX_IP_CKSUM) + txd->flags |= PCIE_DESC_TX_IP4_CSUM; + + switch (ol_flags & RTE_MBUF_F_TX_L4_MASK) { + case RTE_MBUF_F_TX_UDP_CKSUM: + txd->flags |= PCIE_DESC_TX_UDP_CSUM; + break; + case RTE_MBUF_F_TX_TCP_CKSUM: + txd->flags |= PCIE_DESC_TX_TCP_CSUM; + break; + } + + if (ol_flags & (RTE_MBUF_F_TX_IP_CKSUM | RTE_MBUF_F_TX_L4_MASK)) + txd->flags |= PCIE_DESC_TX_CSUM; +} + int nfp_net_rx_freelist_setup(struct rte_eth_dev *dev); uint32_t nfp_net_rx_queue_count(void *rx_queue); uint16_t nfp_net_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, From patchwork Mon Sep 26 06:59:56 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Chaoyong He X-Patchwork-Id: 116853 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id A3DAAA00C2; Mon, 26 Sep 2022 09:02:03 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 9389242B98; Mon, 26 Sep 2022 09:00:47 +0200 (CEST) Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2090.outbound.protection.outlook.com [40.107.244.90]) by mails.dpdk.org (Postfix) with ESMTP id C5A464069B for ; Mon, 26 Sep 2022 09:00:44 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=enyURqtSSl1eZDk7ecGd7wSxakfNcsBPQ57SDmShU1IGffWJjSBWfSs+o09II4ADwfIJ/Y6P+aKoR8o20poY2jhOqI4vhT0cesjufL2gAOd1akyal6HzIgm8iXFpGmB+gxM3Ug6qR3L2IAqKLoamkxx/5krtjhwqy0phKyz6EY33tWGwJmHsC79U1gj9hg5kufVjv8PhvmvWRZyNpF14GQF8kCAQfim34qJJ5mRSZxEzUjw5dm86pH3z1C24AUizjsHX/9VfPJbMIwHy5I/SAcqsa18r8A0YxsnhomfYAghZbwkH3KY4Y0rpiC0sOUZOGq5QBgiGAfNepVMvFPlgmQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=D9n3cYMoWdflgzasmP+38zc0wdSOi/CLd3FqQZNZSF0=; b=B2h1DFIGq3sIGQOS5Uw83JDRjUSMhZ2SC53sX10jCMwr6p4ZLdU9YRhnA5iB3FTLu+t26zF2UFpQf70/a1VvSlsODZIrZM+6qmAxiHzwG2aM+YrdUIuLG+2E5Bj34jzZOZXiQLsHUDbwdXwICncCuqiElJ+r2A+gB3Q1MQ0t7tuN7XlkUKZgakJ7XB06v35vy2tK0vJ2m8icK3I6zDVirDSVzwySV2PWg5CjZl1Y4QsHjVWORoYqNQz/hVnQDKdDnEuVnXn3p9mFEEDEQzKhL0NwHdZW68c2UwDstN5UT3+LQVEa3i2BX+k5YIe0m0DYcPBIL85bZTvUhVl0l1lR5Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=corigine.com; dmarc=pass action=none header.from=corigine.com; dkim=pass header.d=corigine.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=corigine.onmicrosoft.com; s=selector2-corigine-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=D9n3cYMoWdflgzasmP+38zc0wdSOi/CLd3FqQZNZSF0=; b=peWLraYAs9htHcCBOF1gUFpIlvIIvhrjVfV/wF3H48BVHPMvLxO4QMvkoGdsz++14CW/QZhvMxgLESSbhDOCblw9mbvPQr9+q+kK54QAagqVyohAQJNDOXIFJcBzuRTVx1xcs+Ad5oFt6j+k7CmkUEs9Rxe8zgCqLi8vpNqg648= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=corigine.com; Received: from SJ0PR13MB5545.namprd13.prod.outlook.com (2603:10b6:a03:424::5) by SA1PR13MB4893.namprd13.prod.outlook.com (2603:10b6:806:18a::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5676.14; Mon, 26 Sep 2022 07:00:43 +0000 Received: from SJ0PR13MB5545.namprd13.prod.outlook.com ([fe80::779:2722:a8e5:503b]) by SJ0PR13MB5545.namprd13.prod.outlook.com ([fe80::779:2722:a8e5:503b%7]) with mapi id 15.20.5676.014; Mon, 26 Sep 2022 07:00:43 +0000 From: Chaoyong He To: dev@dpdk.org Cc: oss-drivers@corigine.com, niklas.soderlund@corigine.com, Chaoyong He Subject: [PATCH v10 12/13] net/nfp: add flower PF rxtx logic Date: Mon, 26 Sep 2022 14:59:56 +0800 Message-Id: <1664175597-37248-13-git-send-email-chaoyong.he@corigine.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1664175597-37248-1-git-send-email-chaoyong.he@corigine.com> References: <1664175597-37248-1-git-send-email-chaoyong.he@corigine.com> X-ClientProxiedBy: SG2P153CA0005.APCP153.PROD.OUTLOOK.COM (2603:1096::15) To SJ0PR13MB5545.namprd13.prod.outlook.com (2603:10b6:a03:424::5) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SJ0PR13MB5545:EE_|SA1PR13MB4893:EE_ X-MS-Office365-Filtering-Correlation-Id: cdba6f40-64fc-4397-2c40-08da9f8cd43e X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: AjJOLtn1tY0uYq0kXEBwy1+QVrLOUMtqsMfTtzDRPkH/zRSQMojWE9ufuhyCCdtP7G+PF2riyRlTLpT5DjklsJVhQ8YOD2MEoXDORhHsPYMjx0RBpdD4j2Ru2D9Fp+O/bjP93xo9UGNC91KdqBaJV8pftMACjxwO1g1Z9LjlloIefLPNttirtW/29b/w1axbCKVOOS5U4YZeC8yY/V+XHyKDJ9RladK6Xxoxg6NDO3LCPRcPCRiMWOrgwJflEQTWLaeVhM+/PP/xR8VVn98mp4Wu2CnlWUUYS9IRt/F+2afurh+shVqgKwb2ksU3y4bCdKNOOvuo2ldF0/oeDnukmU1PAmVLGLG+AlRcnxPASmsjoifjTsegkgGnZAPmkkC2SeWss6FII5OJ0q6D7G5Amt6z0l24aT7vRxOoB2NZnlDTgERHMuN/nMJ/VMMUanNrzDyWKcQEOCNG8FJ54voLWVohS0rbizD6d8YhNXLzn/kSF3TtiK0ha7zSGf2zLFO7NZnaGqxB86JKTUQ3JQyRFe3AYEjhGB6l3ACZ/JUPrnxVw0kCapdWxsyucvyN7zN87fD/n8nJCCJQwifQpnSBRDXjjQXHIUSDyA9pzYGlRo57rDDrP06Cl3PWPMD42wCEI1iww/iG/xltJw5RQ0Tdk9J3X36kFnNGYt/N+NfyvbkxILUHUMPBVRvI020lGlyikSozaYNxC/N1bl6raHxTjhg+BLIoj3P7WC3d3aUoaaeex7uAUu2ySZQWg812JXiQUQhogirox41g1IjPt4xl5w== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SJ0PR13MB5545.namprd13.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230022)(4636009)(366004)(396003)(39830400003)(136003)(376002)(346002)(451199015)(36756003)(38100700002)(38350700002)(2906002)(86362001)(44832011)(30864003)(66946007)(8936002)(66556008)(66476007)(5660300002)(8676002)(4326008)(83380400001)(186003)(66574015)(6512007)(6916009)(316002)(52116002)(6666004)(2616005)(26005)(6506007)(478600001)(41300700001)(107886003)(6486002); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?kC5eSII5IP7H+qgAW1zL/9VdcOye?= =?utf-8?q?EFvDR/V0fqwxF26yvExLLooTISGOsots3I/SoZbsvJgc5iGizspB/k3nptnpzL6dZ?= =?utf-8?q?xj1KglRFd3ldPzL/4Lix5yej8XQgarChz7ZQSpsiegP0M8CdNl/OnrwKdvI8KazOP?= =?utf-8?q?izbiO/NhjDDFPQgHmHkOfFq8JdygT/KdZHM3KvVSR9HhAZPPouuPZUINLaMA2416A?= =?utf-8?q?uUf8jdK7A2WWAaiiUpBZJBgDMvNg+x6oUf+4Js2+N25+ccAG2Z2OS+uufsPg1Z7ZP?= =?utf-8?q?gLnRVFuYiEwX2LZSmICYsX+PN3JAUuTygfVL8SIH7MiuxiZDJN4DEokx52rAy1ZJN?= =?utf-8?q?Yo72yIF6qu0oYebm7JAY7dyM1CkNNePzhW42+SDcw5WKNdn48J5qmL3MatX3slot7?= =?utf-8?q?FW+uFPIOxLVaN/RA7e0C/0tAZSXS8CA9BhO/1isqKjIVf3xW0JcYQ6enflmToyp48?= =?utf-8?q?TjNQU9B9CBP4KdNfGqLcC2XAo/vC2zKF99lFMpJ6AI9LoHw2547aSUfoPs4eEVABS?= =?utf-8?q?PXoFQDbV1eyABP+CDR6SwHaQn54iBtmfisWFo6lSA5LF9fb5BkgT4QmCfCG85CsTf?= =?utf-8?q?xJW0Dtm9rPSYD15BU8Agll4b2v+aTdRtL7rfP39XWZPNtrJuwTEVd7PjCY7V80Abb?= =?utf-8?q?/rth3zw3VwEa8cspDbfQeX0puxfdhUdN+MLBiAeR/DTIhgpzlvU883pMZjr9IWOsD?= =?utf-8?q?N12F3fJqTHLDt1ev8/+sBnTsJ9cIm6+HgJATCgB7QHNhqbKr926LFfrMy7cH4dFnH?= =?utf-8?q?NfoIhlyf6bSJvD2g8W2ZWcuzphl+vSCSN3oGeRSpVDul5cJOl+Tz1kKOt1CZZCB8A?= =?utf-8?q?UDfBhwDinbWXLW/MZydX6gkFX94bS/RaekITD44L0/mbNODjUxHya5xBfCpFIr9HW?= =?utf-8?q?MVPcvezgctBK1r2MlkH4Q7q/+qxv3O9MZv+WQVJUsQRYSYfDjt6TrcMmCUT9BmHXY?= =?utf-8?q?KBDzr+garcxnQRAA5iaSw2sm4dZS54Rh2F60EemE/wd5oPb4KM/aPZ0+BA94mYhGv?= =?utf-8?q?REuM+yV235NV34uFu3k/zelmv3opB3uL+Yl2bZQu/uoNs0IvBcwUl8bwBbZiWyu+x?= =?utf-8?q?aQr90t/Ejc782xv1CfRiezp69fZHyJ4U1VKLCGYz04vw4RfH3t3PWypkdcP+2f/Ec?= =?utf-8?q?P71/Qa4GOxcB+7JzQa2JjpcUuI82t1C7A2Gil87fm6pajRkeX1g3lr/bDSsV6IrfG?= =?utf-8?q?KLT0LsdZaPOtBNjqkOu4+7j2XQIbx+fmlgFJZVWKhjNZyyL0bgGKY71Q9A/j8GUS5?= =?utf-8?q?D7Zt15QVknPRNXH6dhl85b8+BE473zD3ixjG1c8Lm1ye4qCx1K/wwEBdHS025a10S?= =?utf-8?q?7fIfG93+PkNfQX61s2Km1DDxQvtMF+EmfK3XLFuDFI/65TNygKS3zI1yWWYeodQam?= =?utf-8?q?4AuPhO39WHzS0CdTZYwcYKLedju4XWudMqq3c9sbv80h04Um5DH1E1wuVGYTbBmAx?= =?utf-8?q?oVE0iOOygQKKSTC5iagaXV9t5bDRWxL2tJtD3Iyjvz3Xeis2ISEiS9oqFZFhqGmla?= =?utf-8?q?fC4Li5PWZnG8HwU6mSOoGhVWJaYRb1tMyw=3D=3D?= X-OriginatorOrg: corigine.com X-MS-Exchange-CrossTenant-Network-Message-Id: cdba6f40-64fc-4397-2c40-08da9f8cd43e X-MS-Exchange-CrossTenant-AuthSource: SJ0PR13MB5545.namprd13.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 26 Sep 2022 07:00:43.4240 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: fe128f2c-073b-4c20-818e-7246a585940c X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: FNHYWvvo8PCqiuXZ/vWfv42gqGvpwAraJ93ySYYgfR812x+ftHI3CJsPAxgP5Age3RDUeEIxnOh62KlTnMter/VQRk9lgDjIPi3a4GgNfQk= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR13MB4893 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org For the Rx logic, fallback packets are multiplexed to the correct representor port based on the prepended metadata. For the Tx logic, because fallback packets are prepended metadata, the start of the packet has to be adjusted for in the Tx descriptor. Signed-off-by: Chaoyong He Reviewed-by: Niklas Söderlund --- doc/guides/nics/nfp.rst | 13 ++ drivers/net/nfp/flower/nfp_flower.c | 380 ++++++++++++++++++++++++++++++++++++ drivers/net/nfp/flower/nfp_flower.h | 4 + 3 files changed, 397 insertions(+) diff --git a/doc/guides/nics/nfp.rst b/doc/guides/nics/nfp.rst index c62b1fb..b74067c 100644 --- a/doc/guides/nics/nfp.rst +++ b/doc/guides/nics/nfp.rst @@ -200,3 +200,16 @@ The flower firmware application support representor port for VF and physical port. There will always exist a representor port for each physical port, and the number of the representor port for VF is specified by the user through parameter. + +In the Rx direction, the flower firmware application will prepend the input +port information into metadata for each packet which can't offloaded. The PF +vNIC service will keep polling packets from the firmware, and multiplex them +to the corresponding representor port. + +In the Tx direction, the representor port will prepend the output port +information into metadata for each packet, and then send it to firmware through +PF vNIC. + +The ctrl vNIC service handling various control message, like the creation and +configuration of representor port, the pattern and action of flow rules, the +statistics of flow rules, and so on. diff --git a/drivers/net/nfp/flower/nfp_flower.c b/drivers/net/nfp/flower/nfp_flower.c index c287683..1daddd4 100644 --- a/drivers/net/nfp/flower/nfp_flower.c +++ b/drivers/net/nfp/flower/nfp_flower.c @@ -22,6 +22,7 @@ #include "nfp_flower.h" #include "nfp_flower_ctrl.h" #include "nfp_flower_representor.h" +#include "nfp_flower_cmsg.h" #define CTRL_VNIC_NB_DESC 512 #define DEFAULT_FLBUF_SIZE 9216 @@ -241,6 +242,383 @@ .dev_close = nfp_flower_pf_close, }; +static inline void +nfp_flower_parse_metadata(struct nfp_net_rxq *rxq, + struct nfp_net_rx_desc *rxd, + struct rte_mbuf *mbuf, + uint32_t *portid) +{ + uint32_t meta_info; + uint8_t *meta_offset; + struct nfp_net_hw *hw; + + hw = rxq->hw; + if (!((hw->ctrl & NFP_NET_CFG_CTRL_RSS) || + (hw->ctrl & NFP_NET_CFG_CTRL_RSS2))) + return; + + meta_offset = rte_pktmbuf_mtod(mbuf, uint8_t *); + meta_offset -= NFP_DESC_META_LEN(rxd); + meta_info = rte_be_to_cpu_32(*(uint32_t *)meta_offset); + meta_offset += 4; + + while (meta_info != 0) { + switch (meta_info & NFP_NET_META_FIELD_MASK) { + /* Expect flower firmware to only send packets with META_PORTID */ + case NFP_NET_META_PORTID: + *portid = rte_be_to_cpu_32(*(uint32_t *)meta_offset); + meta_offset += 4; + meta_info >>= NFP_NET_META_FIELD_SIZE; + break; + default: + /* Unsupported metadata can be a performance issue */ + return; + } + } +} + +static inline struct nfp_flower_representor * +nfp_flower_get_repr(struct nfp_net_hw *hw, + uint32_t port_id) +{ + uint8_t port; + struct nfp_app_fw_flower *app_fw_flower; + + /* Obtain handle to app_fw_flower here */ + app_fw_flower = NFP_PRIV_TO_APP_FW_FLOWER(hw->pf_dev->app_fw_priv); + + switch (NFP_FLOWER_CMSG_PORT_TYPE(port_id)) { + case NFP_FLOWER_CMSG_PORT_TYPE_PHYS_PORT: + port = NFP_FLOWER_CMSG_PORT_PHYS_PORT_NUM(port_id); + return app_fw_flower->phy_reprs[port]; + case NFP_FLOWER_CMSG_PORT_TYPE_PCIE_PORT: + port = NFP_FLOWER_CMSG_PORT_VNIC(port_id); + return app_fw_flower->vf_reprs[port]; + default: + break; + } + + return NULL; +} + +uint16_t +nfp_flower_pf_recv_pkts(void *rx_queue, + struct rte_mbuf **rx_pkts, + uint16_t nb_pkts) +{ + /* + * We need different counters for packets given to the caller + * and packets sent to representors + */ + int avail = 0; + int avail_multiplexed = 0; + uint64_t dma_addr; + uint32_t meta_portid; + uint16_t nb_hold = 0; + struct rte_mbuf *mb; + struct nfp_net_hw *hw; + struct rte_mbuf *new_mb; + struct nfp_net_rxq *rxq; + struct nfp_net_rx_buff *rxb; + struct nfp_net_rx_desc *rxds; + struct nfp_flower_representor *repr; + + rxq = rx_queue; + if (unlikely(rxq == NULL)) { + /* + * DPDK just checks the queue is lower than max queues + * enabled. But the queue needs to be configured + */ + RTE_LOG_DP(ERR, PMD, "RX Bad queue\n"); + return 0; + } + + hw = rxq->hw; + + /* + * This is tunable as we could allow to receive more packets than + * requested if most are multiplexed. + */ + while (avail + avail_multiplexed < nb_pkts) { + rxb = &rxq->rxbufs[rxq->rd_p]; + if (unlikely(rxb == NULL)) { + RTE_LOG_DP(ERR, PMD, "rxb does not exist!\n"); + break; + } + + rxds = &rxq->rxds[rxq->rd_p]; + if ((rxds->rxd.meta_len_dd & PCIE_DESC_RX_DD) == 0) + break; + + /* + * Memory barrier to ensure that we won't do other + * reads before the DD bit. + */ + rte_rmb(); + + /* + * We got a packet. Let's alloc a new mbuf for refilling the + * free descriptor ring as soon as possible + */ + new_mb = rte_pktmbuf_alloc(rxq->mem_pool); + if (unlikely(new_mb == NULL)) { + RTE_LOG_DP(DEBUG, PMD, + "RX mbuf alloc failed port_id=%u queue_id=%d\n", + rxq->port_id, rxq->qidx); + nfp_net_mbuf_alloc_failed(rxq); + break; + } + + nb_hold++; + + /* + * Grab the mbuf and refill the descriptor with the + * previously allocated mbuf + */ + mb = rxb->mbuf; + rxb->mbuf = new_mb; + + PMD_RX_LOG(DEBUG, "Packet len: %u, mbuf_size: %u", + rxds->rxd.data_len, rxq->mbuf_size); + + /* Size of this segment */ + mb->data_len = rxds->rxd.data_len - NFP_DESC_META_LEN(rxds); + /* Size of the whole packet. We just support 1 segment */ + mb->pkt_len = rxds->rxd.data_len - NFP_DESC_META_LEN(rxds); + + if (unlikely((mb->data_len + hw->rx_offset) > rxq->mbuf_size)) { + /* + * This should not happen and the user has the + * responsibility of avoiding it. But we have + * to give some info about the error + */ + RTE_LOG_DP(ERR, PMD, + "mbuf overflow likely due to the RX offset.\n" + "\t\tYour mbuf size should have extra space for" + " RX offset=%u bytes.\n" + "\t\tCurrently you just have %u bytes available" + " but the received packet is %u bytes long", + hw->rx_offset, + rxq->mbuf_size - hw->rx_offset, + mb->data_len); + rte_pktmbuf_free(mb); + break; + } + + /* Filling the received mbuf with packet info */ + if (hw->rx_offset != 0) + mb->data_off = RTE_PKTMBUF_HEADROOM + hw->rx_offset; + else + mb->data_off = RTE_PKTMBUF_HEADROOM + NFP_DESC_META_LEN(rxds); + + /* No scatter mode supported */ + mb->nb_segs = 1; + mb->next = NULL; + + mb->port = rxq->port_id; + meta_portid = 0; + + /* Checking the RSS flag */ + nfp_flower_parse_metadata(rxq, rxds, mb, &meta_portid); + PMD_RX_LOG(DEBUG, "Received from port %u type %u", + NFP_FLOWER_CMSG_PORT_VNIC(meta_portid), + NFP_FLOWER_CMSG_PORT_VNIC_TYPE(meta_portid)); + + /* Checking the checksum flag */ + nfp_net_rx_cksum(rxq, rxds, mb); + + if ((rxds->rxd.flags & PCIE_DESC_RX_VLAN) && + (hw->ctrl & NFP_NET_CFG_CTRL_RXVLAN)) { + mb->vlan_tci = rte_cpu_to_le_32(rxds->rxd.vlan); + mb->ol_flags |= RTE_MBUF_F_RX_VLAN | RTE_MBUF_F_RX_VLAN_STRIPPED; + } + + repr = nfp_flower_get_repr(hw, meta_portid); + if (repr != NULL && repr->ring != NULL) { + PMD_RX_LOG(DEBUG, "Using representor %s", repr->name); + rte_ring_enqueue(repr->ring, (void *)mb); + avail_multiplexed++; + } else if (repr != NULL) { + PMD_RX_LOG(ERR, "[%u] No ring available for repr_port %s\n", + hw->idx, repr->name); + PMD_RX_LOG(DEBUG, "Adding the mbuf to the mbuf array passed by the app"); + rx_pkts[avail++] = mb; + } else { + PMD_RX_LOG(DEBUG, "Adding the mbuf to the mbuf array passed by the app"); + rx_pkts[avail++] = mb; + } + + /* Now resetting and updating the descriptor */ + rxds->vals[0] = 0; + rxds->vals[1] = 0; + dma_addr = rte_cpu_to_le_64(RTE_MBUF_DMA_ADDR_DEFAULT(new_mb)); + rxds->fld.dd = 0; + rxds->fld.dma_addr_hi = (dma_addr >> 32) & 0xff; + rxds->fld.dma_addr_lo = dma_addr & 0xffffffff; + + rxq->rd_p++; + if (unlikely(rxq->rd_p == rxq->rx_count)) + rxq->rd_p = 0; + } + + if (nb_hold == 0) + return nb_hold; + + PMD_RX_LOG(DEBUG, "RX port_id=%u queue_id=%d, %d packets received", + rxq->port_id, rxq->qidx, nb_hold); + + nb_hold += rxq->nb_rx_hold; + + /* + * FL descriptors needs to be written before incrementing the + * FL queue WR pointer + */ + rte_wmb(); + if (nb_hold > rxq->rx_free_thresh) { + PMD_RX_LOG(DEBUG, "port=%u queue=%d nb_hold=%u avail=%d", + rxq->port_id, rxq->qidx, nb_hold, avail); + nfp_qcp_ptr_add(rxq->qcp_fl, NFP_QCP_WRITE_PTR, nb_hold); + nb_hold = 0; + } + + rxq->nb_rx_hold = nb_hold; + + return avail; +} + +uint16_t +nfp_flower_pf_xmit_pkts(void *tx_queue, + struct rte_mbuf **tx_pkts, + uint16_t nb_pkts) +{ + int i = 0; + int pkt_size; + int dma_size; + uint64_t dma_addr; + uint16_t free_descs; + uint16_t issued_descs; + struct rte_mbuf *pkt; + struct nfp_net_hw *hw; + struct rte_mbuf **lmbuf; + struct nfp_net_txq *txq; + struct nfp_net_nfd3_tx_desc txd; + struct nfp_net_nfd3_tx_desc *txds; + + txq = tx_queue; + hw = txq->hw; + txds = &txq->txds[txq->wr_p]; + + PMD_TX_LOG(DEBUG, "working for queue %d at pos %u and %u packets", + txq->qidx, txq->wr_p, nb_pkts); + + if ((nfp_net_nfd3_free_tx_desc(txq) < nb_pkts) || (nfp_net_nfd3_txq_full(txq))) + nfp_net_tx_free_bufs(txq); + + free_descs = (uint16_t)nfp_net_nfd3_free_tx_desc(txq); + if (unlikely(free_descs == 0)) + return 0; + + pkt = *tx_pkts; + issued_descs = 0; + + /* Sending packets */ + while ((i < nb_pkts) && free_descs) { + /* Grabbing the mbuf linked to the current descriptor */ + lmbuf = &txq->txbufs[txq->wr_p].mbuf; + /* Warming the cache for releasing the mbuf later on */ + RTE_MBUF_PREFETCH_TO_FREE(*lmbuf); + + pkt = *(tx_pkts + i); + + if (unlikely(pkt->nb_segs > 1 && + !(hw->cap & NFP_NET_CFG_CTRL_GATHER))) { + PMD_INIT_LOG(INFO, "NFP_NET_CFG_CTRL_GATHER not set"); + PMD_INIT_LOG(INFO, "Multisegment packet unsupported"); + goto xmit_end; + } + + /* Checking if we have enough descriptors */ + if (unlikely(pkt->nb_segs > free_descs)) + goto xmit_end; + + /* + * Checksum and VLAN flags just in the first descriptor for a + * multisegment packet, but TSO info needs to be in all of them. + */ + txd.data_len = pkt->pkt_len; + nfp_net_nfd3_tx_tso(txq, &txd, pkt); + nfp_net_nfd3_tx_cksum(txq, &txd, pkt); + + if ((pkt->ol_flags & RTE_MBUF_F_TX_VLAN) && + (hw->cap & NFP_NET_CFG_CTRL_TXVLAN)) { + txd.flags |= PCIE_DESC_TX_VLAN; + txd.vlan = pkt->vlan_tci; + } + + /* + * mbuf data_len is the data in one segment and pkt_len data + * in the whole packet. When the packet is just one segment, + * then data_len = pkt_len + */ + pkt_size = pkt->pkt_len; + + while (pkt != NULL) { + /* Copying TSO, VLAN and cksum info */ + *txds = txd; + + /* Releasing mbuf used by this descriptor previously*/ + if (*lmbuf != NULL) + rte_pktmbuf_free_seg(*lmbuf); + + /* + * Linking mbuf with descriptor for being released + * next time descriptor is used + */ + *lmbuf = pkt; + + dma_size = pkt->data_len; + dma_addr = rte_mbuf_data_iova(pkt); + + /* Filling descriptors fields */ + txds->dma_len = dma_size; + txds->data_len = txd.data_len; + txds->dma_addr_hi = (dma_addr >> 32) & 0xff; + txds->dma_addr_lo = (dma_addr & 0xffffffff); + ASSERT(free_descs > 0); + free_descs--; + + txq->wr_p++; + if (unlikely(txq->wr_p == txq->tx_count)) /* wrapping?*/ + txq->wr_p = 0; + + pkt_size -= dma_size; + + /* + * Making the EOP, packets with just one segment + * the priority + */ + if (likely(pkt_size == 0)) + txds->offset_eop = PCIE_DESC_TX_EOP | FLOWER_PKT_DATA_OFFSET; + else + txds->offset_eop = 0; + + pkt = pkt->next; + /* Referencing next free TX descriptor */ + txds = &txq->txds[txq->wr_p]; + lmbuf = &txq->txbufs[txq->wr_p].mbuf; + issued_descs++; + } + i++; + } + +xmit_end: + /* Increment write pointers. Force memory write before we let HW know */ + rte_wmb(); + nfp_qcp_ptr_add(txq->qcp_q, NFP_QCP_WRITE_PTR, issued_descs); + + return i; +} + static int nfp_flower_init_vnic_common(struct nfp_net_hw *hw, const char *vnic_type) { @@ -818,6 +1196,8 @@ eth_dev->process_private = cpp; eth_dev->dev_ops = &nfp_flower_pf_vnic_ops; + eth_dev->rx_pkt_burst = nfp_flower_pf_recv_pkts; + eth_dev->tx_pkt_burst = nfp_flower_pf_xmit_pkts; rte_eth_dev_probing_finish(eth_dev); return 0; diff --git a/drivers/net/nfp/flower/nfp_flower.h b/drivers/net/nfp/flower/nfp_flower.h index 12b310f..48f597a 100644 --- a/drivers/net/nfp/flower/nfp_flower.h +++ b/drivers/net/nfp/flower/nfp_flower.h @@ -55,6 +55,10 @@ struct nfp_app_fw_flower { int nfp_init_app_fw_flower(struct nfp_pf_dev *pf_dev); int nfp_secondary_init_app_fw_flower(struct nfp_cpp *cpp); +uint16_t nfp_flower_pf_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, + uint16_t nb_pkts); +uint16_t nfp_flower_pf_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, + uint16_t nb_pkts); int nfp_flower_pf_start(struct rte_eth_dev *dev); int nfp_flower_pf_stop(struct rte_eth_dev *dev); From patchwork Mon Sep 26 06:59:57 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Chaoyong He X-Patchwork-Id: 116854 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 2B0E7A00C2; Mon, 26 Sep 2022 09:02:13 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id CF50042BA6; Mon, 26 Sep 2022 09:00:48 +0200 (CEST) Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2120.outbound.protection.outlook.com [40.107.244.120]) by mails.dpdk.org (Postfix) with ESMTP id 42CE9427EC for ; Mon, 26 Sep 2022 09:00:47 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=KtkBgGvgSZlJGSKcqmLwrmo5UDr4PZWYL1qIcLAvfB4gzJLPG66EWeh2pPpNVsrucivni2C5kXIhZ9EhPfEuOSFkZN35BI1MZ1dvSwNIgi7k3Ywcgf/oBMhF4uQTvi1AO2oEtgBHYZ6Cx9ZkMm17FAWi4X2cdCXxRIEhWGaXb3wLwndIM2KM6BTw8oOZD8zFZAAgT0abMBMgzBwRRMyfbggZayKGqJW3wcF/sBNMJQ1ZXZF6pjbWdyjkRqw2BP4FGraa3fjfo8DFvbBPRS4pC9zHYWwk0GjyeJZaBIANbqR/mp1P+D3/cJY6N8j4FDt0IcOEyy1MjeUyggy5FxLJcg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=TYo1ps100S5z4jqu7CL7/ub9AInvZ68bHuZZmd4m4kU=; b=aAjEOm1pAzxM6VTUV6SkFIiXJAPb6JPlZIsqHuRxYyPuDhZ4FWhjBT32R7ql1gU6JNyz3hKCs+0k/1NXqfsKl/Dnuw4qFzh48kUso+8t5TE54ahmoK/Rr1QpT75KegzfXj06wIxrBbVXNSpzW2Fdx4Ix/q9YXgOQthK3WknI3wjlP9lbZPTZ05axv/d/W5kDDbzIcdpA1YIiSAnn2VQcKXfrcuu4+ejafckSRlyjdRXeESwQQrhNLXNev4/x48HRMyXm+WP6UEdIpTXCaBbrw0Avr/YUBXBMHTbvpHrMESs9YS1O0B4ct3JSgxuYTarPX5ZhqTqH3nt/GXBsQLgFKg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=corigine.com; dmarc=pass action=none header.from=corigine.com; dkim=pass header.d=corigine.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=corigine.onmicrosoft.com; s=selector2-corigine-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=TYo1ps100S5z4jqu7CL7/ub9AInvZ68bHuZZmd4m4kU=; b=Kl8SKjKn1NBCthS+Iskw6OHVZ5PUF95C3hZO2IYLASEoN4qUt+3VV4ZJQpsKAByHwrh4D0YzffJtvgT+D7cry0VNz6RUyB5CPiCTo757YThBNjrgd3rImWW/gAu/5pcH7PruM21weLAouD838oRUU7mhtN6fereFVnaIDl3wkcg= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=corigine.com; Received: from SJ0PR13MB5545.namprd13.prod.outlook.com (2603:10b6:a03:424::5) by SA1PR13MB4893.namprd13.prod.outlook.com (2603:10b6:806:18a::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5676.14; Mon, 26 Sep 2022 07:00:45 +0000 Received: from SJ0PR13MB5545.namprd13.prod.outlook.com ([fe80::779:2722:a8e5:503b]) by SJ0PR13MB5545.namprd13.prod.outlook.com ([fe80::779:2722:a8e5:503b%7]) with mapi id 15.20.5676.014; Mon, 26 Sep 2022 07:00:45 +0000 From: Chaoyong He To: dev@dpdk.org Cc: oss-drivers@corigine.com, niklas.soderlund@corigine.com, Chaoyong He Subject: [PATCH v10 13/13] net/nfp: add the representor port rxtx logic Date: Mon, 26 Sep 2022 14:59:57 +0800 Message-Id: <1664175597-37248-14-git-send-email-chaoyong.he@corigine.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1664175597-37248-1-git-send-email-chaoyong.he@corigine.com> References: <1664175597-37248-1-git-send-email-chaoyong.he@corigine.com> X-ClientProxiedBy: SG2P153CA0005.APCP153.PROD.OUTLOOK.COM (2603:1096::15) To SJ0PR13MB5545.namprd13.prod.outlook.com (2603:10b6:a03:424::5) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SJ0PR13MB5545:EE_|SA1PR13MB4893:EE_ X-MS-Office365-Filtering-Correlation-Id: 89a04c77-041a-4b86-9f77-08da9f8cd576 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: yRR4ipNGlN9o8jf/F9CihO5kn8LItaX8W6u0beuV/IgxgMKO0RIYet9rGCyPl5qYz/tdakrMKxjTm1G6xrQhFrEOmkviwdu9mU5YeBAh3Ewwmy5PZDFYBu1jNHkhB0WpKd6ENLIEKd1VdgJmoei4DynE/PVb8LSUAYkrUXJ/O4fZhdM2iUtt2zdJZS4h3ayxrysz/7FiDoLB3ZYiNM/NOwYYrAis2Had5HFxq2NNYJufb0Eho34B5ezRG8jkdm+qKi5or14dbTdn39+//BAb43G1SPGEyAlpetmMFXH+uRamlqvPLhVyx2rZ6Nd0KN19sGLv3gl8aOjlGEwFIWCHIUGsLyqtEdRITemIIwx52TBzttL+HECNIiFpvVHvg6iZ8GlaxjnKsHzMLgeGehWZGnWJOUQ1i5ut+fRdz8hShZvbTMlvthXfevN4d1MRpe3C8fMJRbJ/RaGNPGiQWEsFaJdsNFimvWuevDy5BF3AmYNifPqkkIxs5TD7znjDqY0PZASHZJSvOqDO+YmLUfd8qG403bONXE7t2YQ+m1b3o3/oylLILSJviZe8pi7PDkkJjRwYqmNwtWj+tlBvCFgiduEKX8AbJGVE6Gz3jlDVUBwa4LnSoGp3yLhbbILvNshRjifbnzEfs4Xx4vagSfvXMhEhKQu0Wihh4kWEbmZWvTYr0+dgnNg67W0SNVWB9TaX4hWGEX6u92x92zcHbg7zQF1UfuvRzWNiZvKViRfZ7cyJZePP39DoDtyfV1llkgvmlM/rlb1sE9EOlFgaIosakQ== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SJ0PR13MB5545.namprd13.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230022)(4636009)(366004)(396003)(39830400003)(136003)(376002)(346002)(451199015)(36756003)(38100700002)(38350700002)(2906002)(86362001)(44832011)(66946007)(8936002)(66556008)(66476007)(5660300002)(8676002)(4326008)(186003)(6512007)(6916009)(316002)(52116002)(6666004)(2616005)(26005)(6506007)(478600001)(41300700001)(107886003)(6486002); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?2RuNivwiWL+2MNIw9kRWkVozl85p?= =?utf-8?q?StMN91tfl+18NVZB65bWtbCS5XoOGpvcthn8dyB0yxu6YrUs+x4ViLs7lIH6ZSPBW?= =?utf-8?q?oWTRMgABT6Pq6Na1ySJ2hq7sJGZ7RrebfmriKMcPTuf5VWWo/W9Ga2pciannxWaoq?= =?utf-8?q?++cI3Y+VldC4me118I002G3YFkhN73+MF4liHq0c19DQ2PaPPuQGY1DhPK8X1P2R8?= =?utf-8?q?mSCrQe5CmPI6/GhdbgbRDB4K13wqhVsy54wL0OAlXNAskIVqn9xXvd8t81FkjXiWQ?= =?utf-8?q?/1AlQUkqjhKxPJns43kNJMs7uh25pCHNqlLWJNj+cnCVknbjaZ9wEvs3fsuz3Fd7j?= =?utf-8?q?S3ms7auZhTPrywYthKIRDfS58s/Ydp44dwH/qozmDrfZVZECd0iiAsYsdcFS8pk3u?= =?utf-8?q?K8WCNEirbWTz2UGM5a0M7hYXn1BXtQvVSdYYauJA/WJfaTa5nkDPkZ0sXyzFNkYpO?= =?utf-8?q?dXyoxpA1hPV534T5TGiACSCWZdaUA0JB9L2v/Lfgh4AMZv0anUnpCHo9eUizmzJaw?= =?utf-8?q?g/AJt1qBgLTSZUpZwIoXNMVXQQirfdM3FjKEmROd1ny2yVOhMxN/VlkH+7cptBI16?= =?utf-8?q?TkQnbOTUgkhhYeQLrQeK1QqG4yEb1ppWqNxCBVe1SoofW/NIsLLlg3+P2/NTwO2fu?= =?utf-8?q?4lCADQbKmzMqmLRG8V4xtgtukPV8aT4ECI8+6MsPDRzQfMNIJv73ORngEYHQrYQW4?= =?utf-8?q?FMfUYhvaZlArcnA4/Dm/zexREA5lctNB4CNuLt7Au5J6jnSDygvVZ76rfI5PhYqFn?= =?utf-8?q?90s22MRkZb4ON9n279xwD+4PXbW+SGBM3pFqaWEXdIn5fNiv0UBuOG/3hzxJNApIJ?= =?utf-8?q?PJxNPDXX+Y/+NA5QMB1gEp7L1t7gkfdDulBEHe/3UmjLwNlXlMia0RHMsM/xqEuxE?= =?utf-8?q?Xasdfp3Z2O6+9PuKHu30urtpzXtuChIJdsuSBzSFHOqcNVUbYn0vzS+13hTjR5f5x?= =?utf-8?q?F9sJgGQi/U5ADgQ7Z8ZdWYxlQCtltoXap7lERtb2WCTxoADsSf4xVW3tn4lIMgK9M?= =?utf-8?q?49F4qc+tbcIawgrFTctZMpV8zcyA2G8O3y0JRmliQ1iTVnCnT7Lsg98p+HwfrDqcX?= =?utf-8?q?C726B5FVmlcNl3BH3zeYjQwbURd+aRl7k1zmhsyfk/AU3jcdJJjhFIfFwE+YFIWmI?= =?utf-8?q?j+S756YRT2hug4sLXeuL9COqR/Y0xwjQD7kU29/RqhGqk8IKcq5kl4OtNOcTohJGk?= =?utf-8?q?kxMNJZ79jH5+F2Pn200h1xhsCyqaEVC+RVOfqeQtHRVvGtaRGRtw4BtAGM1I+5Lxh?= =?utf-8?q?cLjp0y5b0Mq8J5UngliSlod4DNw6QAxOO3Gp/Ua/Cx4E3TXKwl7Hir7jr7gE74O3+?= =?utf-8?q?Yyo4cI7xHjvdxBNjRzkjgtjrh3WCSs5vOmN4Dgyj9lw1KisBLBv+f01gCYphL8rVB?= =?utf-8?q?l9/QzzdVOqm9O+I7YNVBz6ii2F1gtnkWuBOgcL55dkORywZDl8OKdiOqEWu+ueJww?= =?utf-8?q?ozZF2qRcdAAzuu2zoyAYuf2utRhfA5SQ9ozIxeeOpCUBrq2Nlpz33qFsX/sjzqYS+?= =?utf-8?q?ceDZ4YvimOMwLy1Lu8N6peqQyXb3D/PBuA=3D=3D?= X-OriginatorOrg: corigine.com X-MS-Exchange-CrossTenant-Network-Message-Id: 89a04c77-041a-4b86-9f77-08da9f8cd576 X-MS-Exchange-CrossTenant-AuthSource: SJ0PR13MB5545.namprd13.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 26 Sep 2022 07:00:45.4580 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: fe128f2c-073b-4c20-818e-7246a585940c X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 7lfyS8HwXBDfFgq1yw3WrfE7WvmrOJB8pDkPHz/jzAHjYpk1/U/AKGkMGqy1uPJCdi481Fh+C5SScGR20/SfV2qCgtIBad6fT9FfAkzSwxw= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR13MB4893 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org For the Rx logic, the representor port decap packet from the corresponding ring. For the Tx logic, the representor port prepend the metadata into packet, and send to firmware through the queue 0 of pf vNIC. Signed-off-by: Chaoyong He Reviewed-by: Niklas Söderlund --- drivers/net/nfp/flower/nfp_flower_representor.c | 90 +++++++++++++++++++++++++ 1 file changed, 90 insertions(+) diff --git a/drivers/net/nfp/flower/nfp_flower_representor.c b/drivers/net/nfp/flower/nfp_flower_representor.c index b47b0a5..0e60f50 100644 --- a/drivers/net/nfp/flower/nfp_flower_representor.c +++ b/drivers/net/nfp/flower/nfp_flower_representor.c @@ -464,6 +464,92 @@ return 0; } +static uint16_t +nfp_flower_repr_rx_burst(void *rx_queue, + struct rte_mbuf **rx_pkts, + uint16_t nb_pkts) +{ + unsigned int available = 0; + unsigned int total_dequeue; + struct nfp_net_rxq *rxq; + struct rte_eth_dev *dev; + struct nfp_flower_representor *repr; + + rxq = rx_queue; + if (unlikely(rxq == NULL)) { + PMD_RX_LOG(ERR, "RX Bad queue"); + return 0; + } + + dev = &rte_eth_devices[rxq->port_id]; + repr = dev->data->dev_private; + if (unlikely(repr->ring == NULL)) { + PMD_RX_LOG(ERR, "representor %s has no ring configured!", + repr->name); + return 0; + } + + total_dequeue = rte_ring_dequeue_burst(repr->ring, (void *)rx_pkts, + nb_pkts, &available); + if (total_dequeue != 0) { + PMD_RX_LOG(DEBUG, "Representor Rx burst for %s, port_id: 0x%x, " + "received: %u, available: %u", repr->name, + repr->port_id, total_dequeue, available); + + repr->repr_stats.ipackets += total_dequeue; + } + + return total_dequeue; +} + +static uint16_t +nfp_flower_repr_tx_burst(void *tx_queue, + struct rte_mbuf **tx_pkts, + uint16_t nb_pkts) +{ + uint16_t i; + uint16_t sent; + char *meta_offset; + void *pf_tx_queue; + struct nfp_net_txq *txq; + struct nfp_net_hw *pf_hw; + struct rte_eth_dev *dev; + struct rte_eth_dev *repr_dev; + struct nfp_flower_representor *repr; + + txq = tx_queue; + if (unlikely(txq == NULL)) { + PMD_TX_LOG(ERR, "TX Bad queue"); + return 0; + } + + /* Grab a handle to the representor struct */ + repr_dev = &rte_eth_devices[txq->port_id]; + repr = repr_dev->data->dev_private; + + for (i = 0; i < nb_pkts; i++) { + meta_offset = rte_pktmbuf_prepend(tx_pkts[i], FLOWER_PKT_DATA_OFFSET); + *(uint32_t *)meta_offset = rte_cpu_to_be_32(NFP_NET_META_PORTID); + meta_offset += 4; + *(uint32_t *)meta_offset = rte_cpu_to_be_32(repr->port_id); + } + + /* This points to the PF vNIC that owns this representor */ + pf_hw = txq->hw; + dev = pf_hw->eth_dev; + + /* Only using Tx queue 0 for now. */ + pf_tx_queue = dev->data->tx_queues[0]; + sent = nfp_flower_pf_xmit_pkts(pf_tx_queue, tx_pkts, nb_pkts); + if (sent != 0) { + PMD_TX_LOG(DEBUG, "Representor Tx burst for %s, port_id: 0x%x transmitted: %u", + repr->name, repr->port_id, sent); + repr->repr_stats.opackets += sent; + } + + return sent; +} + static const struct eth_dev_ops nfp_flower_pf_repr_dev_ops = { .dev_infos_get = nfp_flower_repr_dev_infos_get, @@ -553,6 +639,8 @@ snprintf(repr->name, sizeof(repr->name), "%s", init_repr_data->name); eth_dev->dev_ops = &nfp_flower_pf_repr_dev_ops; + eth_dev->rx_pkt_burst = nfp_flower_pf_recv_pkts; + eth_dev->tx_pkt_burst = nfp_flower_pf_xmit_pkts; eth_dev->data->dev_flags |= RTE_ETH_DEV_REPRESENTOR; eth_dev->data->representor_id = 0; @@ -622,6 +710,8 @@ snprintf(repr->name, sizeof(repr->name), "%s", init_repr_data->name); eth_dev->dev_ops = &nfp_flower_repr_dev_ops; + eth_dev->rx_pkt_burst = nfp_flower_repr_rx_burst; + eth_dev->tx_pkt_burst = nfp_flower_repr_tx_burst; eth_dev->data->dev_flags |= RTE_ETH_DEV_REPRESENTOR; if (repr->repr_type == NFP_REPR_TYPE_PHYS_PORT)