From patchwork Thu Sep 15 10:44:22 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Chaoyong He X-Patchwork-Id: 116340 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 2B4A2A00C5; Thu, 15 Sep 2022 12:45:26 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id BAB5E42B6C; Thu, 15 Sep 2022 12:45:02 +0200 (CEST) Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2139.outbound.protection.outlook.com [40.107.220.139]) by mails.dpdk.org (Postfix) with ESMTP id E9258427F9 for ; Thu, 15 Sep 2022 12:45:00 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Y4x2hu1Bmh0V9ZmcLZ+wBp6cpTaaGAHdLP/L14kgXGJAOsiL9Rprd39holGUcoGoSYNb8nX+vBbZ0LyxuzF1Km5Y4n+ba8rqEdiRZQ51qrx6jDKA7tPY+ZlyR9y2ng7LuN6NQB0aMSBZWAa47zuv46g0p+iVs7NlonaDwCREQRVLlBlvhBaKJSmH88BeALlwC4YqZPYBIuCqqyIhC2zUg889zJHCe+cDXTv1IymUh8bzp+S7a3QAVdrjl3cL4yIvrQ2Uj4HE79LkciBpFbdgmNNt0mZRjUs0GNxnSbPootAhcMB7LqVoQWcsN+Fe1eldlKy8iXGGqga8vX9UaeMlxg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=5/r66El63F913JmN4r7GK3hVH73nAEp20IMYIghF9lM=; b=C+rnlnAlrcnqYjCQse3woGIQu5SdhZK1Qs01jEaBP+T3nSp3keR0pX1cN5rctLjLtCbEaKiUTMATURm/ebq8WxyIGaEdKMyNZvk06ra4WvbRoxpquHkePaQjX95ObMlFEwUTwH97t9MJQYmAeAPhsFPZMD+9rzV1Q1wiWcDVbizoHG3SZ97UgsjuDKF122Rg2ZOxZAPlolKACgcyUTSuDBBLdyZpGggoPxXP3Ncb1VxcH/eXALN82kqC/zbu9OV3QAy7K6WqPmBpesATM42yxmil0tf+3KcxINq5+iyPLLEOKarNkUGDralPWY0CSHJEt1yoTZ+AubnxNC5lummW7g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=corigine.com; dmarc=pass action=none header.from=corigine.com; dkim=pass header.d=corigine.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=corigine.onmicrosoft.com; s=selector2-corigine-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=5/r66El63F913JmN4r7GK3hVH73nAEp20IMYIghF9lM=; b=X/FcCE6drHbwdJd50udvQcHDMJ/ZajGaiyO19SZK/LiaxTD4wAeoy9oS5Mb6sbkVnyfKJp0nbFMzQAStzCNWJ1F7f5u2qzO3eZa0961Bb19q1m+NJmvpmZOvW4gAmiqUhHdX6SUNVinRkAJczkfZjGlIEEfVmqnJtNMKDEbRZE0= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=corigine.com; Received: from SJ0PR13MB5545.namprd13.prod.outlook.com (2603:10b6:a03:424::5) by SA0PR13MB4158.namprd13.prod.outlook.com (2603:10b6:806:9b::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5654.5; Thu, 15 Sep 2022 10:44:59 +0000 Received: from SJ0PR13MB5545.namprd13.prod.outlook.com ([fe80::819:38c:6513:bf15]) by SJ0PR13MB5545.namprd13.prod.outlook.com ([fe80::819:38c:6513:bf15%4]) with mapi id 15.20.5632.012; Thu, 15 Sep 2022 10:44:59 +0000 From: Chaoyong He To: dev@dpdk.org Cc: oss-drivers@corigine.com, niklas.soderlund@corigine.com, Chaoyong He Subject: [PATCH v9 05/12] net/nfp: add flower PF setup logic Date: Thu, 15 Sep 2022 18:44:22 +0800 Message-Id: <1663238669-12244-6-git-send-email-chaoyong.he@corigine.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1663238669-12244-1-git-send-email-chaoyong.he@corigine.com> References: <1663238669-12244-1-git-send-email-chaoyong.he@corigine.com> X-ClientProxiedBy: BYAPR11CA0071.namprd11.prod.outlook.com (2603:10b6:a03:80::48) To SJ0PR13MB5545.namprd13.prod.outlook.com (2603:10b6:a03:424::5) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SJ0PR13MB5545:EE_|SA0PR13MB4158:EE_ X-MS-Office365-Filtering-Correlation-Id: 979f61e2-e739-439f-1921-08da97075659 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: ssef3qQkLjiEFMG4jAtl7pQiEA5IpSXKheu+9RkJGpH8thvrJLlr73suEsTG3kTzsZyp0WKtsFfexBNB94vdqEkCSWWMHu8VFWZv3hdXUpwi6wKNmiUkx34oNwU+ob31uvpnzH1cHNSH/FpafxGCOD428xoqTjQwDWZ0TD3/IVSz2n2N2Kc3gjSZ6ggzZHbuY0DFKWuaRvKq98UXuU+dfDY2zcFiiNzyzbTQ50U/PdK+qXSxTL6NOi5+JuA1YTwK9vikhYbgV9T220Ui/FwXnPsOafURgQmBhyh2iJq3tWexSl/FSvGwLynEJ/EfMIfXTpRgQ9jCX9s10mtYcpey9u+FQWujg+O1/IzBR/ctP2jiup/LKBCdmxLrkyKjGQwAEDExM7TGAZfLPghXXT4bX8gaYUv76W64Tw8bU2wTvevK/9oJYEssQ+9ImSUwaIWz8meolsV+AnAt6CsPQ9NXS9SoCTow/bgTVYjBLnl5Nl3XRN/EzSa7Aca5Oj0p9hAICIYIxXR0jiYSky6hr47uLrzJj9aZVLgvIhN26Vm76ICE+4pZl2RryPrr1gSSPRVcdOIYpVHW/IpTTSGVKt2s6RbZ6bSZVoIeuzP1CYHjIP/dR/uCpX5Pw4OMWD89jbRxpAbAlRvqt6isiJ2Pw3u8eInIt5M7hEAtKneaip9FfPBR73PmKdIWj1fCcTNhvT+LmC2uYDpFH6a8BKBQuHjxmd3QqABgCew0hjlrkINrhKPz3uDuLkLXhOoz0pZmZy52QujdWUptlw9chmUy6c/vIA== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SJ0PR13MB5545.namprd13.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230022)(4636009)(396003)(346002)(366004)(376002)(39840400004)(136003)(451199015)(2616005)(66574015)(38350700002)(38100700002)(186003)(86362001)(83380400001)(107886003)(8676002)(66946007)(66476007)(4326008)(66556008)(44832011)(8936002)(41300700001)(2906002)(5660300002)(6506007)(478600001)(6666004)(316002)(30864003)(6512007)(26005)(6486002)(52116002)(6916009)(36756003); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?elCJ95xBv9R1dw0zlO8ymqeksZYw?= =?utf-8?q?40zgwTe3IY3NdIPWQ/sdyRNgh6+JI1iYd34dIX5r4t2rCAQIBX/DcaTkyjyr4Rrf/?= =?utf-8?q?QTVr4oGqK781MfCQY9hdarFGdcl3y+R+b674/SUhkVvq3Z4+PJEctxFoSdC7TcibG?= =?utf-8?q?yHfDInvcLz2fQc9KSYaH1S0HTvlZFS3ouEX4+yd9RBICan5Uas6Zk9m+Zeb7vmZcq?= =?utf-8?q?JpoS7dCnM4OCP3zw/S87buuvNLnS2t4j5n9gFXnw9WnreLKsaaGvsObuXAwPH7X+7?= =?utf-8?q?8cKbewCAtIkSYwqvp0DfigtNdZ6zwPLPJiOr5uenwcOeKjjIUHPuYzSIsT5tP4U6M?= =?utf-8?q?+rjQ+lTzCyfwhyw6pxrAlqgpoRH1/dyUsa/4suT1fqbMz266Lsh4RClIdC4oWmxFm?= =?utf-8?q?rXI4eeiQfLgKIT/o+l9SZEA7lIaLpb3YzGI7uVtCurWt//yZSWtQOxAWInv+t3jT+?= =?utf-8?q?lTSK+Aw2Z+NIN3INlyKw6Lq5qXQgiE9TduDZMCY0R2Db9/xqBEyL5oRtn3rS87aWs?= =?utf-8?q?axnMReqkPkXHt18Z5ydv7QRy6d4+k8MCp/DKqNMuT8DT2kv/F3FO5amZ/NwQXZPq4?= =?utf-8?q?AqUvBgS+MhDntTQ3jcLGWdOCPc1A05RrUBJIS+CxjIoq/X23EmzyTM4+ZwvIykPOY?= =?utf-8?q?munHRFdvZi+EuqN1riDw14fItWsO83dEvEg2FxlPmL+ZDxSPA5jlKq8xUn5RtZXwH?= =?utf-8?q?odg1EsA5PGgY36LxlPZ4BLnzlOxZ8Bp28TRpzu/n11RPdQfnHsV/Fd8tET6qlDFiU?= =?utf-8?q?DKBBt1OgIGSJbjpowGcVG8PcYVzpF7VkfL4A5XTVT1x5I+sR6L7fyzhoBViU26Q5t?= =?utf-8?q?+cUEAQGABzQKOo2A4mfwJ7UGEQP4a/NRBQMRXeyAEzD6X16auLqcsGgT4uj6AOKdf?= =?utf-8?q?2FNh5ETrw9Axz101B2Q6qvleS8+kutZdsUzxcrTcCqKmUaNnHHEKqOvvpCK5CNt1H?= =?utf-8?q?My7VEEytf/JtbqBkGyhJUJDSmODUbgmHxRtGH0EJB5HogMA9OI7waR8DMyjeArs0n?= =?utf-8?q?TG7BVFdlgCUB4gNC6aY5gYzFSQlpohreccGa9dMw2UQLRG1ttZRYgbVJ5kxGLw2ti?= =?utf-8?q?+VwqbgShZYSEpbODJcT23LXhDn0J7icmYadwUO3AIY4suS32mHBoYdLiazDEOv5c2?= =?utf-8?q?pzk7KYPmEOXrKSE8CSn+r+gGQn3TmKeYputgcxZ3phZF9pcGXH8zf683Yi4yb+Uk4?= =?utf-8?q?22RarjKpJbDCnT/99AEDYvtxSuVVE7HB6KnP8gkrRHAwLrvIuaT+WhYU2jAh7au2Z?= =?utf-8?q?dR3imOcWmuPczoz8TGXUCH0BR9DCi4GL9vQc/hdDF+tX4G46TS595kMojd65rTxqm?= =?utf-8?q?FIUoWEzSw0gd/o+rinBD10P2NusTK6311ymdrNeHsVt83kSkhrkGp8Srod2sZIdvD?= =?utf-8?q?Q/Ho0GdbOgAYtNc1X/5qBzWjcckEjTgNVyX/dS6PlXyoHtLWP6ImGDKDYZnKgLqnr?= =?utf-8?q?TQSENTapT6YBb53vWDfAngUKxUR7h+tbt4XGly0Ks7axEn7fiwaaMctyngLOXU3/t?= =?utf-8?q?aT78f7ZMWV7FjLC3vXSaKqiduQPqS13+1w=3D=3D?= X-OriginatorOrg: corigine.com X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA0PR13MB4158 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Adds the vNIC initialization logic for the flower PF vNIC. The flower firmware application exposes this vNIC for the purposes of fallback traffic in the switchdev use-case. Adds minimal dev_ops for this PF vNIC device. Because the device is being exposed externally to DPDK it needs to implements a minimal set of dev_ops. Signed-off-by: Chaoyong He Reviewed-by: Niklas Söderlund --- drivers/net/nfp/flower/nfp_flower.c | 369 +++++++++++++++++++++++++++++++++++- drivers/net/nfp/flower/nfp_flower.h | 8 + drivers/net/nfp/nfp_common.h | 3 + 3 files changed, 377 insertions(+), 3 deletions(-) diff --git a/drivers/net/nfp/flower/nfp_flower.c b/drivers/net/nfp/flower/nfp_flower.c index 87cb922..34e60f8 100644 --- a/drivers/net/nfp/flower/nfp_flower.c +++ b/drivers/net/nfp/flower/nfp_flower.c @@ -14,12 +14,312 @@ #include "../nfp_logs.h" #include "../nfp_ctrl.h" #include "../nfp_cpp_bridge.h" +#include "../nfp_rxtx.h" +#include "../nfpcore/nfp_mip.h" +#include "../nfpcore/nfp_rtsym.h" +#include "../nfpcore/nfp_nsp.h" #include "nfp_flower.h" +#define MAX_PKT_BURST 32 +#define MBUF_PRIV_SIZE 128 +#define MEMPOOL_CACHE_SIZE 512 +#define DEFAULT_FLBUF_SIZE 9216 + +#define PF_VNIC_NB_DESC 1024 + +static const struct rte_eth_rxconf rx_conf = { + .rx_free_thresh = DEFAULT_RX_FREE_THRESH, + .rx_drop_en = 1, +}; + +static const struct rte_eth_txconf tx_conf = { + .tx_thresh = { + .pthresh = DEFAULT_TX_PTHRESH, + .hthresh = DEFAULT_TX_HTHRESH, + .wthresh = DEFAULT_TX_WTHRESH, + }, + .tx_free_thresh = DEFAULT_TX_FREE_THRESH, +}; + +static const struct eth_dev_ops nfp_flower_pf_vnic_ops = { + .dev_infos_get = nfp_net_infos_get, +}; + +struct dp_packet { + struct rte_mbuf mbuf; + uint32_t source; +}; + +static void +nfp_flower_pf_mp_init(__rte_unused struct rte_mempool *mp, + __rte_unused void *opaque_arg, + void *packet, + __rte_unused unsigned int i) +{ + struct dp_packet *pkt = packet; + /* Indicate that this pkt is from DPDK */ + pkt->source = 3; +} + +static struct rte_mempool * +nfp_flower_pf_mp_create(void) +{ + uint32_t nb_mbufs; + unsigned int numa_node; + struct rte_mempool *pktmbuf_pool; + uint32_t n_rxd = PF_VNIC_NB_DESC; + uint32_t n_txd = PF_VNIC_NB_DESC; + + nb_mbufs = RTE_MAX(n_rxd + n_txd + MAX_PKT_BURST + MEMPOOL_CACHE_SIZE, 81920U); + + numa_node = rte_socket_id(); + pktmbuf_pool = rte_pktmbuf_pool_create("flower_pf_mbuf_pool", nb_mbufs, + MEMPOOL_CACHE_SIZE, MBUF_PRIV_SIZE, + RTE_MBUF_DEFAULT_BUF_SIZE, numa_node); + if (pktmbuf_pool == NULL) { + PMD_INIT_LOG(ERR, "Cannot init pf vnic mbuf pool"); + return NULL; + } + + rte_mempool_obj_iter(pktmbuf_pool, nfp_flower_pf_mp_init, NULL); + + return pktmbuf_pool; +} + +static int +nfp_flower_init_vnic_common(struct nfp_net_hw *hw, const char *vnic_type) +{ + uint32_t start_q; + uint64_t rx_bar_off; + uint64_t tx_bar_off; + const int stride = 4; + struct nfp_pf_dev *pf_dev; + struct rte_pci_device *pci_dev; + + pf_dev = hw->pf_dev; + pci_dev = hw->pf_dev->pci_dev; + + /* NFP can not handle DMA addresses requiring more than 40 bits */ + if (rte_mem_check_dma_mask(40)) { + PMD_INIT_LOG(ERR, "Device %s can not be used: restricted dma mask to 40 bits!\n", + pci_dev->device.name); + return -ENODEV; + }; + + hw->device_id = pci_dev->id.device_id; + hw->vendor_id = pci_dev->id.vendor_id; + hw->subsystem_device_id = pci_dev->id.subsystem_device_id; + hw->subsystem_vendor_id = pci_dev->id.subsystem_vendor_id; + + PMD_INIT_LOG(DEBUG, "%s vNIC ctrl bar: %p", vnic_type, hw->ctrl_bar); + + /* Read the number of available rx/tx queues from hardware */ + hw->max_rx_queues = nn_cfg_readl(hw, NFP_NET_CFG_MAX_RXRINGS); + hw->max_tx_queues = nn_cfg_readl(hw, NFP_NET_CFG_MAX_TXRINGS); + + /* Work out where in the BAR the queues start */ + start_q = nn_cfg_readl(hw, NFP_NET_CFG_START_TXQ); + tx_bar_off = (uint64_t)start_q * NFP_QCP_QUEUE_ADDR_SZ; + start_q = nn_cfg_readl(hw, NFP_NET_CFG_START_RXQ); + rx_bar_off = (uint64_t)start_q * NFP_QCP_QUEUE_ADDR_SZ; + + hw->tx_bar = pf_dev->hw_queues + tx_bar_off; + hw->rx_bar = pf_dev->hw_queues + rx_bar_off; + + /* Get some of the read-only fields from the config BAR */ + hw->ver = nn_cfg_readl(hw, NFP_NET_CFG_VERSION); + hw->cap = nn_cfg_readl(hw, NFP_NET_CFG_CAP); + hw->max_mtu = nn_cfg_readl(hw, NFP_NET_CFG_MAX_MTU); + /* Set the current MTU to the maximum supported */ + hw->mtu = hw->max_mtu; + hw->flbufsz = DEFAULT_FLBUF_SIZE; + + /* read the Rx offset configured from firmware */ + if (NFD_CFG_MAJOR_VERSION_of(hw->ver) < 2) + hw->rx_offset = NFP_NET_RX_OFFSET; + else + hw->rx_offset = nn_cfg_readl(hw, NFP_NET_CFG_RX_OFFSET_ADDR); + + hw->ctrl = 0; + hw->stride_rx = stride; + hw->stride_tx = stride; + + /* Reuse cfg queue setup function */ + nfp_net_cfg_queue_setup(hw); + + PMD_INIT_LOG(INFO, "%s vNIC max_rx_queues: %u, max_tx_queues: %u", + vnic_type, hw->max_rx_queues, hw->max_tx_queues); + + /* Initializing spinlock for reconfigs */ + rte_spinlock_init(&hw->reconfig_lock); + + return 0; +} + +static int +nfp_flower_init_pf_vnic(struct nfp_net_hw *hw) +{ + int ret; + uint16_t i; + uint16_t n_txq; + uint16_t n_rxq; + unsigned int numa_node; + struct rte_mempool *mp; + struct nfp_pf_dev *pf_dev; + struct rte_eth_dev *eth_dev; + struct nfp_app_fw_flower *app_fw_flower; + + static const struct rte_eth_conf port_conf = { + .rxmode = { + .mq_mode = RTE_ETH_MQ_RX_RSS, + .offloads = RTE_ETH_RX_OFFLOAD_CHECKSUM, + }, + }; + + /* Set up some pointers here for ease of use */ + pf_dev = hw->pf_dev; + app_fw_flower = NFP_PRIV_TO_APP_FW_FLOWER(pf_dev->app_fw_priv); + + /* + * Perform the "common" part of setting up a flower vNIC. + * Mostly reading configuration from hardware. + */ + ret = nfp_flower_init_vnic_common(hw, "pf_vnic"); + if (ret != 0) { + PMD_INIT_LOG(ERR, "Could not init pf vnic"); + return -EINVAL; + } + + hw->eth_dev = rte_eth_dev_allocate("nfp_pf_vnic"); + if (hw->eth_dev == NULL) { + PMD_INIT_LOG(ERR, "Could not allocate pf vnic"); + return -ENOMEM; + } + + /* Grab the pointer to the newly created rte_eth_dev here */ + eth_dev = hw->eth_dev; + + numa_node = rte_socket_id(); + + /* Create a mbuf pool for the PF */ + app_fw_flower->pf_pktmbuf_pool = nfp_flower_pf_mp_create(); + if (app_fw_flower->pf_pktmbuf_pool == NULL) { + PMD_INIT_LOG(ERR, "Could not create mempool for pf vnic"); + ret = -ENOMEM; + goto port_release; + } + + mp = app_fw_flower->pf_pktmbuf_pool; + + /* Add Rx/Tx functions */ + eth_dev->dev_ops = &nfp_flower_pf_vnic_ops; + + /* PF vNIC gets a random MAC */ + eth_dev->data->mac_addrs = rte_zmalloc("mac_addr", RTE_ETHER_ADDR_LEN, 0); + if (eth_dev->data->mac_addrs == NULL) { + PMD_INIT_LOG(ERR, "Could not allocate mac addr"); + ret = -ENOMEM; + goto mempool_cleanup; + } + + rte_eth_random_addr(eth_dev->data->mac_addrs->addr_bytes); + rte_eth_dev_probing_finish(eth_dev); + + /* Configure the PF device now */ + n_rxq = hw->max_rx_queues; + n_txq = hw->max_tx_queues; + memcpy(ð_dev->data->dev_conf, &port_conf, sizeof(struct rte_eth_conf)); + eth_dev->data->rx_queues = rte_zmalloc("ethdev->rx_queues", + sizeof(eth_dev->data->rx_queues[0]) * n_rxq, RTE_CACHE_LINE_SIZE); + if (eth_dev->data->rx_queues == NULL) { + PMD_INIT_LOG(ERR, "rte_zmalloc failed for PF vNIC rx queues"); + ret = -ENOMEM; + goto mac_cleanup; + } + + eth_dev->data->tx_queues = rte_zmalloc("ethdev->tx_queues", + sizeof(eth_dev->data->tx_queues[0]) * n_txq, RTE_CACHE_LINE_SIZE); + if (eth_dev->data->tx_queues == NULL) { + PMD_INIT_LOG(ERR, "rte_zmalloc failed for PF vNIC tx queues"); + ret = -ENOMEM; + goto rx_queue_free; + } + + /* Fill in some of the eth_dev fields */ + eth_dev->device = &pf_dev->pci_dev->device; + eth_dev->data->nb_tx_queues = n_rxq; + eth_dev->data->nb_rx_queues = n_txq; + eth_dev->data->dev_private = hw; + eth_dev->data->dev_configured = 1; + + /* Set up the Rx queues */ + for (i = 0; i < n_rxq; i++) { + ret = nfp_net_rx_queue_setup(eth_dev, i, PF_VNIC_NB_DESC, numa_node, + &rx_conf, mp); + if (ret != 0) { + PMD_INIT_LOG(ERR, "Configure flower PF vNIC Rx queue %d failed", i); + goto rx_queue_cleanup; + } + } + + /* Set up the Tx queues */ + for (i = 0; i < n_txq; i++) { + ret = nfp_net_nfd3_tx_queue_setup(eth_dev, i, PF_VNIC_NB_DESC, numa_node, + &tx_conf); + if (ret != 0) { + PMD_INIT_LOG(ERR, "Configure flower PF vNIC Tx queue %d failed", i); + goto tx_queue_cleanup; + } + } + + return 0; + +tx_queue_cleanup: + for (i = 0; i < n_txq; i++) + nfp_net_tx_queue_release(eth_dev, i); +rx_queue_cleanup: + for (i = 0; i < n_rxq; i++) + nfp_net_rx_queue_release(eth_dev, i); + rte_free(eth_dev->data->tx_queues); +rx_queue_free: + rte_free(eth_dev->data->rx_queues); +mac_cleanup: + rte_free(eth_dev->data->mac_addrs); +mempool_cleanup: + rte_mempool_free(mp); +port_release: + rte_eth_dev_release_port(hw->eth_dev); + + return ret; +} + +__rte_unused static void +nfp_flower_cleanup_pf_vnic(struct nfp_net_hw *hw) +{ + uint16_t i; + struct nfp_app_fw_flower *app_fw_flower; + + app_fw_flower = NFP_PRIV_TO_APP_FW_FLOWER(hw->pf_dev->app_fw_priv); + + for (i = 0; i < hw->max_tx_queues; i++) + nfp_net_tx_queue_release(hw->eth_dev, i); + + for (i = 0; i < hw->max_tx_queues; i++) + nfp_net_rx_queue_release(hw->eth_dev, i); + + rte_free(hw->eth_dev->data->tx_queues); + rte_free(hw->eth_dev->data->rx_queues); + rte_free(hw->eth_dev->data->mac_addrs); + rte_mempool_free(app_fw_flower->pf_pktmbuf_pool); + rte_eth_dev_release_port(hw->eth_dev); +} + int nfp_init_app_fw_flower(struct nfp_pf_dev *pf_dev) { + int ret; unsigned int numa_node; + struct nfp_net_hw *pf_hw; struct nfp_app_fw_flower *app_fw_flower; numa_node = rte_socket_id(); @@ -34,12 +334,75 @@ pf_dev->app_fw_priv = app_fw_flower; + /* Allocate memory for the PF AND ctrl vNIC here (hence the * 2) */ + pf_hw = rte_zmalloc_socket("nfp_pf_vnic", 2 * sizeof(struct nfp_net_adapter), + RTE_CACHE_LINE_SIZE, numa_node); + if (pf_hw == NULL) { + PMD_INIT_LOG(ERR, "Could not malloc nfp pf vnic"); + ret = -ENOMEM; + goto app_cleanup; + } + + /* Grab the number of physical ports present on hardware */ + app_fw_flower->nfp_eth_table = nfp_eth_read_ports(pf_dev->cpp); + if (app_fw_flower->nfp_eth_table == NULL) { + PMD_INIT_LOG(ERR, "error reading nfp ethernet table"); + ret = -EIO; + goto vnic_cleanup; + } + + /* Map the PF ctrl bar */ + pf_dev->ctrl_bar = nfp_rtsym_map(pf_dev->sym_tbl, "_pf0_net_bar0", + 32768, &pf_dev->ctrl_area); + if (pf_dev->ctrl_bar == NULL) { + PMD_INIT_LOG(ERR, "Cloud not map the PF vNIC ctrl bar"); + ret = -ENODEV; + goto eth_tbl_cleanup; + } + + /* Fill in the PF vNIC and populate app struct */ + app_fw_flower->pf_hw = pf_hw; + pf_hw->ctrl_bar = pf_dev->ctrl_bar; + pf_hw->pf_dev = pf_dev; + pf_hw->cpp = pf_dev->cpp; + + ret = nfp_flower_init_pf_vnic(app_fw_flower->pf_hw); + if (ret != 0) { + PMD_INIT_LOG(ERR, "Could not initialize flower PF vNIC"); + goto pf_cpp_area_cleanup; + } + return 0; + +pf_cpp_area_cleanup: + nfp_cpp_area_free(pf_dev->ctrl_area); +eth_tbl_cleanup: + free(app_fw_flower->nfp_eth_table); +vnic_cleanup: + rte_free(pf_hw); +app_cleanup: + rte_free(app_fw_flower); + + return ret; } int -nfp_secondary_init_app_fw_flower(__rte_unused struct nfp_cpp *cpp) +nfp_secondary_init_app_fw_flower(struct nfp_cpp *cpp) { - PMD_INIT_LOG(ERR, "Flower firmware not supported"); - return -ENOTSUP; + struct rte_eth_dev *eth_dev; + const char *port_name = "pf_vnic_eth_dev"; + + PMD_INIT_LOG(DEBUG, "Secondary attaching to port %s", port_name); + + eth_dev = rte_eth_dev_attach_secondary(port_name); + if (eth_dev == NULL) { + PMD_INIT_LOG(ERR, "Secondary process attach to port %s failed", port_name); + return -ENODEV; + } + + eth_dev->process_private = cpp; + eth_dev->dev_ops = &nfp_flower_pf_vnic_ops; + rte_eth_dev_probing_finish(eth_dev); + + return 0; } diff --git a/drivers/net/nfp/flower/nfp_flower.h b/drivers/net/nfp/flower/nfp_flower.h index 8b9ef95..981d88d 100644 --- a/drivers/net/nfp/flower/nfp_flower.h +++ b/drivers/net/nfp/flower/nfp_flower.h @@ -8,6 +8,14 @@ /* The flower application's private structure */ struct nfp_app_fw_flower { + /* Pointer to a mempool for the PF vNIC */ + struct rte_mempool *pf_pktmbuf_pool; + + /* Pointer to the PF vNIC */ + struct nfp_net_hw *pf_hw; + + /* the eth table as reported by firmware */ + struct nfp_eth_table *nfp_eth_table; }; int nfp_init_app_fw_flower(struct nfp_pf_dev *pf_dev); diff --git a/drivers/net/nfp/nfp_common.h b/drivers/net/nfp/nfp_common.h index cefe717..aa6fdd4 100644 --- a/drivers/net/nfp/nfp_common.h +++ b/drivers/net/nfp/nfp_common.h @@ -446,6 +446,9 @@ int nfp_net_rss_hash_conf_get(struct rte_eth_dev *dev, #define NFP_PRIV_TO_APP_FW_NIC(app_fw_priv)\ ((struct nfp_app_fw_nic *)app_fw_priv) +#define NFP_PRIV_TO_APP_FW_FLOWER(app_fw_priv)\ + ((struct nfp_app_fw_flower *)app_fw_priv) + #endif /* _NFP_COMMON_H_ */ /* * Local variables: