From patchwork Fri Sep 11 01:55:48 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ajit Khaparde X-Patchwork-Id: 77272 X-Patchwork-Delegate: ajit.khaparde@broadcom.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 4F92FA04B5; Fri, 11 Sep 2020 03:58:11 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 453DC1C13B; Fri, 11 Sep 2020 03:56:56 +0200 (CEST) Received: from mail-pj1-f98.google.com (mail-pj1-f98.google.com [209.85.216.98]) by dpdk.org (Postfix) with ESMTP id 15F3B1C13B for ; Fri, 11 Sep 2020 03:56:55 +0200 (CEST) Received: by mail-pj1-f98.google.com with SMTP id mm21so937212pjb.4 for ; Thu, 10 Sep 2020 18:56:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=G005tTvawCEfzEhkqcq+WBPU3/Lnn+p/S4nkWXadFTE=; b=WHwqphWDRqc/wB6UgJ4cv8PP8CBsqVR/9WwS9tJTW6WLAqy3tXOTub6+velp3bYnYn ZfkWObdFlunkvQZ9borN+n6JLmWiYdY8na9RMBLAF3QzfFdYb6eRY7Cmp/Cn+l/EdaZ4 0OxvwKFyP7k/YTzmMJdl9X6NyaL6o+niFBYnQ= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=G005tTvawCEfzEhkqcq+WBPU3/Lnn+p/S4nkWXadFTE=; b=Adl6KYUqH7mUYKLyui8cgSFZ4/X4jB98vkAqcK7AADYMY2tzjngy4X8pmbNqW65EHU LaKYsGWBZv53PMH+Rp1iD1+3+FuURnezmYUz8Ss3iKSxVS81cjmQT10D+W3Ir/CrPc60 nYLrfy0z6bWDUAtS6IveQVc3LLwUmgDmsSxXd+2J1RvezGO0KAo6B70+5c3yTemyKqf/ robTIpXMOCZ/gVj1QEizAYiLwAF7jfsvPZEtqVw3FTUnZL/En0EHxdK6Vrp2dSG6HAXL K7ui3eNMN0zkYK51LeKVGEC+aL8hNP45ccAo5iXBQg7a/V/dCBnEXAvUDqY/vQORBLqO h7ew== X-Gm-Message-State: AOAM530sl2kYHsjoz+zk4cROvNgXzrabHq8opXlJAEebZyzaTmCI6q+X NVOqxbxxexy37Tq1yzGzZBdChwsC76BFCvduI8MOZPwxCE4wNX5xrxwk9QGr2UDBtT+1u2eNiKH Duzki9cp0Sx7+giieGNhISSK/YtLupLjk67caDcyqO640lMUdkllsKqiB6bGz0paeTmEzmDjEuh hZ9g== X-Google-Smtp-Source: ABdhPJzExXP9ocm8RmpBXGMMAG5VDUUIftIsSAW8MeBEmOVtOSiatyD5vWFOxX9LC/VU9OLLyJ2abgwP/YFP X-Received: by 2002:a17:90a:ea08:: with SMTP id w8mr46556pjy.41.1599789414157; Thu, 10 Sep 2020 18:56:54 -0700 (PDT) Received: from localhost.localdomain ([192.19.223.252]) by smtp-relay.gmail.com with ESMTPS id cl6sm80986pjb.15.2020.09.10.18.56.51 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 10 Sep 2020 18:56:54 -0700 (PDT) X-Relaying-Domain: broadcom.com From: Ajit Khaparde To: dev@dpdk.org Cc: Kishore Padmanabha , Mike Baucom , Shahaji Bhosle Date: Thu, 10 Sep 2020 18:55:48 -0700 Message-Id: <20200911015603.88359-11-ajit.khaparde@broadcom.com> X-Mailer: git-send-email 2.21.1 (Apple Git-122.3) In-Reply-To: <20200911015603.88359-1-ajit.khaparde@broadcom.com> References: <20200911015603.88359-1-ajit.khaparde@broadcom.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH 10/25] net/bnxt: vfr port clean up during port stop X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Kishore Padmanabha When parent VF or PF ports are cleaned up, the child VF representor ports also need to be cleaned up. If not cleaned up, then deleting the parent VF shall result in not cleaning up the hardware rules and updating the firmware of vfr removal. The issue can occur even when OVS is exited without deleting vfr ports. Signed-off-by: Kishore Padmanabha Reviewed-by: Mike Baucom Reviewed-by: Shahaji Bhosle --- drivers/net/bnxt/bnxt_ethdev.c | 20 ++++++++++++++-- drivers/net/bnxt/bnxt_reps.c | 38 ++++++++++++++++++++++++++---- drivers/net/bnxt/bnxt_reps.h | 1 + drivers/net/bnxt/tf_ulp/bnxt_ulp.c | 4 ++-- 4 files changed, 55 insertions(+), 8 deletions(-) diff --git a/drivers/net/bnxt/bnxt_ethdev.c b/drivers/net/bnxt/bnxt_ethdev.c index bc9aeba08..ea81ff149 100644 --- a/drivers/net/bnxt/bnxt_ethdev.c +++ b/drivers/net/bnxt/bnxt_ethdev.c @@ -1307,6 +1307,9 @@ static void bnxt_dev_stop_op(struct rte_eth_dev *eth_dev) /* disable uio/vfio intr/eventfd mapping */ rte_intr_disable(intr_handle); + /* Stop the child representors for this device */ + bnxt_vf_rep_stop_all(bp); + /* delete the bnxt ULP port details */ bnxt_ulp_port_deinit(bp); @@ -3709,8 +3712,13 @@ bnxt_filter_ctrl_op(struct rte_eth_dev *dev, struct bnxt_vf_representor *vfr = dev->data->dev_private; bp = vfr->parent_dev->data->dev_private; /* parent is deleted while children are still valid */ - if (!bp) + if (!bp) { + PMD_DRV_LOG(DEBUG, "BNXT Port:%d VFR Error %d:%d\n", + dev->data->port_id, + filter_type, + filter_op); return -EIO; + } } ret = is_bnxt_in_error(bp); @@ -5912,8 +5920,12 @@ static int bnxt_pci_remove_dev_with_reps(struct rte_eth_dev *eth_dev) vf_rep_eth_dev = bp->rep_info[i].vfr_eth_dev; if (!vf_rep_eth_dev) continue; + PMD_DRV_LOG(DEBUG, "BNXT Port:%d VFR pci remove\n", + vf_rep_eth_dev->data->port_id); rte_eth_dev_destroy(vf_rep_eth_dev, bnxt_vf_representor_uninit); } + PMD_DRV_LOG(DEBUG, "BNXT Port:%d pci remove\n", + eth_dev->data->port_id); ret = rte_eth_dev_destroy(eth_dev, bnxt_dev_uninit); return ret; @@ -6040,6 +6052,8 @@ static int bnxt_rep_port_probe(struct rte_pci_device *pci_dev, ret = -ENODEV; return ret; } + PMD_DRV_LOG(DEBUG, "BNXT Port:%d VFR pci probe\n", + backing_eth_dev->data->port_id); backing_bp->rep_info[representor.vf_id].vfr_eth_dev = vf_rep_eth_dev; backing_bp->num_reps++; @@ -6088,7 +6102,8 @@ static int bnxt_pci_probe(struct rte_pci_driver *pci_drv __rte_unused, backing_eth_dev = rte_eth_dev_allocated(pci_dev->device.name); } - + PMD_DRV_LOG(DEBUG, "BNXT Port:%d pci probe\n", + backing_eth_dev->data->port_id); /* probe representor ports now */ ret = bnxt_rep_port_probe(pci_dev, eth_da, backing_eth_dev); @@ -6107,6 +6122,7 @@ static int bnxt_pci_remove(struct rte_pci_device *pci_dev) * +ve value will at least help in proper cleanup */ + PMD_DRV_LOG(DEBUG, "BNXT Port:%d pci remove\n", eth_dev->data->port_id); if (rte_eal_process_type() == RTE_PROC_PRIMARY) { if (eth_dev->data->dev_flags & RTE_ETH_DEV_REPRESENTOR) return rte_eth_dev_destroy(eth_dev, diff --git a/drivers/net/bnxt/bnxt_reps.c b/drivers/net/bnxt/bnxt_reps.c index 3101512c3..9f535c0ce 100644 --- a/drivers/net/bnxt/bnxt_reps.c +++ b/drivers/net/bnxt/bnxt_reps.c @@ -167,6 +167,7 @@ int bnxt_vf_representor_init(struct rte_eth_dev *eth_dev, void *params) struct rte_eth_link *link; struct bnxt *parent_bp; + PMD_DRV_LOG(DEBUG, "BNXT Port:%d VFR init\n", eth_dev->data->port_id); vf_rep_bp->vf_id = rep_params->vf_id; vf_rep_bp->switch_domain_id = rep_params->switch_domain_id; vf_rep_bp->parent_dev = rep_params->parent_dev; @@ -217,15 +218,18 @@ int bnxt_vf_representor_uninit(struct rte_eth_dev *eth_dev) struct bnxt *parent_bp; struct bnxt_vf_representor *rep = (struct bnxt_vf_representor *)eth_dev->data->dev_private; - uint16_t vf_id; + PMD_DRV_LOG(DEBUG, "BNXT Port:%d VFR uninit\n", eth_dev->data->port_id); eth_dev->data->mac_addrs = NULL; eth_dev->dev_ops = NULL; parent_bp = rep->parent_dev->data->dev_private; - if (!parent_bp) + if (!parent_bp) { + PMD_DRV_LOG(DEBUG, "BNXT Port:%d already freed\n", + eth_dev->data->port_id); return 0; + } parent_bp->num_reps--; vf_id = rep->vf_id; @@ -297,7 +301,8 @@ static int bnxt_tf_vfr_alloc(struct rte_eth_dev *vfr_ethdev) vfr->vf_id, rc); (void)bnxt_ulp_delete_vfr_default_rules(vfr); } - + BNXT_TF_DBG(DEBUG, "BNXT Port:%d VFR created and initialized\n", + vfr->dpdk_port_id); return rc; } @@ -362,6 +367,7 @@ int bnxt_vf_rep_dev_start_op(struct rte_eth_dev *eth_dev) parent_bp = rep_bp->parent_dev->data->dev_private; rep_info = &parent_bp->rep_info[rep_bp->vf_id]; + BNXT_TF_DBG(DEBUG, "BNXT Port:%d VFR start\n", eth_dev->data->port_id); pthread_mutex_lock(&rep_info->vfr_start_lock); if (!rep_info->conduit_valid) { rc = bnxt_get_dflt_vnic_svif(parent_bp, rep_bp); @@ -387,6 +393,7 @@ int bnxt_vf_rep_dev_start_op(struct rte_eth_dev *eth_dev) static int bnxt_tf_vfr_free(struct bnxt_vf_representor *vfr) { + BNXT_TF_DBG(DEBUG, "BNXT Port:%d VFR ulp free\n", vfr->dpdk_port_id); return bnxt_ulp_delete_vfr_default_rules(vfr); } @@ -402,8 +409,11 @@ static int bnxt_vfr_free(struct bnxt_vf_representor *vfr) } parent_bp = vfr->parent_dev->data->dev_private; - if (!parent_bp) + if (!parent_bp) { + PMD_DRV_LOG(DEBUG, "BNXT Port:%d VFR already freed\n", + vfr->dpdk_port_id); return 0; + } /* Check if representor has been already freed in FW */ if (!vfr->vfr_tx_cfa_action) @@ -433,6 +443,8 @@ void bnxt_vf_rep_dev_stop_op(struct rte_eth_dev *eth_dev) eth_dev->rx_pkt_burst = &bnxt_dummy_recv_pkts; eth_dev->tx_pkt_burst = &bnxt_dummy_xmit_pkts; + BNXT_TF_DBG(DEBUG, "BNXT Port:%d VFR stop\n", eth_dev->data->port_id); + bnxt_vfr_free(vfr_bp); if (eth_dev->data->dev_started) @@ -443,6 +455,7 @@ void bnxt_vf_rep_dev_stop_op(struct rte_eth_dev *eth_dev) void bnxt_vf_rep_dev_close_op(struct rte_eth_dev *eth_dev) { + BNXT_TF_DBG(DEBUG, "BNXT Port:%d VFR close\n", eth_dev->data->port_id); bnxt_vf_representor_uninit(eth_dev); } @@ -722,3 +735,20 @@ int bnxt_vf_rep_stats_reset_op(struct rte_eth_dev *eth_dev) } return 0; } + +void bnxt_vf_rep_stop_all(struct bnxt *bp) +{ + uint16_t vf_id; + struct rte_eth_dev *rep_eth_dev; + + /* No vfrep ports just exit */ + if (!bp->rep_info) + return; + + for (vf_id = 0; vf_id < BNXT_MAX_VF_REPS; vf_id++) { + rep_eth_dev = bp->rep_info[vf_id].vfr_eth_dev; + if (!rep_eth_dev) + continue; + bnxt_vf_rep_dev_stop_op(rep_eth_dev); + } +} diff --git a/drivers/net/bnxt/bnxt_reps.h b/drivers/net/bnxt/bnxt_reps.h index 418b95afc..d877b0823 100644 --- a/drivers/net/bnxt/bnxt_reps.h +++ b/drivers/net/bnxt/bnxt_reps.h @@ -42,4 +42,5 @@ void bnxt_vf_rep_dev_close_op(struct rte_eth_dev *eth_dev); int bnxt_vf_rep_stats_get_op(struct rte_eth_dev *eth_dev, struct rte_eth_stats *stats); int bnxt_vf_rep_stats_reset_op(struct rte_eth_dev *eth_dev); +void bnxt_vf_rep_stop_all(struct bnxt *bp); #endif /* _BNXT_REPS_H_ */ diff --git a/drivers/net/bnxt/tf_ulp/bnxt_ulp.c b/drivers/net/bnxt/tf_ulp/bnxt_ulp.c index 63d453ffb..272536473 100644 --- a/drivers/net/bnxt/tf_ulp/bnxt_ulp.c +++ b/drivers/net/bnxt/tf_ulp/bnxt_ulp.c @@ -905,7 +905,7 @@ bnxt_ulp_port_init(struct bnxt *bp) } /* create the default rules */ bnxt_ulp_create_df_rules(bp); - BNXT_TF_DBG(DEBUG, "ULP Port:%d created and initialized\n", + BNXT_TF_DBG(DEBUG, "BNXT Port:%d ULP port init\n", bp->eth_dev->data->port_id); return rc; @@ -940,7 +940,7 @@ bnxt_ulp_port_deinit(struct bnxt *bp) return; } - BNXT_TF_DBG(DEBUG, "ULP Port:%d destroyed\n", + BNXT_TF_DBG(DEBUG, "BNXT Port:%d ULP port deinit\n", bp->eth_dev->data->port_id); /* Get the session details */