From patchwork Fri Jul 31 17:23:02 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ajit Khaparde X-Patchwork-Id: 75111 X-Patchwork-Delegate: ajit.khaparde@broadcom.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 16F46A052B; Fri, 31 Jul 2020 19:23:53 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 955251C0AD; Fri, 31 Jul 2020 19:23:23 +0200 (CEST) Received: from mail-ej1-f97.google.com (mail-ej1-f97.google.com [209.85.218.97]) by dpdk.org (Postfix) with ESMTP id DD4381C023 for ; Fri, 31 Jul 2020 19:23:16 +0200 (CEST) Received: by mail-ej1-f97.google.com with SMTP id kq25so19195638ejb.3 for ; Fri, 31 Jul 2020 10:23:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=WjIAlMDdrIKCwN6MLUuejyzcw25tgSn3/Hw9FQK/yiE=; b=M2BxArBLM1z/J0bQ7Dpfw/sv+Rhe01i4HZkwqmmSl5vOCnGo7/F7YdtP3ehORxVXS1 05UC/3YrJspbaNlQMszzHafSMsZ1ng/OheKyNmwCHFQkMKJIp6iAzbpmPx7bSUFlDupd YdM5Ll3lmnLuATi9HJYMTReJoq2Rv2wsPJ020= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=WjIAlMDdrIKCwN6MLUuejyzcw25tgSn3/Hw9FQK/yiE=; b=Jf4Xk2xyDhop4J+b/KLFKObflu+R2t4hxwqrTxrVHJ0dQ4GgZke3bxk6z8x2M2iSI4 yW7nT9GLR59K9uuC2cP5D2H2D52SHhmTYpDyL8Mv/0nxbLYl9zUx3IFLLJuNRHuIFldq k6K9EI0z/We85FhU5y8siCjmtuu9Gb5SLyPe7wRxjxJ/ijPM2t7dUDe9/dTCDZmYb8nI VHxnrpIti4aYweo/6hZ9vpvhmbyw/7gvaFIRa28B0hAKKEASbJ6ndwwjkasOYUCK5KMX 62FDuclRxtUgiCuXFopilK4p0DMV3/eaXzPka8Sur6jAkiIS2R2RVpJ0lbsrO9NhHBND Ri8A== X-Gm-Message-State: AOAM531m8Y7jZgWDlROvXCRwUje2DnmtOHRu7AgHnnmLPNkPZJdz9W3J HYl9pdKob0AsTlCt1LH8Omg4McWMEDNJg+0XNqdRtc1c90odAs53IBomhqdW7P0mDVtfNqeuk5V QeJ2U9kZR8kJ9C7JpDAt5BmycNp3NykA1BBisLLs8HoBwahrryY8oLfDtQU//wJJDn4gXznlNwA vxlA== X-Google-Smtp-Source: ABdhPJykX8AK6B9lWuF8vjRbtzr49K9k8WaNwpINTqZq99gKI+zsYXH+KuucxUlJSmAQsJ/GnyV7k51PT700 X-Received: by 2002:a17:906:4bc5:: with SMTP id x5mr5302492ejv.177.1596216196385; Fri, 31 Jul 2020 10:23:16 -0700 (PDT) Received: from localhost.localdomain ([192.19.223.252]) by smtp-relay.gmail.com with ESMTPS id d2sm75344ejm.71.2020.07.31.10.23.14 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 31 Jul 2020 10:23:16 -0700 (PDT) X-Relaying-Domain: broadcom.com From: Ajit Khaparde To: dev@dpdk.org Cc: ferruh.yigit@intel.com, Venkat Duvvuru , Somnath Kotur Date: Fri, 31 Jul 2020 10:23:02 -0700 Message-Id: <20200731172302.5292-5-ajit.khaparde@broadcom.com> X-Mailer: git-send-email 2.21.1 (Apple Git-122.3) In-Reply-To: <20200731172302.5292-1-ajit.khaparde@broadcom.com> References: <20200731172302.5292-1-ajit.khaparde@broadcom.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v2 4/4] net/bnxt: fix vfrep add when VF interface is down X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Venkat Duvvuru While adding vfrep port to OVS bridge, vnic & svif information of vfrep's endpoint(VF) would be needed to program default flow rules. However, if the endpoint interface is down when vfrep port is added, firmware will return invalid vnic & svif information. This patch fixes the problem by registering to DEFAULT_VNIC_CHANGE async event and once the async event is received, use the endpoint information(VF's fid) to fetch it's vnic & svif information and program the default flow rules. Fixes: 322bd6e70272 ("net/bnxt: add port representor infrastructure") Signed-off-by: Venkat Duvvuru Reviewed-by: Somnath Kotur --- drivers/net/bnxt/bnxt.h | 21 ++++++++++ drivers/net/bnxt/bnxt_cpr.c | 51 +++++++++++++++++++++++++ drivers/net/bnxt/bnxt_ethdev.c | 12 +++++- drivers/net/bnxt/bnxt_hwrm.c | 4 ++ drivers/net/bnxt/bnxt_hwrm.h | 2 + drivers/net/bnxt/bnxt_reps.c | 70 +++++++++++++++++++++++++--------- 6 files changed, 140 insertions(+), 20 deletions(-) diff --git a/drivers/net/bnxt/bnxt.h b/drivers/net/bnxt/bnxt.h index f4b2a3f92..74e2c9a70 100644 --- a/drivers/net/bnxt/bnxt.h +++ b/drivers/net/bnxt/bnxt.h @@ -147,6 +147,23 @@ #define BNXT_CMPL_AGGR_DMA_TMR_DURING_INT 50 #define BNXT_NUM_CMPL_DMA_AGGR_DURING_INT 12 +#define BNXT_DEFAULT_VNIC_STATE_MASK \ + HWRM_ASYNC_EVENT_CMPL_DEFAULT_VNIC_CHANGE_EVENT_DATA1_DEF_VNIC_STATE_MASK +#define BNXT_DEFAULT_VNIC_STATE_SFT \ + HWRM_ASYNC_EVENT_CMPL_DEFAULT_VNIC_CHANGE_EVENT_DATA1_DEF_VNIC_STATE_SFT +#define BNXT_DEFAULT_VNIC_ALLOC \ + HWRM_ASYNC_EVENT_CMPL_DEFAULT_VNIC_CHANGE_EVENT_DATA1_DEF_VNIC_STATE_DEF_VNIC_ALLOC +#define BNXT_DEFAULT_VNIC_FREE \ + HWRM_ASYNC_EVENT_CMPL_DEFAULT_VNIC_CHANGE_EVENT_DATA1_DEF_VNIC_STATE_DEF_VNIC_FREE +#define BNXT_DEFAULT_VNIC_CHANGE_PF_ID_MASK \ + HWRM_ASYNC_EVENT_CMPL_DEFAULT_VNIC_CHANGE_EVENT_DATA1_PF_ID_MASK +#define BNXT_DEFAULT_VNIC_CHANGE_PF_ID_SFT \ + HWRM_ASYNC_EVENT_CMPL_DEFAULT_VNIC_CHANGE_EVENT_DATA1_PF_ID_SFT +#define BNXT_DEFAULT_VNIC_CHANGE_VF_ID_MASK \ + HWRM_ASYNC_EVENT_CMPL_DEFAULT_VNIC_CHANGE_EVENT_DATA1_VF_ID_MASK +#define BNXT_DEFAULT_VNIC_CHANGE_VF_ID_SFT \ + HWRM_ASYNC_EVENT_CMPL_DEFAULT_VNIC_CHANGE_EVENT_DATA1_VF_ID_SFT + struct bnxt_led_info { uint8_t num_leds; uint8_t led_id; @@ -498,6 +515,8 @@ struct bnxt_mark_info { struct bnxt_rep_info { struct rte_eth_dev *vfr_eth_dev; pthread_mutex_t vfr_lock; + pthread_mutex_t vfr_start_lock; + bool conduit_valid; }; /* address space location of register */ @@ -796,6 +815,7 @@ struct bnxt_vf_representor { uint16_t switch_domain_id; uint16_t vf_id; uint16_t fw_fid; +#define BNXT_DFLT_VNIC_ID_INVALID 0xFFFF uint16_t dflt_vnic_id; uint16_t svif; uint16_t vfr_tx_cfa_action; @@ -884,6 +904,7 @@ uint16_t bnxt_get_phy_port_id(uint16_t port); uint16_t bnxt_get_vport(uint16_t port); enum bnxt_ulp_intf_type bnxt_get_interface_type(uint16_t port); +int bnxt_vf_rep_dev_start_op(struct rte_eth_dev *eth_dev); void bnxt_cancel_fc_thread(struct bnxt *bp); void bnxt_flow_cnt_alarm_cb(void *arg); diff --git a/drivers/net/bnxt/bnxt_cpr.c b/drivers/net/bnxt/bnxt_cpr.c index 40e5350f6..464ca8b6f 100644 --- a/drivers/net/bnxt/bnxt_cpr.c +++ b/drivers/net/bnxt/bnxt_cpr.c @@ -46,6 +46,54 @@ void bnxt_wait_for_device_shutdown(struct bnxt *bp) } while (timeout); } +static void +bnxt_process_default_vnic_change(struct bnxt *bp, + struct hwrm_async_event_cmpl *async_cmp) +{ + uint16_t fid, vnic_state, parent_id, vf_fid, vf_id; + struct bnxt_vf_representor *vf_rep_bp; + struct rte_eth_dev *eth_dev; + bool vfr_found = false; + uint32_t event_data; + + if (!BNXT_TRUFLOW_EN(bp)) + return; + + PMD_DRV_LOG(INFO, "Default vnic change async event received\n"); + event_data = rte_le_to_cpu_32(async_cmp->event_data1); + + vnic_state = (event_data & BNXT_DEFAULT_VNIC_STATE_MASK) >> + BNXT_DEFAULT_VNIC_STATE_SFT; + if (vnic_state != BNXT_DEFAULT_VNIC_ALLOC) + return; + + parent_id = (event_data & BNXT_DEFAULT_VNIC_CHANGE_PF_ID_MASK) >> + BNXT_DEFAULT_VNIC_CHANGE_PF_ID_SFT; + fid = BNXT_PF(bp) ? bp->fw_fid : bp->parent->fid; + if (parent_id != fid || !bp->rep_info) + return; + + vf_fid = (event_data & BNXT_DEFAULT_VNIC_CHANGE_VF_ID_MASK) >> + BNXT_DEFAULT_VNIC_CHANGE_VF_ID_SFT; + PMD_DRV_LOG(INFO, "async event received vf_id 0x%x\n", vf_fid); + + for (vf_id = 0; vf_id < BNXT_MAX_VF_REPS; vf_id++) { + eth_dev = bp->rep_info[vf_id].vfr_eth_dev; + if (!eth_dev) + continue; + vf_rep_bp = eth_dev->data->dev_private; + if (vf_rep_bp && + vf_rep_bp->fw_fid == vf_fid) { + vfr_found = true; + break; + } + } + if (!vfr_found) + return; + + bnxt_vf_rep_dev_start_op(eth_dev); +} + /* * Async event handling */ @@ -144,6 +192,9 @@ void bnxt_handle_async_event(struct bnxt *bp, rte_le_to_cpu_32(async_cmp->event_data1), rte_le_to_cpu_32(async_cmp->event_data2)); break; + case HWRM_ASYNC_EVENT_CMPL_EVENT_ID_DEFAULT_VNIC_CHANGE: + bnxt_process_default_vnic_change(bp, async_cmp); + break; default: PMD_DRV_LOG(DEBUG, "handle_async_event id = 0x%x\n", event_id); break; diff --git a/drivers/net/bnxt/bnxt_ethdev.c b/drivers/net/bnxt/bnxt_ethdev.c index 510a0d9e0..c2dda072b 100644 --- a/drivers/net/bnxt/bnxt_ethdev.c +++ b/drivers/net/bnxt/bnxt_ethdev.c @@ -5833,8 +5833,10 @@ bnxt_uninit_locks(struct bnxt *bp) { pthread_mutex_destroy(&bp->flow_lock); pthread_mutex_destroy(&bp->def_cp_lock); - if (bp->rep_info) + if (bp->rep_info) { pthread_mutex_destroy(&bp->rep_info->vfr_lock); + pthread_mutex_destroy(&bp->rep_info->vfr_start_lock); + } } static int @@ -5937,6 +5939,14 @@ static int bnxt_init_rep_info(struct bnxt *bp) bnxt_free_rep_info(bp); return rc; } + + rc = pthread_mutex_init(&bp->rep_info->vfr_start_lock, NULL); + if (rc) { + PMD_DRV_LOG(ERR, "Unable to initialize vfr_start_lock\n"); + bnxt_free_rep_info(bp); + return rc; + } + return rc; } diff --git a/drivers/net/bnxt/bnxt_hwrm.c b/drivers/net/bnxt/bnxt_hwrm.c index 8296d1d44..b0a0aab9e 100644 --- a/drivers/net/bnxt/bnxt_hwrm.c +++ b/drivers/net/bnxt/bnxt_hwrm.c @@ -918,6 +918,10 @@ int bnxt_hwrm_func_driver_register(struct bnxt *bp) req.async_event_fwd[1] |= rte_cpu_to_le_32(ASYNC_CMPL_EVENT_ID_DBG_NOTIFICATION); + if (BNXT_VF_IS_TRUSTED(bp)) + req.async_event_fwd[1] |= + rte_cpu_to_le_32(ASYNC_CMPL_EVENT_ID_DEFAULT_VNIC_CHANGE); + rc = bnxt_hwrm_send_message(bp, &req, sizeof(req), BNXT_USE_CHIMP_MB); HWRM_CHECK_RESULT(); diff --git a/drivers/net/bnxt/bnxt_hwrm.h b/drivers/net/bnxt/bnxt_hwrm.h index 4a2af13c9..f55b6f91d 100644 --- a/drivers/net/bnxt/bnxt_hwrm.h +++ b/drivers/net/bnxt/bnxt_hwrm.h @@ -34,6 +34,8 @@ struct hwrm_func_qstats_output; (1 << (HWRM_ASYNC_EVENT_CMPL_EVENT_ID_VF_CFG_CHANGE - 32)) #define ASYNC_CMPL_EVENT_ID_DBG_NOTIFICATION \ (1 << (HWRM_ASYNC_EVENT_CMPL_EVENT_ID_DEBUG_NOTIFICATION - 32)) +#define ASYNC_CMPL_EVENT_ID_DEFAULT_VNIC_CHANGE \ + (1 << (HWRM_ASYNC_EVENT_CMPL_EVENT_ID_DEFAULT_VNIC_CHANGE - 32)) #define HWRM_QUEUE_SERVICE_PROFILE_LOSSY \ HWRM_QUEUE_QPORTCFG_OUTPUT_QUEUE_ID0_SERVICE_PROFILE_LOSSY diff --git a/drivers/net/bnxt/bnxt_reps.c b/drivers/net/bnxt/bnxt_reps.c index 2941aff7b..a1b2c4bf9 100644 --- a/drivers/net/bnxt/bnxt_reps.c +++ b/drivers/net/bnxt/bnxt_reps.c @@ -135,6 +135,32 @@ bnxt_vf_rep_tx_burst(void *tx_queue, return rc; } +static int +bnxt_get_dflt_vnic_svif(struct bnxt *bp, struct bnxt_vf_representor *vf_rep_bp) +{ + struct bnxt_rep_info *rep_info; + int rc; + + rc = bnxt_hwrm_get_dflt_vnic_svif(bp, vf_rep_bp->fw_fid, + &vf_rep_bp->dflt_vnic_id, + &vf_rep_bp->svif); + if (rc) { + PMD_DRV_LOG(ERR, "Failed to get default vnic id of VF\n"); + vf_rep_bp->dflt_vnic_id = BNXT_DFLT_VNIC_ID_INVALID; + vf_rep_bp->svif = BNXT_SVIF_INVALID; + } else { + PMD_DRV_LOG(INFO, "vf_rep->dflt_vnic_id = %d\n", + vf_rep_bp->dflt_vnic_id); + } + if (vf_rep_bp->dflt_vnic_id != BNXT_DFLT_VNIC_ID_INVALID && + vf_rep_bp->svif != BNXT_SVIF_INVALID) { + rep_info = &bp->rep_info[vf_rep_bp->vf_id]; + rep_info->conduit_valid = true; + } + + return rc; +} + int bnxt_vf_representor_init(struct rte_eth_dev *eth_dev, void *params) { struct bnxt_vf_representor *vf_rep_bp = eth_dev->data->dev_private; @@ -142,7 +168,6 @@ int bnxt_vf_representor_init(struct rte_eth_dev *eth_dev, void *params) (struct bnxt_vf_representor *)params; struct rte_eth_link *link; struct bnxt *parent_bp; - int rc = 0; vf_rep_bp->vf_id = rep_params->vf_id; vf_rep_bp->switch_domain_id = rep_params->switch_domain_id; @@ -172,17 +197,6 @@ int bnxt_vf_representor_init(struct rte_eth_dev *eth_dev, void *params) eth_dev->data->dev_link.link_status = link->link_status; eth_dev->data->dev_link.link_autoneg = link->link_autoneg; - vf_rep_bp->fw_fid = rep_params->vf_id + parent_bp->first_vf_id; - PMD_DRV_LOG(INFO, "vf_rep->fw_fid = %d\n", vf_rep_bp->fw_fid); - rc = bnxt_hwrm_get_dflt_vnic_svif(parent_bp, vf_rep_bp->fw_fid, - &vf_rep_bp->dflt_vnic_id, - &vf_rep_bp->svif); - if (rc) - PMD_DRV_LOG(ERR, "Failed to get default vnic id of VF\n"); - else - PMD_DRV_LOG(INFO, "vf_rep->dflt_vnic_id = %d\n", - vf_rep_bp->dflt_vnic_id); - PMD_DRV_LOG(INFO, "calling bnxt_print_link_info\n"); bnxt_print_link_info(eth_dev); @@ -194,6 +208,9 @@ int bnxt_vf_representor_init(struct rte_eth_dev *eth_dev, void *params) "Switch domain id %d: Representor Device %d init done\n", vf_rep_bp->switch_domain_id, vf_rep_bp->vf_id); + vf_rep_bp->fw_fid = rep_params->vf_id + parent_bp->first_vf_id; + PMD_DRV_LOG(INFO, "vf_rep->fw_fid = %d\n", vf_rep_bp->fw_fid); + return 0; } @@ -369,21 +386,36 @@ static void bnxt_vf_rep_free_rx_mbufs(struct bnxt_vf_representor *rep_bp) int bnxt_vf_rep_dev_start_op(struct rte_eth_dev *eth_dev) { struct bnxt_vf_representor *rep_bp = eth_dev->data->dev_private; + struct bnxt_rep_info *rep_info; + struct bnxt *parent_bp; int rc; - rc = bnxt_vfr_alloc(eth_dev); + parent_bp = rep_bp->parent_dev->data->dev_private; + rep_info = &parent_bp->rep_info[rep_bp->vf_id]; - if (!rc) { - eth_dev->rx_pkt_burst = &bnxt_vf_rep_rx_burst; - eth_dev->tx_pkt_burst = &bnxt_vf_rep_tx_burst; + pthread_mutex_lock(&rep_info->vfr_start_lock); + if (rep_info->conduit_valid) { + pthread_mutex_unlock(&rep_info->vfr_start_lock); + return 0; + } + rc = bnxt_get_dflt_vnic_svif(parent_bp, rep_bp); + if (rc || !rep_info->conduit_valid) { + pthread_mutex_unlock(&rep_info->vfr_start_lock); + return rc; + } + pthread_mutex_unlock(&rep_info->vfr_start_lock); - bnxt_vf_rep_link_update_op(eth_dev, 1); - } else { + rc = bnxt_vfr_alloc(eth_dev); + if (rc) { eth_dev->data->dev_link.link_status = 0; bnxt_vf_rep_free_rx_mbufs(rep_bp); + return rc; } + eth_dev->rx_pkt_burst = &bnxt_vf_rep_rx_burst; + eth_dev->tx_pkt_burst = &bnxt_vf_rep_tx_burst; + bnxt_vf_rep_link_update_op(eth_dev, 1); - return rc; + return 0; } static int bnxt_tf_vfr_free(struct bnxt_vf_representor *vfr)