From patchwork Mon Nov 4 20:27:47 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lance Richardson X-Patchwork-Id: 62395 Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 0B887A0352; Mon, 4 Nov 2019 21:28:29 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 671D74C8E; Mon, 4 Nov 2019 21:28:11 +0100 (CET) Received: from mail-wr1-f65.google.com (mail-wr1-f65.google.com [209.85.221.65]) by dpdk.org (Postfix) with ESMTP id 388D0322C for ; Mon, 4 Nov 2019 21:28:10 +0100 (CET) Received: by mail-wr1-f65.google.com with SMTP id o28so18634058wro.7 for ; Mon, 04 Nov 2019 12:28:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=uwdslJjRLdVn1N1xxSV2QD2YnG9fSmHaZLsjcuGS1vw=; b=NpzscMxShosk/HLyNWVBNx98ued71b82YxXMP7rsEf+Wpfx6ITU95qtSr8Zjv/PGF3 42HTGsUUIy1v7kEuO7hKj1pA42LkifkLMlrfkG0uPxoscEIMxy5Gcff93kGHcfoGgyRm adR6YXVL+xqvyN1btXyDvd2QgoFtBF3Wbk9LQ= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=uwdslJjRLdVn1N1xxSV2QD2YnG9fSmHaZLsjcuGS1vw=; b=PR+kgYiL/0Br6DntL8pVHIcH+wdR0rRdLCgfr6iL7Sk4SLdQ2CKdHLbmx43trwyCmY tMi6N/aDNAyoSVSBxgZFXQT/l1t2nV3hqzHQyznc2AOnFH1uLhjw/5qelKDyLHSqb7hU WgMPQh4PHdh5A2GqJeO508XOcB9ioh1qQR+aSJZoZ9zAIx3FlLGxOWqjKA57Z9i/Ce/V ujNuo01qVWbHD3C2KPaW7if3ZE7fantsF48pwbX0jmDO1TjDkV5BOuLKm47oZwVlWjea CZBg2BakGiRf16MwASpu4NnEJZ9iOlW6MP0rA1b0DpC/Ky/VoF2o3x/ofZSpzX115AdN A68g== X-Gm-Message-State: APjAAAVldT/pRk9LOV0NdC67wyjF+wr2KyK/wUEOmLQiuLfHN0+ozyva smUGgnkqobfNJkvjsICby3QUL7Qi61kE0R8iQ5wGYVO1FWAhu6yFhmWZehsFulEUEzu3J082nXl xv1mVeb/jNVzrrQB0BM6V7n5ckeuEkedDgsFSx10by1U8t3P4jZVl9GRRQ33l8xT0 X-Google-Smtp-Source: APXvYqz6122X5EJk13E7H79az7Q6mBQJ1D+jl+w4razNIh8/gHksFi5B6tns/ozpSPz0cWc7HhubpQ== X-Received: by 2002:a5d:4c83:: with SMTP id z3mr25181717wrs.92.1572899289447; Mon, 04 Nov 2019 12:28:09 -0800 (PST) Received: from lrichardson-VirtualBox.dhcp.broadcom.net ([192.19.231.250]) by smtp.gmail.com with ESMTPSA id i3sm17995365wrw.69.2019.11.04.12.28.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 04 Nov 2019 12:28:08 -0800 (PST) From: Lance Richardson To: dev@dpdk.org Cc: ajit.khaparde@broadcom.com, ferruh.yigit@intel.com, Lance Richardson , stable@dpdk.org Date: Mon, 4 Nov 2019 15:27:47 -0500 Message-Id: <20191104202747.12897-5-lance.richardson@broadcom.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191104202747.12897-1-lance.richardson@broadcom.com> References: <20191104202747.12897-1-lance.richardson@broadcom.com> Subject: [dpdk-dev] [PATCH 4/4] net/bnxt: fix rxq start/stop for Thor based NICs X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Controller-specific handling is required for Thor-based NICs when stopping or starting a receive queue, otherwise packet reception may not be reliably resumed when a stopped receive queue is restarted: - The VNIC default receive ring needs to be recomputed when a receive queue is stopped or started. - When stopping the last (or only) receive queue for a given VNIC, ensure that no packets can reach the default receive ring by temporarily setting the VNIC MRU to zero. Fixes: f8168ca0e690 ("net/bnxt: support thor controller") Cc: stable@dpdk.org Signed-off-by: Lance Richardson Reviewed-by: Ajit Kumar Khaparde Signed-off-by: Ajit Khaparde --- drivers/net/bnxt/bnxt_hwrm.c | 27 +++++++++++++++++++++---- drivers/net/bnxt/bnxt_rxq.c | 38 +++++++++++++++++++++++++++++++++--- 2 files changed, 58 insertions(+), 7 deletions(-) diff --git a/drivers/net/bnxt/bnxt_hwrm.c b/drivers/net/bnxt/bnxt_hwrm.c index bb4dcf4b2..2cba007ea 100644 --- a/drivers/net/bnxt/bnxt_hwrm.c +++ b/drivers/net/bnxt/bnxt_hwrm.c @@ -1696,10 +1696,29 @@ int bnxt_hwrm_vnic_cfg(struct bnxt *bp, struct bnxt_vnic_info *vnic) HWRM_PREP(req, VNIC_CFG, BNXT_USE_CHIMP_MB); if (BNXT_CHIP_THOR(bp)) { - struct bnxt_rx_queue *rxq = - bp->eth_dev->data->rx_queues[vnic->start_grp_id]; - struct bnxt_rx_ring_info *rxr = rxq->rx_ring; - struct bnxt_cp_ring_info *cpr = rxq->cp_ring; + int dflt_rxq = vnic->start_grp_id; + struct bnxt_rx_ring_info *rxr; + struct bnxt_cp_ring_info *cpr; + struct bnxt_rx_queue *rxq; + int i; + + /* + * The first active receive ring is used as the VNIC + * default receive ring. If there are no active receive + * rings (all corresponding receive queues are stopped), + * the first receive ring is used. + */ + for (i = vnic->start_grp_id; i < vnic->end_grp_id; i++) { + rxq = bp->eth_dev->data->rx_queues[i]; + if (rxq->rx_started) { + dflt_rxq = i; + break; + } + } + + rxq = bp->eth_dev->data->rx_queues[dflt_rxq]; + rxr = rxq->rx_ring; + cpr = rxq->cp_ring; req.default_rx_ring_id = rte_cpu_to_le_16(rxr->rx_ring_struct->fw_ring_id); diff --git a/drivers/net/bnxt/bnxt_rxq.c b/drivers/net/bnxt/bnxt_rxq.c index e82eda9d7..e7c012f68 100644 --- a/drivers/net/bnxt/bnxt_rxq.c +++ b/drivers/net/bnxt/bnxt_rxq.c @@ -451,6 +451,10 @@ int bnxt_rx_queue_start(struct rte_eth_dev *dev, uint16_t rx_queue_id) if (rc) return rc; + if (BNXT_CHIP_THOR(bp)) { + /* Reconfigure default receive ring and MRU. */ + bnxt_hwrm_vnic_cfg(bp, rxq->vnic); + } PMD_DRV_LOG(INFO, "Rx queue started %d\n", rx_queue_id); if (dev_conf->rxmode.mq_mode & ETH_MQ_RX_RSS_FLAG) { @@ -491,7 +495,8 @@ int bnxt_rx_queue_stop(struct rte_eth_dev *dev, uint16_t rx_queue_id) struct rte_eth_conf *dev_conf = &bp->eth_dev->data->dev_conf; struct bnxt_vnic_info *vnic = NULL; struct bnxt_rx_queue *rxq = NULL; - int rc = 0; + int active_queue_cnt = 0; + int i, rc = 0; rc = is_bnxt_in_error(bp); if (rc) @@ -507,8 +512,9 @@ int bnxt_rx_queue_stop(struct rte_eth_dev *dev, uint16_t rx_queue_id) } rxq = bp->rx_queues[rx_queue_id]; + vnic = rxq->vnic; - if (rxq == NULL) { + if (!rxq || !vnic) { PMD_DRV_LOG(ERR, "Invalid Rx queue %d\n", rx_queue_id); return -EINVAL; } @@ -518,7 +524,6 @@ int bnxt_rx_queue_stop(struct rte_eth_dev *dev, uint16_t rx_queue_id) PMD_DRV_LOG(DEBUG, "Rx queue stopped\n"); if (dev_conf->rxmode.mq_mode & ETH_MQ_RX_RSS_FLAG) { - vnic = rxq->vnic; if (BNXT_HAS_RING_GRPS(bp)) vnic->fw_grp_ids[rx_queue_id] = INVALID_HW_RING_ID; @@ -526,6 +531,33 @@ int bnxt_rx_queue_stop(struct rte_eth_dev *dev, uint16_t rx_queue_id) rc = bnxt_vnic_rss_configure(bp, vnic); } + if (BNXT_CHIP_THOR(bp)) { + /* Compute current number of active receive queues. */ + for (i = vnic->start_grp_id; i < vnic->end_grp_id; i++) + if (bp->rx_queues[i]->rx_started) + active_queue_cnt++; + + /* + * For Thor, we need to ensure that the VNIC default receive + * ring corresponds to an active receive queue. When no queue + * is active, we need to temporarily set the MRU to zero so + * that packets are dropped early in the receive pipeline in + * order to prevent the VNIC default receive ring from being + * accessed. + */ + if (active_queue_cnt == 0) { + uint16_t saved_mru = vnic->mru; + + vnic->mru = 0; + /* Reconfigure default receive ring and MRU. */ + bnxt_hwrm_vnic_cfg(bp, vnic); + vnic->mru = saved_mru; + } else { + /* Reconfigure default receive ring. */ + bnxt_hwrm_vnic_cfg(bp, vnic); + } + } + if (rc == 0) bnxt_rx_queue_release_mbufs(rxq);