From patchwork Fri Sep 1 08:07:06 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adrien Mazarguil X-Patchwork-Id: 28250 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [IPv6:::1]) by dpdk.org (Postfix) with ESMTP id 666839B9B; Fri, 1 Sep 2017 10:08:37 +0200 (CEST) Received: from mail-wm0-f54.google.com (mail-wm0-f54.google.com [74.125.82.54]) by dpdk.org (Postfix) with ESMTP id ED47999EC for ; Fri, 1 Sep 2017 10:08:26 +0200 (CEST) Received: by mail-wm0-f54.google.com with SMTP id 187so3723834wmn.1 for ; Fri, 01 Sep 2017 01:08:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=6wind-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=ynuw3fCk7zMwNpaF5GtnfOGzodh7kEpOFIIA/EiBr0A=; b=Hm4ed1HpUeuIJBCMuHrnlxSHj+9v7Rgh3OuYXQt1iTAQkrDPhKbFikhBNa0u10MzKU KjQg8idvpsnc3iGKi9Ydx694h2mP+MjCZro2Ri+l6MZ+Au3mEnA/2urdyhoKmkpgRG8r cyKEdyN+nDo7uLNSkkXJUDiEVhcJKjIuvw+SdvFxCJxj6RsMoxd6Dv67C5vLofwWLsWS g4dA3ENZ2jOZcYIdYYkZPzJL84OS6SQaDqGEIak0UEL/naOw3sJWT8oE6Kvt2/5nIE6X pn0uK44fi/JQZmO91okSGVyYcivPwg86m5cUtbqT82vW98Db9cz1fDVE3saYpbY1VOCS zyEA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=ynuw3fCk7zMwNpaF5GtnfOGzodh7kEpOFIIA/EiBr0A=; b=YOtVRhq0jACn5JTZuDcmM0h+l4fYEelmLJo5kfA2f0rvWvQ+HMSfRcauxBcPveCpfk 9ZqQhSrXO7ZmTfEQKqeePTpEyKYR+UZnIaPJLOyochZwW4og+OwToD7JAUo4wRnH69oA FwzAblpoDGbwWRk+TC4oMdUr4rhBA2mspqn+QuLlYYRzA7UHBsnPOaftiy2apdb+K/iX qftvm9qumqGSwVYpFL1GXwpsHxGHU+Tb5+bVrT42L25PdFfiCk5/6bsehUH392mDTOH+ m3vndLsF5srHd33SRUTYJJq5QBd/qt721dKfcnlWAin2LD+kJ++21guy84BmdV9Kxjod H2TQ== X-Gm-Message-State: AHPjjUgCmd3e7BcvYssOP7RciILpRJs4APjhhN4eFq/tvZv3DVhocl5p TOMrGpApUiUGC6Kwhvo= X-Google-Smtp-Source: ADKCNb4Qxw5WjOOhLLZVYdRf4su0HZWo7H28sGsjsf1tvwgVDg7jTV8JrNonjMh1nLMYEGhUJl65qw== X-Received: by 10.28.109.24 with SMTP id i24mr670289wmc.68.1504253306195; Fri, 01 Sep 2017 01:08:26 -0700 (PDT) Received: from 6wind.com (host.78.145.23.62.rev.coltfrance.com. [62.23.145.78]) by smtp.gmail.com with ESMTPSA id b72sm7870546wmd.0.2017.09.01.01.08.25 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 01 Sep 2017 01:08:25 -0700 (PDT) From: Adrien Mazarguil To: dev@dpdk.org Date: Fri, 1 Sep 2017 10:07:06 +0200 Message-Id: X-Mailer: git-send-email 2.1.4 In-Reply-To: References: Subject: [dpdk-dev] [PATCH v2 51/51] net/mlx4: rely on ethdev for Tx/Rx queue arrays X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Allocation and management of Tx/Rx queue arrays is done by wrappers at the ethdev level. The resulting information is copied to the private structure while configuring the device, where it is managed separately by the PMD. This is redundant and consumes space in the private structure. Relying more on ethdev also means there is no need to protect the PMD against burst function calls while closing the device anymore. Signed-off-by: Adrien Mazarguil --- drivers/net/mlx4/mlx4.c | 58 ++++--------------------------------- drivers/net/mlx4/mlx4.h | 5 ---- drivers/net/mlx4/mlx4_ethdev.c | 41 ++++++++++++-------------- drivers/net/mlx4/mlx4_flow.c | 5 ++-- drivers/net/mlx4/mlx4_intr.c | 10 +++---- drivers/net/mlx4/mlx4_rxq.c | 20 ++++++------- drivers/net/mlx4/mlx4_txq.c | 16 +++++----- 7 files changed, 48 insertions(+), 107 deletions(-) diff --git a/drivers/net/mlx4/mlx4.c b/drivers/net/mlx4/mlx4.c index 317d0e6..b084903 100644 --- a/drivers/net/mlx4/mlx4.c +++ b/drivers/net/mlx4/mlx4.c @@ -44,7 +44,6 @@ #include #include #include -#include /* Verbs headers do not support -pedantic. */ #ifdef PEDANTIC @@ -88,8 +87,6 @@ const char *pmd_mlx4_init_params[] = { /** * DPDK callback for Ethernet device configuration. * - * Prepare the driver for a given number of TX and RX queues. - * * @param dev * Pointer to Ethernet device structure. * @@ -99,22 +96,7 @@ const char *pmd_mlx4_init_params[] = { static int mlx4_dev_configure(struct rte_eth_dev *dev) { - struct priv *priv = dev->data->dev_private; - unsigned int rxqs_n = dev->data->nb_rx_queues; - unsigned int txqs_n = dev->data->nb_tx_queues; - - priv->rxqs = (void *)dev->data->rx_queues; - priv->txqs = (void *)dev->data->tx_queues; - if (txqs_n != priv->txqs_n) { - INFO("%p: TX queues number update: %u -> %u", - (void *)dev, priv->txqs_n, txqs_n); - priv->txqs_n = txqs_n; - } - if (rxqs_n != priv->rxqs_n) { - INFO("%p: Rx queues number update: %u -> %u", - (void *)dev, priv->rxqs_n, rxqs_n); - priv->rxqs_n = rxqs_n; - } + (void)dev; return 0; } @@ -196,7 +178,6 @@ static void mlx4_dev_close(struct rte_eth_dev *dev) { struct priv *priv = dev->data->dev_private; - void *tmp; unsigned int i; if (priv == NULL) @@ -205,41 +186,12 @@ mlx4_dev_close(struct rte_eth_dev *dev) (void *)dev, ((priv->ctx != NULL) ? priv->ctx->device->name : "")); mlx4_mac_addr_del(priv); - /* - * Prevent crashes when queues are still in use. This is unfortunately - * still required for DPDK 1.3 because some programs (such as testpmd) - * never release them before closing the device. - */ dev->rx_pkt_burst = mlx4_rx_burst_removed; dev->tx_pkt_burst = mlx4_tx_burst_removed; - if (priv->rxqs != NULL) { - /* XXX race condition if mlx4_rx_burst() is still running. */ - usleep(1000); - for (i = 0; (i != priv->rxqs_n); ++i) { - tmp = (*priv->rxqs)[i]; - if (tmp == NULL) - continue; - (*priv->rxqs)[i] = NULL; - mlx4_rxq_cleanup(tmp); - rte_free(tmp); - } - priv->rxqs_n = 0; - priv->rxqs = NULL; - } - if (priv->txqs != NULL) { - /* XXX race condition if mlx4_tx_burst() is still running. */ - usleep(1000); - for (i = 0; (i != priv->txqs_n); ++i) { - tmp = (*priv->txqs)[i]; - if (tmp == NULL) - continue; - (*priv->txqs)[i] = NULL; - mlx4_txq_cleanup(tmp); - rte_free(tmp); - } - priv->txqs_n = 0; - priv->txqs = NULL; - } + for (i = 0; i != dev->data->nb_rx_queues; ++i) + mlx4_rx_queue_release(dev->data->rx_queues[i]); + for (i = 0; i != dev->data->nb_tx_queues; ++i) + mlx4_tx_queue_release(dev->data->tx_queues[i]); if (priv->pd != NULL) { assert(priv->ctx != NULL); claim_zero(ibv_dealloc_pd(priv->pd)); diff --git a/drivers/net/mlx4/mlx4.h b/drivers/net/mlx4/mlx4.h index 1cd4db3..93e5502 100644 --- a/drivers/net/mlx4/mlx4.h +++ b/drivers/net/mlx4/mlx4.h @@ -104,11 +104,6 @@ struct priv { unsigned int vf:1; /* This is a VF device. */ unsigned int intr_alarm:1; /* An interrupt alarm is scheduled. */ unsigned int isolated:1; /* Toggle isolated mode. */ - /* RX/TX queues. */ - unsigned int rxqs_n; /* RX queues array size. */ - unsigned int txqs_n; /* TX queues array size. */ - struct rxq *(*rxqs)[]; /* RX queues. */ - struct txq *(*txqs)[]; /* TX queues. */ struct rte_intr_handle intr_handle; /* Port interrupt handle. */ struct rte_flow_drop *flow_drop_queue; /* Flow drop queue. */ LIST_HEAD(mlx4_flows, rte_flow) flows; diff --git a/drivers/net/mlx4/mlx4_ethdev.c b/drivers/net/mlx4/mlx4_ethdev.c index 5f1dba2..a9e8059 100644 --- a/drivers/net/mlx4/mlx4_ethdev.c +++ b/drivers/net/mlx4/mlx4_ethdev.c @@ -574,17 +574,14 @@ mlx4_dev_infos_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *info) void mlx4_stats_get(struct rte_eth_dev *dev, struct rte_eth_stats *stats) { - struct priv *priv = dev->data->dev_private; struct rte_eth_stats tmp; unsigned int i; unsigned int idx; - if (priv == NULL) - return; memset(&tmp, 0, sizeof(tmp)); /* Add software counters. */ - for (i = 0; (i != priv->rxqs_n); ++i) { - struct rxq *rxq = (*priv->rxqs)[i]; + for (i = 0; i != dev->data->nb_rx_queues; ++i) { + struct rxq *rxq = dev->data->rx_queues[i]; if (rxq == NULL) continue; @@ -600,8 +597,8 @@ mlx4_stats_get(struct rte_eth_dev *dev, struct rte_eth_stats *stats) tmp.ierrors += rxq->stats.idropped; tmp.rx_nombuf += rxq->stats.rx_nombuf; } - for (i = 0; (i != priv->txqs_n); ++i) { - struct txq *txq = (*priv->txqs)[i]; + for (i = 0; i != dev->data->nb_tx_queues; ++i) { + struct txq *txq = dev->data->tx_queues[i]; if (txq == NULL) continue; @@ -627,25 +624,23 @@ mlx4_stats_get(struct rte_eth_dev *dev, struct rte_eth_stats *stats) void mlx4_stats_reset(struct rte_eth_dev *dev) { - struct priv *priv = dev->data->dev_private; unsigned int i; - unsigned int idx; - if (priv == NULL) - return; - for (i = 0; (i != priv->rxqs_n); ++i) { - if ((*priv->rxqs)[i] == NULL) - continue; - idx = (*priv->rxqs)[i]->stats.idx; - (*priv->rxqs)[i]->stats = - (struct mlx4_rxq_stats){ .idx = idx }; + for (i = 0; i != dev->data->nb_rx_queues; ++i) { + struct rxq *rxq = dev->data->rx_queues[i]; + + if (rxq) + rxq->stats = (struct mlx4_rxq_stats){ + .idx = rxq->stats.idx, + }; } - for (i = 0; (i != priv->txqs_n); ++i) { - if ((*priv->txqs)[i] == NULL) - continue; - idx = (*priv->txqs)[i]->stats.idx; - (*priv->txqs)[i]->stats = - (struct mlx4_txq_stats){ .idx = idx }; + for (i = 0; i != dev->data->nb_tx_queues; ++i) { + struct txq *txq = dev->data->tx_queues[i]; + + if (txq) + txq->stats = (struct mlx4_txq_stats){ + .idx = txq->stats.idx, + }; } } diff --git a/drivers/net/mlx4/mlx4_flow.c b/drivers/net/mlx4/mlx4_flow.c index e177545..0885a91 100644 --- a/drivers/net/mlx4/mlx4_flow.c +++ b/drivers/net/mlx4/mlx4_flow.c @@ -691,7 +691,8 @@ mlx4_flow_prepare(struct priv *priv, (const struct rte_flow_action_queue *) actions->conf; - if (!queue || (queue->index > (priv->rxqs_n - 1))) + if (!queue || (queue->index > + (priv->dev->data->nb_rx_queues - 1))) goto exit_action_not_supported; action.queue = 1; } else { @@ -841,7 +842,7 @@ mlx4_flow_create_action_queue(struct priv *priv, if (action->drop) { qp = priv->flow_drop_queue ? priv->flow_drop_queue->qp : NULL; } else { - struct rxq *rxq = (*priv->rxqs)[action->queue_id]; + struct rxq *rxq = priv->dev->data->rx_queues[action->queue_id]; qp = rxq->qp; rte_flow->qp = qp; diff --git a/drivers/net/mlx4/mlx4_intr.c b/drivers/net/mlx4/mlx4_intr.c index 76d2e01..e3449ee 100644 --- a/drivers/net/mlx4/mlx4_intr.c +++ b/drivers/net/mlx4/mlx4_intr.c @@ -91,7 +91,7 @@ static int mlx4_rx_intr_vec_enable(struct priv *priv) { unsigned int i; - unsigned int rxqs_n = priv->rxqs_n; + unsigned int rxqs_n = priv->dev->data->nb_rx_queues; unsigned int n = RTE_MIN(rxqs_n, (uint32_t)RTE_MAX_RXTX_INTR_VEC_ID); unsigned int count = 0; struct rte_intr_handle *intr_handle = &priv->intr_handle; @@ -105,7 +105,7 @@ mlx4_rx_intr_vec_enable(struct priv *priv) return -rte_errno; } for (i = 0; i != n; ++i) { - struct rxq *rxq = (*priv->rxqs)[i]; + struct rxq *rxq = priv->dev->data->rx_queues[i]; /* Skip queues that cannot request interrupts. */ if (!rxq || !rxq->channel) { @@ -324,8 +324,7 @@ mlx4_intr_install(struct priv *priv) int mlx4_rx_intr_disable(struct rte_eth_dev *dev, uint16_t idx) { - struct priv *priv = dev->data->dev_private; - struct rxq *rxq = (*priv->rxqs)[idx]; + struct rxq *rxq = dev->data->rx_queues[idx]; struct ibv_cq *ev_cq; void *ev_ctx; int ret; @@ -361,8 +360,7 @@ mlx4_rx_intr_disable(struct rte_eth_dev *dev, uint16_t idx) int mlx4_rx_intr_enable(struct rte_eth_dev *dev, uint16_t idx) { - struct priv *priv = dev->data->dev_private; - struct rxq *rxq = (*priv->rxqs)[idx]; + struct rxq *rxq = dev->data->rx_queues[idx]; int ret; if (!rxq || !rxq->channel) diff --git a/drivers/net/mlx4/mlx4_rxq.c b/drivers/net/mlx4/mlx4_rxq.c index 7f675a4..409983f 100644 --- a/drivers/net/mlx4/mlx4_rxq.c +++ b/drivers/net/mlx4/mlx4_rxq.c @@ -403,15 +403,15 @@ mlx4_rx_queue_setup(struct rte_eth_dev *dev, uint16_t idx, uint16_t desc, struct rte_mempool *mp) { struct priv *priv = dev->data->dev_private; - struct rxq *rxq = (*priv->rxqs)[idx]; + struct rxq *rxq = dev->data->rx_queues[idx]; int ret; DEBUG("%p: configuring queue %u for %u descriptors", (void *)dev, idx, desc); - if (idx >= priv->rxqs_n) { + if (idx >= dev->data->nb_rx_queues) { rte_errno = EOVERFLOW; ERROR("%p: queue index out of range (%u >= %u)", - (void *)dev, idx, priv->rxqs_n); + (void *)dev, idx, dev->data->nb_rx_queues); return -rte_errno; } if (rxq != NULL) { @@ -421,7 +421,7 @@ mlx4_rx_queue_setup(struct rte_eth_dev *dev, uint16_t idx, uint16_t desc, rte_errno = EEXIST; return -rte_errno; } - (*priv->rxqs)[idx] = NULL; + dev->data->rx_queues[idx] = NULL; if (idx == 0) mlx4_mac_addr_del(priv); mlx4_rxq_cleanup(rxq); @@ -441,7 +441,7 @@ mlx4_rx_queue_setup(struct rte_eth_dev *dev, uint16_t idx, uint16_t desc, rxq->stats.idx = idx; DEBUG("%p: adding Rx queue %p to list", (void *)dev, (void *)rxq); - (*priv->rxqs)[idx] = rxq; + dev->data->rx_queues[idx] = rxq; /* Update receive callback. */ dev->rx_pkt_burst = mlx4_rx_burst; } @@ -464,11 +464,11 @@ mlx4_rx_queue_release(void *dpdk_rxq) if (rxq == NULL) return; priv = rxq->priv; - for (i = 0; (i != priv->rxqs_n); ++i) - if ((*priv->rxqs)[i] == rxq) { + for (i = 0; i != priv->dev->data->nb_rx_queues; ++i) + if (priv->dev->data->rx_queues[i] == rxq) { DEBUG("%p: removing Rx queue %p from list", (void *)priv->dev, (void *)rxq); - (*priv->rxqs)[i] = NULL; + priv->dev->data->rx_queues[i] = NULL; if (i == 0) mlx4_mac_addr_del(priv); break; @@ -522,8 +522,8 @@ mlx4_mac_addr_add(struct priv *priv) return 0; if (priv->isolated) return 0; - if (*priv->rxqs && (*priv->rxqs)[0]) - rxq = (*priv->rxqs)[0]; + if (priv->dev->data->rx_queues && priv->dev->data->rx_queues[0]) + rxq = priv->dev->data->rx_queues[0]; else return 0; diff --git a/drivers/net/mlx4/mlx4_txq.c b/drivers/net/mlx4/mlx4_txq.c index 945833b..e0245b0 100644 --- a/drivers/net/mlx4/mlx4_txq.c +++ b/drivers/net/mlx4/mlx4_txq.c @@ -401,15 +401,15 @@ mlx4_tx_queue_setup(struct rte_eth_dev *dev, uint16_t idx, uint16_t desc, unsigned int socket, const struct rte_eth_txconf *conf) { struct priv *priv = dev->data->dev_private; - struct txq *txq = (*priv->txqs)[idx]; + struct txq *txq = dev->data->tx_queues[idx]; int ret; DEBUG("%p: configuring queue %u for %u descriptors", (void *)dev, idx, desc); - if (idx >= priv->txqs_n) { + if (idx >= dev->data->nb_tx_queues) { rte_errno = EOVERFLOW; ERROR("%p: queue index out of range (%u >= %u)", - (void *)dev, idx, priv->txqs_n); + (void *)dev, idx, dev->data->nb_tx_queues); return -rte_errno; } if (txq != NULL) { @@ -419,7 +419,7 @@ mlx4_tx_queue_setup(struct rte_eth_dev *dev, uint16_t idx, uint16_t desc, rte_errno = EEXIST; return -rte_errno; } - (*priv->txqs)[idx] = NULL; + dev->data->tx_queues[idx] = NULL; mlx4_txq_cleanup(txq); } else { txq = rte_calloc_socket("TXQ", 1, sizeof(*txq), 0, socket); @@ -437,7 +437,7 @@ mlx4_tx_queue_setup(struct rte_eth_dev *dev, uint16_t idx, uint16_t desc, txq->stats.idx = idx; DEBUG("%p: adding Tx queue %p to list", (void *)dev, (void *)txq); - (*priv->txqs)[idx] = txq; + dev->data->tx_queues[idx] = txq; /* Update send callback. */ dev->tx_pkt_burst = mlx4_tx_burst; } @@ -460,11 +460,11 @@ mlx4_tx_queue_release(void *dpdk_txq) if (txq == NULL) return; priv = txq->priv; - for (i = 0; (i != priv->txqs_n); ++i) - if ((*priv->txqs)[i] == txq) { + for (i = 0; i != priv->dev->data->nb_tx_queues; ++i) + if (priv->dev->data->tx_queues[i] == txq) { DEBUG("%p: removing Tx queue %p from list", (void *)priv->dev, (void *)txq); - (*priv->txqs)[i] = NULL; + priv->dev->data->tx_queues[i] = NULL; break; } mlx4_txq_cleanup(txq);