From patchwork Thu Feb 7 12:17:43 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Rybchenko X-Patchwork-Id: 50183 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 594131B555; Thu, 7 Feb 2019 13:18:42 +0100 (CET) Received: from dispatch1-us1.ppe-hosted.com (dispatch1-us1.ppe-hosted.com [148.163.129.52]) by dpdk.org (Postfix) with ESMTP id E6C0F1B4B6 for ; Thu, 7 Feb 2019 13:18:17 +0100 (CET) X-Virus-Scanned: Proofpoint Essentials engine Received: from webmail.solarflare.com (webmail.solarflare.com [12.187.104.26]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mx1-us3.ppe-hosted.com (Proofpoint Essentials ESMTP Server) with ESMTPS id A29D5600067 for ; Thu, 7 Feb 2019 12:18:16 +0000 (UTC) Received: from ocex03.SolarFlarecom.com (10.20.40.36) by ocex03.SolarFlarecom.com (10.20.40.36) with Microsoft SMTP Server (TLS) id 15.0.1395.4; Thu, 7 Feb 2019 04:18:10 -0800 Received: from opal.uk.solarflarecom.com (10.17.10.1) by ocex03.SolarFlarecom.com (10.20.40.36) with Microsoft SMTP Server (TLS) id 15.0.1395.4 via Frontend Transport; Thu, 7 Feb 2019 04:18:10 -0800 Received: from ukv-loginhost.uk.solarflarecom.com (ukv-loginhost.uk.solarflarecom.com [10.17.10.39]) by opal.uk.solarflarecom.com (8.13.8/8.13.8) with ESMTP id x17CI9M5013280 for ; Thu, 7 Feb 2019 12:18:09 GMT Received: from ukv-loginhost.uk.solarflarecom.com (localhost [127.0.0.1]) by ukv-loginhost.uk.solarflarecom.com (Postfix) with ESMTP id 244C61613E4 for ; Thu, 7 Feb 2019 12:18:09 +0000 (GMT) From: Andrew Rybchenko To: Date: Thu, 7 Feb 2019 12:17:43 +0000 Message-ID: <1549541873-17403-21-git-send-email-arybchenko@solarflare.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1549541873-17403-1-git-send-email-arybchenko@solarflare.com> References: <1549541873-17403-1-git-send-email-arybchenko@solarflare.com> MIME-Version: 1.0 X-TM-AS-Product-Ver: SMEX-12.5.0.1300-8.5.1010-24412.006 X-TM-AS-Result: No-2.889200-4.000000-10 X-TMASE-MatchedRID: 15UE9zINTirW5rPMDq6HbwPZZctd3P4BlSphlGf1p5ES+jFO7d+PWyxV Tnd/2+hkO9d867AEK2j+qrXTNqMyZjd/MTfApr+s5O5PclyYqqrt/okBLaEo+HPBWZjlMys2ixP 77M1esQwrVPAnGHaZeJ6uWrWgXC9Fsk3Xm9yiC75pR7+L0B6mE30tCKdnhB589yM15V5aWpj6C0 ePs7A07QkL5VmLaBARtxJbZoBvHP/6fTliHLjX1gh/V/SaZNkXdeSoST5RxXKoIpMziCqFxj7gC +SVb7ZITseBaeP9VOyH2wUPSfLYyj+terldlHrIAAPQWqVl2ibRvisn3a2a0X93tr1Wx/MwKa3j 8X0o5Xs= X-TM-AS-User-Approved-Sender: No X-TM-AS-User-Blocked-Sender: No X-TMASE-Result: 10--2.889200-4.000000 X-TMASE-Version: SMEX-12.5.0.1300-8.5.1010-24412.006 X-MDID: 1549541897-frLjBHmD_EcN Subject: [dpdk-dev] [PATCH 20/30] net/sfc: remove TxQ control from shared TxQ info X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" libefx is not multi-process aware and all related structures should be moved to primary process only. Signed-off-by: Andrew Rybchenko --- drivers/net/sfc/sfc.c | 2 +- drivers/net/sfc/sfc.h | 2 + drivers/net/sfc/sfc_ethdev.c | 2 +- drivers/net/sfc/sfc_tx.c | 75 +++++++++++++++++++++--------------- drivers/net/sfc/sfc_tx.h | 5 +-- 5 files changed, 51 insertions(+), 35 deletions(-) diff --git a/drivers/net/sfc/sfc.c b/drivers/net/sfc/sfc.c index 63bc0a568..dd662df0d 100644 --- a/drivers/net/sfc/sfc.c +++ b/drivers/net/sfc/sfc.c @@ -280,7 +280,7 @@ sfc_set_fw_subvariant(struct sfc_adapter *sa) for (txq_index = 0; txq_index < sa->txq_count; ++txq_index) { struct sfc_txq_info *txq_info = &sa->txq_info[txq_index]; - if (txq_info->txq != NULL) + if (txq_info->state & SFC_TXQ_INITIALIZED) tx_offloads |= txq_info->offloads; } diff --git a/drivers/net/sfc/sfc.h b/drivers/net/sfc/sfc.h index f19a9dafb..7e95170bc 100644 --- a/drivers/net/sfc/sfc.h +++ b/drivers/net/sfc/sfc.h @@ -111,6 +111,7 @@ struct sfc_intr { }; struct sfc_rxq; +struct sfc_txq; struct sfc_rxq_info; struct sfc_txq_info; @@ -262,6 +263,7 @@ struct sfc_adapter { unsigned int txq_count; struct sfc_txq_info *txq_info; + struct sfc_txq *txq_ctrl; boolean_t tso; diff --git a/drivers/net/sfc/sfc_ethdev.c b/drivers/net/sfc/sfc_ethdev.c index 3d38bf661..aa857ec74 100644 --- a/drivers/net/sfc/sfc_ethdev.c +++ b/drivers/net/sfc/sfc_ethdev.c @@ -1248,7 +1248,7 @@ sfc_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id) if (sa->state != SFC_ADAPTER_STARTED) goto fail_not_started; - if (sa->txq_info[tx_queue_id].txq == NULL) + if (sa->txq_info[tx_queue_id].state != SFC_TXQ_INITIALIZED) goto fail_not_setup; rc = sfc_tx_qstart(sa, tx_queue_id); diff --git a/drivers/net/sfc/sfc_tx.c b/drivers/net/sfc/sfc_tx.c index 4357dbfd8..94baf4530 100644 --- a/drivers/net/sfc/sfc_tx.c +++ b/drivers/net/sfc/sfc_tx.c @@ -159,13 +159,7 @@ sfc_tx_qinit(struct sfc_adapter *sa, unsigned int sw_index, if (rc != 0) goto fail_ev_qinit; - rc = ENOMEM; - txq = rte_zmalloc_socket("sfc-txq", sizeof(*txq), 0, socket_id); - if (txq == NULL) - goto fail_txq_alloc; - - txq_info->txq = txq; - + txq = &sa->txq_ctrl[sw_index]; txq->hw_index = sw_index; txq->evq = evq; txq_info->free_thresh = @@ -211,10 +205,6 @@ sfc_tx_qinit(struct sfc_adapter *sa, unsigned int sw_index, sfc_dma_free(sa, &txq->mem); fail_dma_alloc: - txq_info->txq = NULL; - rte_free(txq); - -fail_txq_alloc: sfc_ev_qfini(evq); fail_ev_qinit: @@ -239,22 +229,20 @@ sfc_tx_qfini(struct sfc_adapter *sa, unsigned int sw_index) txq_info = &sa->txq_info[sw_index]; - txq = txq_info->txq; - SFC_ASSERT(txq != NULL); SFC_ASSERT(txq_info->state == SFC_TXQ_INITIALIZED); sa->priv.dp_tx->qdestroy(txq_info->dp); txq_info->dp = NULL; - txq_info->txq = NULL; + txq_info->state &= ~SFC_TXQ_INITIALIZED; txq_info->entries = 0; + txq = &sa->txq_ctrl[sw_index]; + sfc_dma_free(sa, &txq->mem); sfc_ev_qfini(txq->evq); txq->evq = NULL; - - rte_free(txq); } static int @@ -314,7 +302,7 @@ sfc_tx_fini_queues(struct sfc_adapter *sa, unsigned int nb_tx_queues) sw_index = sa->txq_count; while (--sw_index >= (int)nb_tx_queues) { - if (sa->txq_info[sw_index].txq != NULL) + if (sa->txq_info[sw_index].state & SFC_TXQ_INITIALIZED) sfc_tx_qfini(sa, sw_index); } @@ -355,8 +343,18 @@ sfc_tx_configure(struct sfc_adapter *sa) sa->socket_id); if (sa->txq_info == NULL) goto fail_txqs_alloc; + + /* + * Allocate primary process only TxQ control from heap + * since it should not be shared. + */ + rc = ENOMEM; + sa->txq_ctrl = calloc(nb_tx_queues, sizeof(sa->txq_ctrl[0])); + if (sa->txq_ctrl == NULL) + goto fail_txqs_ctrl_alloc; } else { struct sfc_txq_info *new_txq_info; + struct sfc_txq *new_txq_ctrl; if (nb_tx_queues < sa->txq_count) sfc_tx_fini_queues(sa, nb_tx_queues); @@ -367,11 +365,21 @@ sfc_tx_configure(struct sfc_adapter *sa) if (new_txq_info == NULL && nb_tx_queues > 0) goto fail_txqs_realloc; + new_txq_ctrl = realloc(sa->txq_ctrl, + nb_tx_queues * sizeof(sa->txq_ctrl[0])); + if (new_txq_ctrl == NULL && nb_tx_queues > 0) + goto fail_txqs_ctrl_realloc; + sa->txq_info = new_txq_info; - if (nb_tx_queues > sa->txq_count) + sa->txq_ctrl = new_txq_ctrl; + if (nb_tx_queues > sa->txq_count) { memset(&sa->txq_info[sa->txq_count], 0, (nb_tx_queues - sa->txq_count) * sizeof(sa->txq_info[0])); + memset(&sa->txq_ctrl[sa->txq_count], 0, + (nb_tx_queues - sa->txq_count) * + sizeof(sa->txq_ctrl[0])); + } } while (sa->txq_count < nb_tx_queues) { @@ -386,7 +394,9 @@ sfc_tx_configure(struct sfc_adapter *sa) return 0; fail_tx_qinit_info: +fail_txqs_ctrl_realloc: fail_txqs_realloc: +fail_txqs_ctrl_alloc: fail_txqs_alloc: sfc_tx_close(sa); @@ -401,6 +411,9 @@ sfc_tx_close(struct sfc_adapter *sa) { sfc_tx_fini_queues(sa, 0); + free(sa->txq_ctrl); + sa->txq_ctrl = NULL; + rte_free(sa->txq_info); sa->txq_info = NULL; } @@ -423,11 +436,9 @@ sfc_tx_qstart(struct sfc_adapter *sa, unsigned int sw_index) SFC_ASSERT(sw_index < sa->txq_count); txq_info = &sa->txq_info[sw_index]; - txq = txq_info->txq; - - SFC_ASSERT(txq != NULL); SFC_ASSERT(txq_info->state == SFC_TXQ_INITIALIZED); + txq = &sa->txq_ctrl[sw_index]; evq = txq->evq; rc = sfc_ev_qstart(evq, sfc_evq_index_by_txq_sw_index(sa, sw_index)); @@ -504,13 +515,12 @@ sfc_tx_qstop(struct sfc_adapter *sa, unsigned int sw_index) SFC_ASSERT(sw_index < sa->txq_count); txq_info = &sa->txq_info[sw_index]; - txq = txq_info->txq; - - if (txq == NULL || txq_info->state == SFC_TXQ_INITIALIZED) + if (txq_info->state == SFC_TXQ_INITIALIZED) return; SFC_ASSERT(txq_info->state & SFC_TXQ_STARTED); + txq = &sa->txq_ctrl[sw_index]; sa->priv.dp_tx->qstop(txq_info->dp, &txq->evq->read_ptr); /* @@ -583,7 +593,7 @@ sfc_tx_start(struct sfc_adapter *sa) goto fail_efx_tx_init; for (sw_index = 0; sw_index < sa->txq_count; ++sw_index) { - if (sa->txq_info[sw_index].txq != NULL && + if (sa->txq_info[sw_index].state == SFC_TXQ_INITIALIZED && (!(sa->txq_info[sw_index].deferred_start) || sa->txq_info[sw_index].deferred_started)) { rc = sfc_tx_qstart(sa, sw_index); @@ -614,7 +624,7 @@ sfc_tx_stop(struct sfc_adapter *sa) sw_index = sa->txq_count; while (sw_index-- > 0) { - if (sa->txq_info[sw_index].txq != NULL) + if (sa->txq_info[sw_index].state & SFC_TXQ_STARTED) sfc_tx_qstop(sa, sw_index); } @@ -888,12 +898,17 @@ sfc_txq_info_by_dp_txq(const struct sfc_dp_txq *dp_txq) struct sfc_txq * sfc_txq_by_dp_txq(const struct sfc_dp_txq *dp_txq) { - struct sfc_txq_info *txq_info; + const struct sfc_dp_queue *dpq = &dp_txq->dpq; + struct rte_eth_dev *eth_dev; + struct sfc_adapter *sa; - txq_info = sfc_txq_info_by_dp_txq(dp_txq); + SFC_ASSERT(rte_eth_dev_is_valid_port(dpq->port_id)); + eth_dev = &rte_eth_devices[dpq->port_id]; - SFC_ASSERT(txq_info->txq != NULL); - return txq_info->txq; + sa = eth_dev->data->dev_private; + + SFC_ASSERT(dpq->queue_id < sa->txq_count); + return &sa->txq_ctrl[dpq->queue_id]; } static sfc_dp_tx_qsize_up_rings_t sfc_efx_tx_qsize_up_rings; diff --git a/drivers/net/sfc/sfc_tx.h b/drivers/net/sfc/sfc_tx.h index 2d21e6e83..d0e751a26 100644 --- a/drivers/net/sfc/sfc_tx.h +++ b/drivers/net/sfc/sfc_tx.h @@ -47,8 +47,8 @@ enum sfc_txq_state_bit { }; /** - * Transmit queue control information. Not used on datapath. - * Allocated on the socket specified on the queue setup. + * Transmit queue control primary process-only information. + * Not used on datapath. */ struct sfc_txq { unsigned int hw_index; @@ -95,7 +95,6 @@ sfc_efx_txq_by_dp_txq(struct sfc_dp_txq *dp_txq) struct sfc_txq_info { unsigned int state; unsigned int entries; - struct sfc_txq *txq; struct sfc_dp_txq *dp; boolean_t deferred_start; boolean_t deferred_started;