From patchwork Thu Mar 23 23:01:04 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ed Czeck X-Patchwork-Id: 22199 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [IPv6:::1]) by dpdk.org (Postfix) with ESMTP id BF6FDCFC6; Fri, 24 Mar 2017 00:01:09 +0100 (CET) Received: from mail-qk0-f196.google.com (mail-qk0-f196.google.com [209.85.220.196]) by dpdk.org (Postfix) with ESMTP id 38F9D5598 for ; Fri, 24 Mar 2017 00:01:08 +0100 (CET) Received: by mail-qk0-f196.google.com with SMTP id p22so4967910qka.0 for ; Thu, 23 Mar 2017 16:01:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=atomicrules-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :in-reply-to:references; bh=Bse4TcFmhvutBPGZ99GE48CvgylNwW99opEUOmD2BEE=; b=K7NuioNIwYvNi0Sq2e2DOahJRhO3Pc7C1+dSb1/qfjdHKPTkzldEgyb5XDk50JsRdZ C478MG4PYVHNt8k223FfkiQKkMo5O4nqGrd1Mg0YNQhqRNl3NkpQTI9h5hHelLoew9+U yNFBRBwXZQjR0QFBOHnRQkTTyMyyBSNAut9+n34drhe+1MEPTKM1tt6ao0/OBR8agjwK KHKFzIago/JUKUCJjsq3dyoinEUv++hppR/b3cAEWooMvOAi42nN2me5sOJJssoAD9Pn tVyoa0ASTx1aRGhcVIhnuj8SOGHRUoq3DwSrFXywkRa/1inzOEglDmtrS/5mNb8rSnp+ P5PA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:in-reply-to:references; bh=Bse4TcFmhvutBPGZ99GE48CvgylNwW99opEUOmD2BEE=; b=rzx9OjA/GJAU+mC9yc3jG0rVj89AklgFcRe1G9/BXHPLPl2H8IZofHNXvujPMxzSor LD+ALL0Kc1WsLJ/BQx8yze2ie83NEV7zQUGuMVcIvkTfn7WUzcOEfQmAuAk7oh/7EDgF 43xUF7SKK9iCK3jRt0e3c66797b0AaijWfOMQ2VzrxoS7XAzcplQsi+4U1y+yqhNEFlj TT/gr8A7N3NaytyWKHlEhLSersmvYbBAYqFzCbEA39Y5qK0/YgMabjIqc/6k+h5OHYgv ru94O6aacgtBJrE+kWRBUA4hUB0SK8ryr7zDW/C1w5ZKo/ygU2gvBgzwErfgHssB+Ice IqnQ== X-Gm-Message-State: AFeK/H1+5q0AGXhfksSfRypOBHxq9RDmX0YKCMF3LO3tjwXRS2dP661xDkkIA+NtxF1/gg== X-Received: by 10.55.191.6 with SMTP id p6mr4913636qkf.133.1490310067338; Thu, 23 Mar 2017 16:01:07 -0700 (PDT) Received: from z170.home (pool-173-48-118-253.bstnma.fios.verizon.net. [173.48.118.253]) by smtp.gmail.com with ESMTPSA id n19sm323542qtn.35.2017.03.23.16.01.06 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 23 Mar 2017 16:01:06 -0700 (PDT) From: Ed Czeck To: dev@dpdk.org Cc: john.miller@atomicrules.com, shepard.siegel@atomicrules.com, ferruh.yigit@intel.com, stephen@networkplumber.org, Ed Czeck Date: Thu, 23 Mar 2017 19:01:04 -0400 Message-Id: <8a5f46f851c7be85e09e50d7c94b7709edaf881e.1490309515.git.ed.czeck@atomicrules.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1490231015-31748-6-git-send-email-ed.czeck@atomicrules.com> References: <1490231015-31748-6-git-send-email-ed.czeck@atomicrules.com> In-Reply-To: <41265c4fa76265df0144d5d480e4553888df2ee8.1490309515.git.ed.czeck@atomicrules.com> References: <41265c4fa76265df0144d5d480e4553888df2ee8.1490309515.git.ed.czeck@atomicrules.com> Subject: [dpdk-dev] [PATCH v5 6/7] net/ark: Packet RX support initial version X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" * Core RX packet moving functions Signed-off-by: Ed Czeck --- drivers/net/ark/Makefile | 1 + drivers/net/ark/ark_ethdev_rx.c | 680 ++++++++++++++++++++++++++++++++++++++++ drivers/net/ark/ark_ethdev_rx.h | 67 ++++ 3 files changed, 748 insertions(+) create mode 100644 drivers/net/ark/ark_ethdev_rx.c create mode 100644 drivers/net/ark/ark_ethdev_rx.h diff --git a/drivers/net/ark/Makefile b/drivers/net/ark/Makefile index 0def85a..ca1f397 100644 --- a/drivers/net/ark/Makefile +++ b/drivers/net/ark/Makefile @@ -48,6 +48,7 @@ LIBABIVER := 1 # SRCS-$(CONFIG_RTE_LIBRTE_ARK_PMD) += ark_ddm.c SRCS-$(CONFIG_RTE_LIBRTE_ARK_PMD) += ark_ethdev.c +SRCS-$(CONFIG_RTE_LIBRTE_ARK_PMD) += ark_ethdev_rx.c SRCS-$(CONFIG_RTE_LIBRTE_ARK_PMD) += ark_ethdev_tx.c SRCS-$(CONFIG_RTE_LIBRTE_ARK_PMD) += ark_mpu.c SRCS-$(CONFIG_RTE_LIBRTE_ARK_PMD) += ark_pktchkr.c diff --git a/drivers/net/ark/ark_ethdev_rx.c b/drivers/net/ark/ark_ethdev_rx.c new file mode 100644 index 0000000..6881907 --- /dev/null +++ b/drivers/net/ark/ark_ethdev_rx.c @@ -0,0 +1,680 @@ +/*- + * BSD LICENSE + * + * Copyright (c) 2015-2017 Atomic Rules LLC + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * * Neither the name of copyright holder nor the names of its + * contributors may be used to endorse or promote products derived + * from this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + */ + +#include + +#include "ark_ethdev_rx.h" +#include "ark_global.h" +#include "ark_debug.h" +#include "ark_ethdev.h" +#include "ark_mpu.h" +#include "ark_udm.h" + +#define ARK_RX_META_SIZE 32 +#define ARK_RX_META_OFFSET (RTE_PKTMBUF_HEADROOM - ARK_RX_META_SIZE) +#define ARK_RX_MAX_NOCHAIN (RTE_MBUF_DEFAULT_DATAROOM) + +#ifdef RTE_LIBRTE_ARK_DEBUG_RX +#define ARK_RX_DEBUG 1 +#else +#define ARK_RX_DEBUG 0 +#endif + +/* Forward declarations */ +struct ark_rx_queue; +struct ark_rx_meta; + +static void dump_mbuf_data(struct rte_mbuf *mbuf, uint16_t lo, uint16_t hi); +static void ark_ethdev_rx_dump(const char *name, struct ark_rx_queue *queue); +static uint32_t eth_ark_rx_jumbo(struct ark_rx_queue *queue, + struct ark_rx_meta *meta, + struct rte_mbuf *mbuf0, + uint32_t cons_index); +static inline int eth_ark_rx_seed_mbufs(struct ark_rx_queue *queue); + +/* ************************************************************************* */ +struct ark_rx_queue { + /* array of mbufs to populate */ + struct rte_mbuf **reserve_q; + /* array of physical addrresses of the mbuf data pointer */ + /* This point is a virtual address */ + phys_addr_t *paddress_q; + struct rte_mempool *mb_pool; + + struct ark_udm_t *udm; + struct ark_mpu_t *mpu; + + uint32_t queue_size; + uint32_t queue_mask; + + uint32_t seed_index; /* 1 set with an empty mbuf */ + uint32_t cons_index; /* 3 consumed by the driver */ + + /* The queue Id is used to identify the HW Q */ + uint16_t phys_qid; + + /* The queue Index is used within the dpdk device structures */ + uint16_t queue_index; + + uint32_t pad1; + + /* separate cache line */ + /* second cache line - fields only used in slow path */ + MARKER cacheline1 __rte_cache_min_aligned; + + volatile uint32_t prod_index; /* 2 filled by the HW */ +} __rte_cache_aligned; + +/* ************************************************************************* */ + +/* MATCHES struct in UDMDefines.bsv */ + +/* ************************************************************************* */ + +/* TODO pick a better function name */ +static int +eth_ark_rx_hw_setup(struct rte_eth_dev *dev, + struct ark_rx_queue *queue, + uint16_t rx_queue_id __rte_unused, uint16_t rx_queue_idx) +{ + phys_addr_t queue_base; + phys_addr_t phys_addr_q_base; + phys_addr_t phys_addr_prod_index; + + queue_base = rte_malloc_virt2phy(queue); + phys_addr_prod_index = queue_base + + offsetof(struct ark_rx_queue, prod_index); + + phys_addr_q_base = rte_malloc_virt2phy(queue->paddress_q); + + /* Verify HW */ + if (ark_mpu_verify(queue->mpu, sizeof(phys_addr_t))) { + PMD_DRV_LOG(ERR, "ARKP: Illegal configuration rx queue\n"); + return -1; + } + + /* Stop and Reset and configure MPU */ + ark_mpu_configure(queue->mpu, phys_addr_q_base, queue->queue_size, 0); + + ark_udm_write_addr(queue->udm, phys_addr_prod_index); + + /* advance the valid pointer, but don't start until the queue starts */ + ark_mpu_reset_stats(queue->mpu); + + /* The seed is the producer index for the HW */ + ark_mpu_set_producer(queue->mpu, queue->seed_index); + dev->data->rx_queue_state[rx_queue_idx] = RTE_ETH_QUEUE_STATE_STOPPED; + + return 0; +} + +static inline void +eth_ark_rx_update_cons_index(struct ark_rx_queue *queue, uint32_t cons_index) +{ + queue->cons_index = cons_index; + eth_ark_rx_seed_mbufs(queue); + ark_mpu_set_producer(queue->mpu, queue->seed_index); +} + +/* ************************************************************************* */ +int +eth_ark_dev_rx_queue_setup(struct rte_eth_dev *dev, + uint16_t queue_idx, + uint16_t nb_desc, + unsigned int socket_id, + const struct rte_eth_rxconf *rx_conf, + struct rte_mempool *mb_pool) +{ + struct ark_adapter *ark = (struct ark_adapter *)dev->data->dev_private; + static int warning1; /* = 0 */ + + struct ark_rx_queue *queue; + uint32_t i; + int status; + + /* TODO: divide the Q's evenly with the Vports */ + int port = dev->data->port_id; + int qidx = port + queue_idx; /* FIXME for multi queue */ + + // TODO: We may already be setup, check here if there is nothing to do + /* Free memory prior to re-allocation if needed */ + if (dev->data->rx_queues[queue_idx] != NULL) { + // TODO: release any allocated queues + dev->data->rx_queues[queue_idx] = NULL; + } + + if (rx_conf != NULL && warning1 == 0) { + warning1 = 1; + PMD_DRV_LOG(INFO, + "ARKP: Arkville PMD ignores rte_eth_rxconf argument.\n"); + } + + if (RTE_PKTMBUF_HEADROOM < ARK_RX_META_SIZE) { + PMD_DRV_LOG(ERR, + "Error: DPDK Arkville requires head room > %d bytes (%s)\n", + ARK_RX_META_SIZE, __func__); + return -1; /* ERROR CODE */ + } + + if (!rte_is_power_of_2(nb_desc)) { + PMD_DRV_LOG(ERR, + "DPDK Arkville configuration queue size must be power of two %u (%s)\n", + nb_desc, __func__); + return -1; /* ERROR CODE */ + } + + /* Allocate queue struct */ + queue = rte_zmalloc_socket("Ark_rXQueue", + sizeof(struct ark_rx_queue), + 64, + socket_id); + if (queue == 0) { + PMD_DRV_LOG(ERR, "Failed to allocate memory in %s\n", __func__); + return -ENOMEM; + } + + /* NOTE zmalloc is used, no need to 0 indexes, etc. */ + queue->mb_pool = mb_pool; + queue->phys_qid = qidx; + queue->queue_index = queue_idx; + queue->queue_size = nb_desc; + queue->queue_mask = nb_desc - 1; + + queue->reserve_q = + rte_zmalloc_socket("Ark_rx_queue mbuf", + nb_desc * sizeof(struct rte_mbuf *), + 64, + socket_id); + queue->paddress_q = + rte_zmalloc_socket("Ark_rx_queue paddr", + nb_desc * sizeof(phys_addr_t), + 64, + socket_id); + + if (queue->reserve_q == 0 || queue->paddress_q == 0) { + PMD_DRV_LOG(ERR, + "Failed to allocate queue memory in %s\n", + __func__); + rte_free(queue->reserve_q); + rte_free(queue->paddress_q); + rte_free(queue); + return -ENOMEM; + } + + dev->data->rx_queues[queue_idx] = queue; + queue->udm = RTE_PTR_ADD(ark->udm.v, qidx * ARK_UDM_QOFFSET); + queue->mpu = RTE_PTR_ADD(ark->mpurx.v, qidx * ARK_MPU_QOFFSET); + + /* populate mbuf reserve */ + status = eth_ark_rx_seed_mbufs(queue); + + /* MPU Setup */ + if (status == 0) + status = eth_ark_rx_hw_setup(dev, queue, qidx, queue_idx); + + if (unlikely(status != 0)) { + struct rte_mbuf *mbuf; + + PMD_DRV_LOG(ERR, "ARKP Failed to initialize RX queue %d %s\n", + qidx, + __func__); + /* Free the mbufs allocated */ + for (i = 0, mbuf = queue->reserve_q[0]; + i < nb_desc; ++i, mbuf++) { + rte_pktmbuf_free(mbuf); + } + rte_free(queue->reserve_q); + rte_free(queue->paddress_q); + rte_free(queue); + return -1; /* ERROR CODE */ + } + + return 0; +} + +/* ************************************************************************* */ +uint16_t +eth_ark_recv_pkts_noop(void *rx_queue __rte_unused, + struct rte_mbuf **rx_pkts __rte_unused, + uint16_t nb_pkts __rte_unused) +{ + return 0; +} + +/* ************************************************************************* */ +uint16_t +eth_ark_recv_pkts(void *rx_queue, + struct rte_mbuf **rx_pkts, + uint16_t nb_pkts) +{ + struct ark_rx_queue *queue; + register uint32_t cons_index, prod_index; + uint16_t nb; + struct rte_mbuf *mbuf; + struct ark_rx_meta *meta; + + queue = (struct ark_rx_queue *)rx_queue; + if (unlikely(queue == 0)) + return 0; + if (unlikely(nb_pkts == 0)) + return 0; + prod_index = queue->prod_index; + cons_index = queue->cons_index; + nb = 0; + + while (prod_index != cons_index) { + mbuf = queue->reserve_q[cons_index & queue->queue_mask]; + /* prefetch mbuf ? */ + rte_mbuf_prefetch_part1(mbuf); + rte_mbuf_prefetch_part2(mbuf); + + /* META DATA burried in buffer */ + meta = RTE_PTR_ADD(mbuf->buf_addr, ARK_RX_META_OFFSET); + + mbuf->port = meta->port; + mbuf->pkt_len = meta->pkt_len; + mbuf->data_len = meta->pkt_len; + mbuf->data_off = RTE_PKTMBUF_HEADROOM; + mbuf->udata64 = meta->user_data; + if (ARK_RX_DEBUG) { /* debug use */ + if ((meta->pkt_len > (1024 * 16)) || + (meta->pkt_len == 0)) { + PMD_DRV_LOG(INFO, + "ARKP RX: Bad Meta Q: %u" + " cons: %u prod: %u\n", + queue->phys_qid, + cons_index, + queue->prod_index); + + PMD_DRV_LOG(INFO, + " : cons: %u prod: %u" + " seed_index %u\n", + cons_index, + queue->prod_index, + queue->seed_index); + + PMD_DRV_LOG(INFO, " : UDM prod: %u" + " len: %u\n", + queue->udm->rt_cfg.prod_idx, + meta->pkt_len); + ark_mpu_dump(queue->mpu, + " ", + queue->phys_qid); + + dump_mbuf_data(mbuf, 0, 256); + /* its FUBAR so fix it */ + mbuf->pkt_len = 63; + meta->pkt_len = 63; + } + mbuf->seqn = cons_index; + } + + if (unlikely(meta->pkt_len > ARK_RX_MAX_NOCHAIN)) + cons_index = eth_ark_rx_jumbo + (queue, meta, mbuf, cons_index + 1); + else + cons_index += 1; + + rx_pkts[nb] = mbuf; + nb++; + if (nb >= nb_pkts) + break; + } + + if (unlikely(nb != 0)) + /* report next free to FPGA */ + eth_ark_rx_update_cons_index(queue, cons_index); + + return nb; +} + +/* ************************************************************************* */ +static uint32_t +eth_ark_rx_jumbo(struct ark_rx_queue *queue, + struct ark_rx_meta *meta, + struct rte_mbuf *mbuf0, + uint32_t cons_index) +{ + struct rte_mbuf *mbuf_prev; + struct rte_mbuf *mbuf; + + uint16_t remaining; + uint16_t data_len; + uint8_t segments; + + /* first buf populated by called */ + mbuf_prev = mbuf0; + segments = 1; + data_len = RTE_MIN(meta->pkt_len, RTE_MBUF_DEFAULT_DATAROOM); + remaining = meta->pkt_len - data_len; + mbuf0->data_len = data_len; + + /* TODO check that the data does not exceed prod_index! */ + while (remaining != 0) { + data_len = RTE_MIN(remaining, + RTE_MBUF_DEFAULT_DATAROOM + + RTE_PKTMBUF_HEADROOM); + + remaining -= data_len; + segments += 1; + + mbuf = queue->reserve_q[cons_index & queue->queue_mask]; + mbuf_prev->next = mbuf; + mbuf_prev = mbuf; + mbuf->data_len = data_len; + mbuf->data_off = 0; + if (ARK_RX_DEBUG) + mbuf->seqn = cons_index; /* for debug only */ + + cons_index += 1; + } + + mbuf0->nb_segs = segments; + return cons_index; +} + +/* Drain the internal queue allowing hw to clear out. */ +static void +eth_ark_rx_queue_drain(struct ark_rx_queue *queue) +{ + register uint32_t cons_index; + struct rte_mbuf *mbuf; + + cons_index = queue->cons_index; + + /* NOT performance optimized, since this is a one-shot call */ + while ((cons_index ^ queue->prod_index) & queue->queue_mask) { + mbuf = queue->reserve_q[cons_index & queue->queue_mask]; + rte_pktmbuf_free(mbuf); + cons_index++; + eth_ark_rx_update_cons_index(queue, cons_index); + } +} + +uint32_t +eth_ark_dev_rx_queue_count(struct rte_eth_dev *dev, uint16_t queue_id) +{ + struct ark_rx_queue *queue; + + queue = dev->data->rx_queues[queue_id]; + return (queue->prod_index - queue->cons_index); /* mod arith */ +} + +/* ************************************************************************* */ +int +eth_ark_rx_start_queue(struct rte_eth_dev *dev, uint16_t queue_id) +{ + struct ark_rx_queue *queue; + + queue = dev->data->rx_queues[queue_id]; + if (queue == 0) + return -1; + + dev->data->rx_queue_state[queue_id] = RTE_ETH_QUEUE_STATE_STARTED; + + ark_mpu_set_producer(queue->mpu, queue->seed_index); + ark_mpu_start(queue->mpu); + + ark_udm_queue_enable(queue->udm, 1); + + return 0; +} + +/* ************************************************************************* */ + +/* Queue can be restarted. data remains + */ +int +eth_ark_rx_stop_queue(struct rte_eth_dev *dev, uint16_t queue_id) +{ + struct ark_rx_queue *queue; + + queue = dev->data->rx_queues[queue_id]; + if (queue == 0) + return -1; + + ark_udm_queue_enable(queue->udm, 0); + + dev->data->rx_queue_state[queue_id] = RTE_ETH_QUEUE_STATE_STOPPED; + + return 0; +} + +/* ************************************************************************* */ +static inline int +eth_ark_rx_seed_mbufs(struct ark_rx_queue *queue) +{ + uint32_t limit = queue->cons_index + queue->queue_size; + uint32_t seed_index = queue->seed_index; + + uint32_t count = 0; + uint32_t seed_m = queue->seed_index & queue->queue_mask; + + uint32_t nb = limit - seed_index; + + /* Handle wrap around -- remainder is filled on the next call */ + if (unlikely(seed_m + nb > queue->queue_size)) + nb = queue->queue_size - seed_m; + + struct rte_mbuf **mbufs = &queue->reserve_q[seed_m]; + int status = rte_pktmbuf_alloc_bulk(queue->mb_pool, mbufs, nb); + + if (unlikely(status != 0)) + return -1; + + if (ARK_RX_DEBUG) { /* DEBUG */ + while (count != nb) { + struct rte_mbuf *mbuf_init = + queue->reserve_q[seed_m + count]; + + memset(mbuf_init->buf_addr, -1, 512); + *((uint32_t *)mbuf_init->buf_addr) = + seed_index + count; + *(uint16_t *)RTE_PTR_ADD(mbuf_init->buf_addr, 4) = + queue->phys_qid; + count++; + } + count = 0; + } /* DEBUG */ + queue->seed_index += nb; + + /* Duff's device https://en.wikipedia.org/wiki/Duff's_device */ + switch (nb % 4) { + case 0: + while (count != nb) { + queue->paddress_q[seed_m++] = + (*mbufs++)->buf_physaddr; + count++; + /* FALLTHROUGH */ + case 3: + queue->paddress_q[seed_m++] = + (*mbufs++)->buf_physaddr; + count++; + /* FALLTHROUGH */ + case 2: + queue->paddress_q[seed_m++] = + (*mbufs++)->buf_physaddr; + count++; + /* FALLTHROUGH */ + case 1: + queue->paddress_q[seed_m++] = + (*mbufs++)->buf_physaddr; + count++; + /* FALLTHROUGH */ + + } /* while (count != nb) */ + } /* switch */ + + return 0; +} + +void +eth_ark_rx_dump_queue(struct rte_eth_dev *dev, uint16_t queue_id, + const char *msg) +{ + struct ark_rx_queue *queue; + + queue = dev->data->rx_queues[queue_id]; + + ark_ethdev_rx_dump(msg, queue); +} + +/* ************************************************************************* */ +/* Call on device closed no user API, queue is stopped */ +void +eth_ark_dev_rx_queue_release(void *vqueue) +{ + struct ark_rx_queue *queue; + uint32_t i; + + queue = (struct ark_rx_queue *)vqueue; + if (queue == 0) + return; + + ark_udm_queue_enable(queue->udm, 0); + /* Stop the MPU since pointer are going away */ + ark_mpu_stop(queue->mpu); + + /* Need to clear out mbufs here, dropping packets along the way */ + eth_ark_rx_queue_drain(queue); + + for (i = 0; i < queue->queue_size; ++i) + rte_pktmbuf_free(queue->reserve_q[i]); + + rte_free(queue->reserve_q); + rte_free(queue->paddress_q); + rte_free(queue); +} + +void +eth_rx_queue_stats_get(void *vqueue, struct rte_eth_stats *stats) +{ + struct ark_rx_queue *queue; + struct ark_udm_t *udm; + + queue = vqueue; + if (queue == 0) + return; + udm = queue->udm; + + uint64_t ibytes = ark_udm_bytes(udm); + uint64_t ipackets = ark_udm_packets(udm); + uint64_t idropped = ark_udm_dropped(queue->udm); + + stats->q_ipackets[queue->queue_index] = ipackets; + stats->q_ibytes[queue->queue_index] = ibytes; + stats->q_errors[queue->queue_index] = idropped; + stats->ipackets += ipackets; + stats->ibytes += ibytes; + stats->imissed += idropped; +} + +void +eth_rx_queue_stats_reset(void *vqueue) +{ + struct ark_rx_queue *queue; + + queue = vqueue; + if (queue == 0) + return; + + ark_mpu_reset_stats(queue->mpu); + ark_udm_queue_stats_reset(queue->udm); +} + +void +eth_ark_udm_force_close(struct rte_eth_dev *dev) +{ + struct ark_adapter *ark = (struct ark_adapter *)dev->data->dev_private; + struct ark_rx_queue *queue; + uint32_t index; + uint16_t i; + + if (!ark_udm_is_flushed(ark->udm.v)) { + /* restart the MPUs */ + PMD_DRV_LOG(ERR, "ARK: %s UDM not flushed\n", __func__); + for (i = 0; i < dev->data->nb_rx_queues; i++) { + queue = (struct ark_rx_queue *)dev->data->rx_queues[i]; + if (queue == 0) + continue; + + ark_mpu_start(queue->mpu); + /* Add some buffers */ + index = 100000 + queue->seed_index; + ark_mpu_set_producer(queue->mpu, index); + } + /* Wait to allow data to pass */ + usleep(100); + + ARK_DEBUG_TRACE("UDM forced flush attempt, stopped = %d\n", + ark_udm_is_flushed(ark->udm.v)); + } + ark_udm_reset(ark->udm.v); +} + +static void +ark_ethdev_rx_dump(const char *name, struct ark_rx_queue *queue) +{ + if (queue == NULL) + return; + ARK_DEBUG_TRACE("RX QUEUE %d -- %s", queue->phys_qid, name); + ARK_DEBUG_TRACE(ARK_SU32 ARK_SU32 ARK_SU32 ARK_SU32 "\n", + "queue_size", queue->queue_size, + "seed_index", queue->seed_index, + "prod_index", queue->prod_index, + "cons_index", queue->cons_index); + + ark_mpu_dump(queue->mpu, name, queue->phys_qid); + ark_mpu_dump_setup(queue->mpu, queue->phys_qid); + ark_udm_dump(queue->udm, name); + ark_udm_dump_setup(queue->udm, queue->phys_qid); +} + +/* Only used in debug */ +static void +dump_mbuf_data(struct rte_mbuf *mbuf, uint16_t lo, uint16_t hi) +{ + uint16_t i, j; + + PMD_DRV_LOG(ERR, " MBUF: %p len %d, off: %d, seq: %u\n", mbuf, + mbuf->pkt_len, mbuf->data_off, mbuf->seqn); + for (i = lo; i < hi; i += 16) { + uint8_t *dp = RTE_PTR_ADD(mbuf->buf_addr, i); + + PMD_DRV_LOG(ERR, " %6d: ", i); + for (j = 0; j < 16; j++) + PMD_DRV_LOG(ERR, " %02x", dp[j]); + + PMD_DRV_LOG(ERR, "\n"); + } +} diff --git a/drivers/net/ark/ark_ethdev_rx.h b/drivers/net/ark/ark_ethdev_rx.h new file mode 100644 index 0000000..e6c9fa2 --- /dev/null +++ b/drivers/net/ark/ark_ethdev_rx.h @@ -0,0 +1,67 @@ +/*- + * BSD LICENSE + * + * Copyright (c) 2015-2017 Atomic Rules LLC + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * * Neither the name of copyright holder nor the names of its + * contributors may be used to endorse or promote products derived + * from this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + */ + +#ifndef _ARK_ETHDEV_RX_H_ +#define _ARK_ETHDEV_RX_H_ + +#include + +/* Forward declarations */ +struct rte_mbuf; +struct rte_mempool; +struct rte_eth_dev; +struct rte_eth_stats; +struct rte_eth_rxconf; + +int eth_ark_dev_rx_queue_setup(struct rte_eth_dev *dev, + uint16_t queue_idx, + uint16_t nb_desc, + unsigned int socket_id, + const struct rte_eth_rxconf *rx_conf, + struct rte_mempool *mp); +uint32_t eth_ark_dev_rx_queue_count(struct rte_eth_dev *dev, + uint16_t rx_queue_id); +int eth_ark_rx_stop_queue(struct rte_eth_dev *dev, uint16_t queue_id); +int eth_ark_rx_start_queue(struct rte_eth_dev *dev, uint16_t queue_id); +uint16_t eth_ark_recv_pkts_noop(void *rx_queue, struct rte_mbuf **rx_pkts, + uint16_t nb_pkts); +uint16_t eth_ark_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, + uint16_t nb_pkts); +void eth_ark_dev_rx_queue_release(void *rx_queue); +void eth_rx_queue_stats_get(void *vqueue, struct rte_eth_stats *stats); +void eth_rx_queue_stats_reset(void *vqueue); +void eth_ark_rx_dump_queue(struct rte_eth_dev *dev, uint16_t queue_id, + const char *msg); +void eth_ark_udm_force_close(struct rte_eth_dev *dev); + +#endif