From patchwork Fri Dec 14 13:18:27 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michal Krawczyk X-Patchwork-Id: 48860 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 0AF1E1BB22; Fri, 14 Dec 2018 14:19:01 +0100 (CET) Received: from mail-lj1-f177.google.com (mail-lj1-f177.google.com [209.85.208.177]) by dpdk.org (Postfix) with ESMTP id 23C751BB0C for ; Fri, 14 Dec 2018 14:18:58 +0100 (CET) Received: by mail-lj1-f177.google.com with SMTP id t9-v6so4874396ljh.6 for ; Fri, 14 Dec 2018 05:18:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=semihalf-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=9LX2PA6+w/Fsq0SYuTT9ZOKbdEU4wUaPyvm909Fcl9k=; b=s+/RL+lr5WigqPzCeCywitiLZWtU96ZAu8SVKiK7X5m/LfAsa+MBel8d1i+bJ4r6Bw klialmFLCUsabC4cKyb+5YISyrme1ny0BvgK2OJHW7HxOPrneotaTxU/uO0jkHlkh15C CLA+L9tiKv0442EuFXpvcoSt2iuOstj5aoMkii9KHDS7IeBCcbOMa4TGVN76A5zt5c6E 3mPUu4Zpm6S4KMkFNNWaoOwagzoEvcu/XrLwD4JCNTBsl6ykiA0XPdBzceHLeTzoOONk RFlD0kzArmaLP281KiAbabYPdrLEqIrEp9mKT53EJ/s+iUaIJW1p2R0c5BSoxJAlyeLn /19A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=9LX2PA6+w/Fsq0SYuTT9ZOKbdEU4wUaPyvm909Fcl9k=; b=s049MEkEih7rwK2uZSuNxekp8xE7H9iB0LT8rbmeacuBG5Y9k2f+aQYWSgTy4Di/WH LTK8R6ZCCMyVQyBFoc5B05SA9MmxFFHN9dcRNFuid/9TM1RqH7gX9gBxFsJhWnn9NvD8 uRsQ1cf+t5Ingti+0P1fqUofH1+c8e1/Owq09VDJ7/+txI6NYDueZf5xEqu9sCTnMIFc XXN+CIhI0PBP+EO7X8cRm0zh0YR7fS/S2r96c4FdtmoyiODiQXO1Mvyjw4z3Mva4Hdrb 7C8RBhfsWSU072AFhIhUMV83TmZl2gZUur65nr1Z3Ra48d/2ewCc/Wljg9zp0Ko7XfXc FP5w== X-Gm-Message-State: AA+aEWav5VWWB/1OpKlF2apdzm+8C9W47XqNvqXaeI+7CEXHsy57803c gRr7AidYMn+qKPjbaU/DiRxEI7admLI= X-Google-Smtp-Source: AFSGD/Vvg5BN2ru79vkh31v7//X5ILu/HZs+eAHJu30/hNospD9xdqkeZqR25JCjPUvY5uVYdZSwzA== X-Received: by 2002:a2e:3603:: with SMTP id d3-v6mr1848589lja.46.1544793535722; Fri, 14 Dec 2018 05:18:55 -0800 (PST) Received: from mkPC.semihalf.local (31-172-191-173.noc.fibertech.net.pl. [31.172.191.173]) by smtp.gmail.com with ESMTPSA id o25sm873884lfd.29.2018.12.14.05.18.54 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 14 Dec 2018 05:18:54 -0800 (PST) From: Michal Krawczyk To: dev@dpdk.org Cc: gtzalik@dpdk.org, mw@dpdk.org, matua@amazon.com, rk@semihalf.com, Michal Krawczyk Date: Fri, 14 Dec 2018 14:18:27 +0100 Message-Id: <20181214131846.22439-2-mk@semihalf.com> X-Mailer: git-send-email 2.14.1 In-Reply-To: <20181214131846.22439-1-mk@semihalf.com> References: <20181214131846.22439-1-mk@semihalf.com> Subject: [dpdk-dev] [PATCH 01/20] net/ena: update ena_com for the ENAv2 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Rafal Kozik ena_com is the communication layer provided by the vendor and common to all ENA drivers. This patch updates it to version from 2018.09.26. It adds support for ENAv2 device together with LLQ feature, adds doorbell optimization and reconfiguration of HW queues depth independently. The driver was adjusted to the new changes in the HAL. Signed-off-by: Rafal Kozik Signed-off-by: Michal Krawczyk --- drivers/net/ena/base/ena_com.c | 479 ++++++++++++++++---- drivers/net/ena/base/ena_com.h | 111 ++++- drivers/net/ena/base/ena_defs/ena_admin_defs.h | 567 ++++++++++++++++-------- drivers/net/ena/base/ena_defs/ena_common_defs.h | 5 +- drivers/net/ena/base/ena_defs/ena_eth_io_defs.h | 244 +++++----- drivers/net/ena/base/ena_defs/ena_gen_info.h | 5 +- drivers/net/ena/base/ena_defs/ena_includes.h | 33 -- drivers/net/ena/base/ena_defs/ena_regs_defs.h | 207 ++++----- drivers/net/ena/base/ena_eth_com.c | 334 +++++++++----- drivers/net/ena/base/ena_eth_com.h | 87 +++- drivers/net/ena/base/ena_plat.h | 12 +- drivers/net/ena/base/ena_plat_dpdk.h | 35 +- drivers/net/ena/ena_ethdev.c | 4 +- 13 files changed, 1462 insertions(+), 661 deletions(-) diff --git a/drivers/net/ena/base/ena_com.c b/drivers/net/ena/base/ena_com.c index 4abf1a28a..b688067f7 100644 --- a/drivers/net/ena/base/ena_com.c +++ b/drivers/net/ena/base/ena_com.c @@ -42,17 +42,6 @@ #define ENA_ASYNC_QUEUE_DEPTH 16 #define ENA_ADMIN_QUEUE_DEPTH 32 -#ifdef ENA_EXTENDED_STATS - -#define ENA_HISTOGRAM_ACTIVE_MASK_OFFSET 0xF08 -#define ENA_EXTENDED_STAT_GET_FUNCT(_funct_queue) (_funct_queue & 0xFFFF) -#define ENA_EXTENDED_STAT_GET_QUEUE(_funct_queue) (_funct_queue >> 16) - -#endif /* ENA_EXTENDED_STATS */ - -#define MIN_ENA_VER (((ENA_COMMON_SPEC_VERSION_MAJOR) << \ - ENA_REGS_VERSION_MAJOR_VERSION_SHIFT) \ - | (ENA_COMMON_SPEC_VERSION_MINOR)) #define ENA_CTRL_MAJOR 0 #define ENA_CTRL_MINOR 0 @@ -70,6 +59,8 @@ #define ENA_MMIO_READ_TIMEOUT 0xFFFFFFFF +#define ENA_COM_BOUNCE_BUFFER_CNTRL_CNT 4 + #define ENA_REGS_ADMIN_INTR_MASK 1 #define ENA_POLL_MS 5 @@ -247,7 +238,7 @@ static struct ena_comp_ctx *__ena_com_submit_admin_cmd(struct ena_com_admin_queu tail_masked = admin_queue->sq.tail & queue_size_mask; /* In case of queue FULL */ - cnt = ATOMIC32_READ(&admin_queue->outstanding_cmds); + cnt = (u16)ATOMIC32_READ(&admin_queue->outstanding_cmds); if (cnt >= admin_queue->q_depth) { ena_trc_dbg("admin queue is full.\n"); admin_queue->stats.out_of_space++; @@ -284,6 +275,7 @@ static struct ena_comp_ctx *__ena_com_submit_admin_cmd(struct ena_com_admin_queu if (unlikely((admin_queue->sq.tail & queue_size_mask) == 0)) admin_queue->sq.phase = !admin_queue->sq.phase; + ENA_DB_SYNC(&admin_queue->sq.mem_handle); ENA_REG_WRITE32(admin_queue->bus, admin_queue->sq.tail, admin_queue->sq.db_addr); @@ -345,12 +337,14 @@ static int ena_com_init_io_sq(struct ena_com_dev *ena_dev, memset(&io_sq->desc_addr, 0x0, sizeof(io_sq->desc_addr)); + io_sq->dma_addr_bits = (u8)ena_dev->dma_addr_bits; io_sq->desc_entry_size = (io_sq->direction == ENA_COM_IO_QUEUE_DIRECTION_TX) ? sizeof(struct ena_eth_io_tx_desc) : sizeof(struct ena_eth_io_rx_desc); size = io_sq->desc_entry_size * io_sq->q_depth; + io_sq->bus = ena_dev->bus; if (io_sq->mem_queue_type == ENA_ADMIN_PLACEMENT_POLICY_HOST) { ENA_MEM_ALLOC_COHERENT_NODE(ena_dev->dmadev, @@ -367,21 +361,47 @@ static int ena_com_init_io_sq(struct ena_com_dev *ena_dev, io_sq->desc_addr.phys_addr, io_sq->desc_addr.mem_handle); } - } else { + + if (!io_sq->desc_addr.virt_addr) { + ena_trc_err("memory allocation failed"); + return ENA_COM_NO_MEM; + } + } + + if (io_sq->mem_queue_type == ENA_ADMIN_PLACEMENT_POLICY_DEV) { + /* Allocate bounce buffers */ + io_sq->bounce_buf_ctrl.buffer_size = ena_dev->llq_info.desc_list_entry_size; + io_sq->bounce_buf_ctrl.buffers_num = ENA_COM_BOUNCE_BUFFER_CNTRL_CNT; + io_sq->bounce_buf_ctrl.next_to_use = 0; + + size = io_sq->bounce_buf_ctrl.buffer_size * io_sq->bounce_buf_ctrl.buffers_num; + ENA_MEM_ALLOC_NODE(ena_dev->dmadev, size, - io_sq->desc_addr.virt_addr, + io_sq->bounce_buf_ctrl.base_buffer, ctx->numa_node, dev_node); - if (!io_sq->desc_addr.virt_addr) { - io_sq->desc_addr.virt_addr = - ENA_MEM_ALLOC(ena_dev->dmadev, size); + if (!io_sq->bounce_buf_ctrl.base_buffer) + io_sq->bounce_buf_ctrl.base_buffer = ENA_MEM_ALLOC(ena_dev->dmadev, size); + + if (!io_sq->bounce_buf_ctrl.base_buffer) { + ena_trc_err("bounce buffer memory allocation failed"); + return ENA_COM_NO_MEM; } - } - if (!io_sq->desc_addr.virt_addr) { - ena_trc_err("memory allocation failed"); - return ENA_COM_NO_MEM; + memcpy(&io_sq->llq_info, &ena_dev->llq_info, sizeof(io_sq->llq_info)); + + /* Initiate the first bounce buffer */ + io_sq->llq_buf_ctrl.curr_bounce_buf = + ena_com_get_next_bounce_buffer(&io_sq->bounce_buf_ctrl); + memset(io_sq->llq_buf_ctrl.curr_bounce_buf, + 0x0, io_sq->llq_info.desc_list_entry_size); + io_sq->llq_buf_ctrl.descs_left_in_line = + io_sq->llq_info.descs_num_before_header; + + if (io_sq->llq_info.max_entries_in_tx_burst > 0) + io_sq->entries_in_tx_burst_left = + io_sq->llq_info.max_entries_in_tx_burst; } io_sq->tail = 0; @@ -407,6 +427,7 @@ static int ena_com_init_io_cq(struct ena_com_dev *ena_dev, sizeof(struct ena_eth_io_rx_cdesc_base); size = io_cq->cdesc_entry_size_in_bytes * io_cq->q_depth; + io_cq->bus = ena_dev->bus; ENA_MEM_ALLOC_COHERENT_NODE(ena_dev->dmadev, size, @@ -473,12 +494,12 @@ static void ena_com_handle_admin_completion(struct ena_com_admin_queue *admin_qu cqe = &admin_queue->cq.entries[head_masked]; /* Go over all the completions */ - while ((cqe->acq_common_descriptor.flags & + while ((READ_ONCE8(cqe->acq_common_descriptor.flags) & ENA_ADMIN_ACQ_COMMON_DESC_PHASE_MASK) == phase) { /* Do not read the rest of the completion entry before the * phase bit was validated */ - rmb(); + dma_rmb(); ena_com_handle_single_admin_completion(admin_queue, cqe); head_masked++; @@ -572,6 +593,168 @@ static int ena_com_wait_and_process_admin_cq_polling(struct ena_comp_ctx *comp_c return ret; } +/** + * Set the LLQ configurations of the firmware + * + * The driver provides only the enabled feature values to the FW, + * which in turn, checks if they are supported. + */ +static int ena_com_set_llq(struct ena_com_dev *ena_dev) +{ + struct ena_com_admin_queue *admin_queue; + struct ena_admin_set_feat_cmd cmd; + struct ena_admin_set_feat_resp resp; + struct ena_com_llq_info *llq_info = &ena_dev->llq_info; + int ret; + + memset(&cmd, 0x0, sizeof(cmd)); + admin_queue = &ena_dev->admin_queue; + + cmd.aq_common_descriptor.opcode = ENA_ADMIN_SET_FEATURE; + cmd.feat_common.feature_id = ENA_ADMIN_LLQ; + + cmd.u.llq.header_location_ctrl_enabled = llq_info->header_location_ctrl; + cmd.u.llq.entry_size_ctrl_enabled = llq_info->desc_list_entry_size_ctrl; + cmd.u.llq.desc_num_before_header_enabled = llq_info->descs_num_before_header; + cmd.u.llq.descriptors_stride_ctrl_enabled = llq_info->desc_stride_ctrl; + + ret = ena_com_execute_admin_command(admin_queue, + (struct ena_admin_aq_entry *)&cmd, + sizeof(cmd), + (struct ena_admin_acq_entry *)&resp, + sizeof(resp)); + + if (unlikely(ret)) + ena_trc_err("Failed to set LLQ configurations: %d\n", ret); + + return ret; +} + +static int ena_com_config_llq_info(struct ena_com_dev *ena_dev, + struct ena_admin_feature_llq_desc *llq_features, + struct ena_llq_configurations *llq_default_cfg) +{ + struct ena_com_llq_info *llq_info = &ena_dev->llq_info; + u16 supported_feat; + int rc; + + memset(llq_info, 0, sizeof(*llq_info)); + + supported_feat = llq_features->header_location_ctrl_supported; + + if (likely(supported_feat & llq_default_cfg->llq_header_location)) { + llq_info->header_location_ctrl = llq_default_cfg->llq_header_location; + } else { + ena_trc_err("Invalid header location control, supported: 0x%x\n", + supported_feat); + return -EINVAL; + } + + if (likely(llq_info->header_location_ctrl == ENA_ADMIN_INLINE_HEADER)) { + llq_info->inline_header = true; + + supported_feat = llq_features->descriptors_stride_ctrl_supported; + if (likely(supported_feat & llq_default_cfg->llq_stride_ctrl)) { + llq_info->desc_stride_ctrl = llq_default_cfg->llq_stride_ctrl; + } else { + if (supported_feat & ENA_ADMIN_MULTIPLE_DESCS_PER_ENTRY) { + llq_info->desc_stride_ctrl = ENA_ADMIN_MULTIPLE_DESCS_PER_ENTRY; + } else if (supported_feat & ENA_ADMIN_SINGLE_DESC_PER_ENTRY) { + llq_info->desc_stride_ctrl = ENA_ADMIN_SINGLE_DESC_PER_ENTRY; + } else { + ena_trc_err("Invalid desc_stride_ctrl, supported: 0x%x\n", + supported_feat); + return -EINVAL; + } + + ena_trc_err("Default llq stride ctrl is not supported, performing fallback," + "default: 0x%x, supported: 0x%x, used: 0x%x\n", + llq_default_cfg->llq_stride_ctrl, + supported_feat, + llq_info->desc_stride_ctrl); + } + } else { + llq_info->inline_header = false; + llq_info->desc_stride_ctrl = 0; + } + + supported_feat = llq_features->entry_size_ctrl_supported; + if (likely(supported_feat & llq_default_cfg->llq_ring_entry_size)) { + llq_info->desc_list_entry_size_ctrl = llq_default_cfg->llq_ring_entry_size; + llq_info->desc_list_entry_size = llq_default_cfg->llq_ring_entry_size_value; + } else { + if (supported_feat & ENA_ADMIN_LIST_ENTRY_SIZE_128B) { + llq_info->desc_list_entry_size_ctrl = ENA_ADMIN_LIST_ENTRY_SIZE_128B; + llq_info->desc_list_entry_size = 128; + } else if (supported_feat & ENA_ADMIN_LIST_ENTRY_SIZE_192B) { + llq_info->desc_list_entry_size_ctrl = ENA_ADMIN_LIST_ENTRY_SIZE_192B; + llq_info->desc_list_entry_size = 192; + } else if (supported_feat & ENA_ADMIN_LIST_ENTRY_SIZE_256B) { + llq_info->desc_list_entry_size_ctrl = ENA_ADMIN_LIST_ENTRY_SIZE_256B; + llq_info->desc_list_entry_size = 256; + } else { + ena_trc_err("Invalid entry_size_ctrl, supported: 0x%x\n", supported_feat); + return -EINVAL; + } + + ena_trc_err("Default llq ring entry size is not supported, performing fallback," + "default: 0x%x, supported: 0x%x, used: 0x%x\n", + llq_default_cfg->llq_ring_entry_size, + supported_feat, + llq_info->desc_list_entry_size); + } + if (unlikely(llq_info->desc_list_entry_size & 0x7)) { + /* The desc list entry size should be whole multiply of 8 + * This requirement comes from __iowrite64_copy() + */ + ena_trc_err("illegal entry size %d\n", + llq_info->desc_list_entry_size); + return -EINVAL; + } + + if (llq_info->desc_stride_ctrl == ENA_ADMIN_MULTIPLE_DESCS_PER_ENTRY) + llq_info->descs_per_entry = llq_info->desc_list_entry_size / + sizeof(struct ena_eth_io_tx_desc); + else + llq_info->descs_per_entry = 1; + + supported_feat = llq_features->desc_num_before_header_supported; + if (likely(supported_feat & llq_default_cfg->llq_num_decs_before_header)) { + llq_info->descs_num_before_header = llq_default_cfg->llq_num_decs_before_header; + } else { + if (supported_feat & ENA_ADMIN_LLQ_NUM_DESCS_BEFORE_HEADER_2) { + llq_info->descs_num_before_header = ENA_ADMIN_LLQ_NUM_DESCS_BEFORE_HEADER_2; + } else if (supported_feat & ENA_ADMIN_LLQ_NUM_DESCS_BEFORE_HEADER_1) { + llq_info->descs_num_before_header = ENA_ADMIN_LLQ_NUM_DESCS_BEFORE_HEADER_1; + } else if (supported_feat & ENA_ADMIN_LLQ_NUM_DESCS_BEFORE_HEADER_4) { + llq_info->descs_num_before_header = ENA_ADMIN_LLQ_NUM_DESCS_BEFORE_HEADER_4; + } else if (supported_feat & ENA_ADMIN_LLQ_NUM_DESCS_BEFORE_HEADER_8) { + llq_info->descs_num_before_header = ENA_ADMIN_LLQ_NUM_DESCS_BEFORE_HEADER_8; + } else { + ena_trc_err("Invalid descs_num_before_header, supported: 0x%x\n", + supported_feat); + return -EINVAL; + } + + ena_trc_err("Default llq num descs before header is not supported, performing fallback," + "default: 0x%x, supported: 0x%x, used: 0x%x\n", + llq_default_cfg->llq_num_decs_before_header, + supported_feat, + llq_info->descs_num_before_header); + } + + llq_info->max_entries_in_tx_burst = + (u16)(llq_features->max_tx_burst_size / llq_default_cfg->llq_ring_entry_size_value); + + rc = ena_com_set_llq(ena_dev); + if (rc) + ena_trc_err("Cannot set LLQ configuration: %d\n", rc); + + return 0; +} + + + static int ena_com_wait_and_process_admin_cq_interrupts(struct ena_comp_ctx *comp_ctx, struct ena_com_admin_queue *admin_queue) { @@ -641,15 +824,11 @@ static u32 ena_com_reg_bar_read32(struct ena_com_dev *ena_dev, u16 offset) mmio_read_reg |= mmio_read->seq_num & ENA_REGS_MMIO_REG_READ_REQ_ID_MASK; - /* make sure read_resp->req_id get updated before the hw can write - * there - */ - wmb(); - - ENA_REG_WRITE32(ena_dev->bus, mmio_read_reg, ena_dev->reg_bar + ENA_REGS_MMIO_REG_READ_OFF); + ENA_REG_WRITE32(ena_dev->bus, mmio_read_reg, + ena_dev->reg_bar + ENA_REGS_MMIO_REG_READ_OFF); for (i = 0; i < timeout; i++) { - if (read_resp->req_id == mmio_read->seq_num) + if (READ_ONCE16(read_resp->req_id) == mmio_read->seq_num) break; ENA_UDELAY(1); @@ -751,17 +930,20 @@ static void ena_com_io_queue_free(struct ena_com_dev *ena_dev, if (io_sq->desc_addr.virt_addr) { size = io_sq->desc_entry_size * io_sq->q_depth; - if (io_sq->mem_queue_type == ENA_ADMIN_PLACEMENT_POLICY_HOST) - ENA_MEM_FREE_COHERENT(ena_dev->dmadev, - size, - io_sq->desc_addr.virt_addr, - io_sq->desc_addr.phys_addr, - io_sq->desc_addr.mem_handle); - else - ENA_MEM_FREE(ena_dev->dmadev, io_sq->desc_addr.virt_addr); + ENA_MEM_FREE_COHERENT(ena_dev->dmadev, + size, + io_sq->desc_addr.virt_addr, + io_sq->desc_addr.phys_addr, + io_sq->desc_addr.mem_handle); io_sq->desc_addr.virt_addr = NULL; } + + if (io_sq->bounce_buf_ctrl.base_buffer) { + size = io_sq->llq_info.desc_list_entry_size * ENA_COM_BOUNCE_BUFFER_CNTRL_CNT; + ENA_MEM_FREE(ena_dev->dmadev, io_sq->bounce_buf_ctrl.base_buffer); + io_sq->bounce_buf_ctrl.base_buffer = NULL; + } } static int wait_for_reset_state(struct ena_com_dev *ena_dev, u32 timeout, @@ -807,7 +989,8 @@ static int ena_com_get_feature_ex(struct ena_com_dev *ena_dev, struct ena_admin_get_feat_resp *get_resp, enum ena_admin_aq_feature_id feature_id, dma_addr_t control_buf_dma_addr, - u32 control_buff_size) + u32 control_buff_size, + u8 feature_ver) { struct ena_com_admin_queue *admin_queue; struct ena_admin_get_feat_cmd get_cmd; @@ -838,7 +1021,7 @@ static int ena_com_get_feature_ex(struct ena_com_dev *ena_dev, } get_cmd.control_buffer.length = control_buff_size; - + get_cmd.feat_common.feature_version = feature_ver; get_cmd.feat_common.feature_id = feature_id; ret = ena_com_execute_admin_command(admin_queue, @@ -858,13 +1041,15 @@ static int ena_com_get_feature_ex(struct ena_com_dev *ena_dev, static int ena_com_get_feature(struct ena_com_dev *ena_dev, struct ena_admin_get_feat_resp *get_resp, - enum ena_admin_aq_feature_id feature_id) + enum ena_admin_aq_feature_id feature_id, + u8 feature_ver) { return ena_com_get_feature_ex(ena_dev, get_resp, feature_id, 0, - 0); + 0, + feature_ver); } static int ena_com_hash_key_allocate(struct ena_com_dev *ena_dev) @@ -934,7 +1119,7 @@ static int ena_com_indirect_table_allocate(struct ena_com_dev *ena_dev, int ret; ret = ena_com_get_feature(ena_dev, &get_resp, - ENA_ADMIN_RSS_REDIRECTION_TABLE_CONFIG); + ENA_ADMIN_RSS_REDIRECTION_TABLE_CONFIG, 0); if (unlikely(ret)) return ret; @@ -1364,14 +1549,14 @@ int ena_com_set_aenq_config(struct ena_com_dev *ena_dev, u32 groups_flag) struct ena_admin_get_feat_resp get_resp; int ret; - ret = ena_com_get_feature(ena_dev, &get_resp, ENA_ADMIN_AENQ_CONFIG); + ret = ena_com_get_feature(ena_dev, &get_resp, ENA_ADMIN_AENQ_CONFIG, 0); if (ret) { ena_trc_info("Can't get aenq configuration\n"); return ret; } if ((get_resp.u.aenq.supported_groups & groups_flag) != groups_flag) { - ena_trc_warn("Trying to set unsupported aenq events. supported flag: %x asked flag: %x\n", + ena_trc_warn("Trying to set unsupported aenq events. supported flag: 0x%x asked flag: 0x%x\n", get_resp.u.aenq.supported_groups, groups_flag); return ENA_COM_UNSUPPORTED; @@ -1446,11 +1631,6 @@ int ena_com_validate_version(struct ena_com_dev *ena_dev) ENA_REGS_VERSION_MAJOR_VERSION_SHIFT, ver & ENA_REGS_VERSION_MINOR_VERSION_MASK); - if (ver < MIN_ENA_VER) { - ena_trc_err("ENA version is lower than the minimal version the driver supports\n"); - return -1; - } - ena_trc_info("ena controller version: %d.%d.%d implementation version %d\n", (ctrl_ver & ENA_REGS_CONTROLLER_VERSION_MAJOR_VERSION_MASK) >> ENA_REGS_CONTROLLER_VERSION_MAJOR_VERSION_SHIFT, @@ -1503,6 +1683,7 @@ void ena_com_admin_destroy(struct ena_com_dev *ena_dev) ENA_MEM_FREE_COHERENT(ena_dev->dmadev, size, aenq->entries, aenq->dma_addr, aenq->mem_handle); aenq->entries = NULL; + ENA_SPINLOCK_DESTROY(admin_queue->q_lock); } void ena_com_set_admin_polling_mode(struct ena_com_dev *ena_dev, bool polling) @@ -1512,7 +1693,8 @@ void ena_com_set_admin_polling_mode(struct ena_com_dev *ena_dev, bool polling) if (polling) mask_value = ENA_REGS_ADMIN_INTR_MASK; - ENA_REG_WRITE32(ena_dev->bus, mask_value, ena_dev->reg_bar + ENA_REGS_INTR_MASK_OFF); + ENA_REG_WRITE32(ena_dev->bus, mask_value, + ena_dev->reg_bar + ENA_REGS_INTR_MASK_OFF); ena_dev->admin_queue.polling = polling; } @@ -1527,7 +1709,7 @@ int ena_com_mmio_reg_read_request_init(struct ena_com_dev *ena_dev) mmio_read->read_resp_dma_addr, mmio_read->read_resp_mem_handle); if (unlikely(!mmio_read->read_resp)) - return ENA_COM_NO_MEM; + goto err; ena_com_mmio_reg_read_request_write_dev_addr(ena_dev); @@ -1536,6 +1718,10 @@ int ena_com_mmio_reg_read_request_init(struct ena_com_dev *ena_dev) mmio_read->readless_supported = true; return 0; + +err: + ENA_SPINLOCK_DESTROY(mmio_read->lock); + return ENA_COM_NO_MEM; } void ena_com_set_mmio_read_mode(struct ena_com_dev *ena_dev, bool readless_supported) @@ -1559,6 +1745,7 @@ void ena_com_mmio_reg_read_request_destroy(struct ena_com_dev *ena_dev) mmio_read->read_resp_mem_handle); mmio_read->read_resp = NULL; + ENA_SPINLOCK_DESTROY(mmio_read->lock); } void ena_com_mmio_reg_read_request_write_dev_addr(struct ena_com_dev *ena_dev) @@ -1574,8 +1761,7 @@ void ena_com_mmio_reg_read_request_write_dev_addr(struct ena_com_dev *ena_dev) } int ena_com_admin_init(struct ena_com_dev *ena_dev, - struct ena_aenq_handlers *aenq_handlers, - bool init_spinlock) + struct ena_aenq_handlers *aenq_handlers) { struct ena_com_admin_queue *admin_queue = &ena_dev->admin_queue; u32 aq_caps, acq_caps, dev_sts, addr_low, addr_high; @@ -1595,14 +1781,14 @@ int ena_com_admin_init(struct ena_com_dev *ena_dev, admin_queue->q_depth = ENA_ADMIN_QUEUE_DEPTH; + admin_queue->bus = ena_dev->bus; admin_queue->q_dmadev = ena_dev->dmadev; admin_queue->polling = false; admin_queue->curr_cmd_id = 0; ATOMIC32_SET(&admin_queue->outstanding_cmds, 0); - if (init_spinlock) - ENA_SPINLOCK_INIT(admin_queue->q_lock); + ENA_SPINLOCK_INIT(admin_queue->q_lock); ret = ena_com_init_comp_ctxt(admin_queue); if (ret) @@ -1742,7 +1928,63 @@ void ena_com_destroy_io_queue(struct ena_com_dev *ena_dev, u16 qid) int ena_com_get_link_params(struct ena_com_dev *ena_dev, struct ena_admin_get_feat_resp *resp) { - return ena_com_get_feature(ena_dev, resp, ENA_ADMIN_LINK_CONFIG); + return ena_com_get_feature(ena_dev, resp, ENA_ADMIN_LINK_CONFIG, 0); +} + +int ena_com_extra_properties_strings_init(struct ena_com_dev *ena_dev) +{ + struct ena_admin_get_feat_resp resp; + struct ena_extra_properties_strings *extra_properties_strings = + &ena_dev->extra_properties_strings; + u32 rc; + extra_properties_strings->size = ENA_ADMIN_EXTRA_PROPERTIES_COUNT * + ENA_ADMIN_EXTRA_PROPERTIES_STRING_LEN; + + ENA_MEM_ALLOC_COHERENT(ena_dev->dmadev, + extra_properties_strings->size, + extra_properties_strings->virt_addr, + extra_properties_strings->dma_addr, + extra_properties_strings->dma_handle); + if (unlikely(!extra_properties_strings->virt_addr)) { + ena_trc_err("Failed to allocate extra properties strings\n"); + return 0; + } + + rc = ena_com_get_feature_ex(ena_dev, &resp, + ENA_ADMIN_EXTRA_PROPERTIES_STRINGS, + extra_properties_strings->dma_addr, + extra_properties_strings->size, 0); + if (rc) { + ena_trc_dbg("Failed to get extra properties strings\n"); + goto err; + } + + return resp.u.extra_properties_strings.count; +err: + ena_com_delete_extra_properties_strings(ena_dev); + return 0; +} + +void ena_com_delete_extra_properties_strings(struct ena_com_dev *ena_dev) +{ + struct ena_extra_properties_strings *extra_properties_strings = + &ena_dev->extra_properties_strings; + + if (extra_properties_strings->virt_addr) { + ENA_MEM_FREE_COHERENT(ena_dev->dmadev, + extra_properties_strings->size, + extra_properties_strings->virt_addr, + extra_properties_strings->dma_addr, + extra_properties_strings->dma_handle); + extra_properties_strings->virt_addr = NULL; + } +} + +int ena_com_get_extra_properties_flags(struct ena_com_dev *ena_dev, + struct ena_admin_get_feat_resp *resp) +{ + return ena_com_get_feature(ena_dev, resp, + ENA_ADMIN_EXTRA_PROPERTIES_FLAGS, 0); } int ena_com_get_dev_attr_feat(struct ena_com_dev *ena_dev, @@ -1752,7 +1994,7 @@ int ena_com_get_dev_attr_feat(struct ena_com_dev *ena_dev, int rc; rc = ena_com_get_feature(ena_dev, &get_resp, - ENA_ADMIN_DEVICE_ATTRIBUTES); + ENA_ADMIN_DEVICE_ATTRIBUTES, 0); if (rc) return rc; @@ -1760,17 +2002,34 @@ int ena_com_get_dev_attr_feat(struct ena_com_dev *ena_dev, sizeof(get_resp.u.dev_attr)); ena_dev->supported_features = get_resp.u.dev_attr.supported_features; - rc = ena_com_get_feature(ena_dev, &get_resp, - ENA_ADMIN_MAX_QUEUES_NUM); - if (rc) - return rc; + if (ena_dev->supported_features & BIT(ENA_ADMIN_MAX_QUEUES_EXT)) { + rc = ena_com_get_feature(ena_dev, &get_resp, + ENA_ADMIN_MAX_QUEUES_EXT, + ENA_FEATURE_MAX_QUEUE_EXT_VER); + if (rc) + return rc; + + if (get_resp.u.max_queue_ext.version != ENA_FEATURE_MAX_QUEUE_EXT_VER) + return -EINVAL; + + memcpy(&get_feat_ctx->max_queue_ext, &get_resp.u.max_queue_ext, + sizeof(get_resp.u.max_queue_ext)); + ena_dev->tx_max_header_size = + get_resp.u.max_queue_ext.max_queue_ext.max_tx_header_size; + } else { + rc = ena_com_get_feature(ena_dev, &get_resp, + ENA_ADMIN_MAX_QUEUES_NUM, 0); + memcpy(&get_feat_ctx->max_queues, &get_resp.u.max_queue, + sizeof(get_resp.u.max_queue)); + ena_dev->tx_max_header_size = + get_resp.u.max_queue.max_header_size; - memcpy(&get_feat_ctx->max_queues, &get_resp.u.max_queue, - sizeof(get_resp.u.max_queue)); - ena_dev->tx_max_header_size = get_resp.u.max_queue.max_header_size; + if (rc) + return rc; + } rc = ena_com_get_feature(ena_dev, &get_resp, - ENA_ADMIN_AENQ_CONFIG); + ENA_ADMIN_AENQ_CONFIG, 0); if (rc) return rc; @@ -1778,7 +2037,7 @@ int ena_com_get_dev_attr_feat(struct ena_com_dev *ena_dev, sizeof(get_resp.u.aenq)); rc = ena_com_get_feature(ena_dev, &get_resp, - ENA_ADMIN_STATELESS_OFFLOAD_CONFIG); + ENA_ADMIN_STATELESS_OFFLOAD_CONFIG, 0); if (rc) return rc; @@ -1788,7 +2047,7 @@ int ena_com_get_dev_attr_feat(struct ena_com_dev *ena_dev, /* Driver hints isn't mandatory admin command. So in case the * command isn't supported set driver hints to 0 */ - rc = ena_com_get_feature(ena_dev, &get_resp, ENA_ADMIN_HW_HINTS); + rc = ena_com_get_feature(ena_dev, &get_resp, ENA_ADMIN_HW_HINTS, 0); if (!rc) memcpy(&get_feat_ctx->hw_hints, &get_resp.u.hw_hints, @@ -1798,6 +2057,26 @@ int ena_com_get_dev_attr_feat(struct ena_com_dev *ena_dev, else return rc; + rc = ena_com_get_feature(ena_dev, &get_resp, ENA_ADMIN_LLQ, 0); + if (!rc) + memcpy(&get_feat_ctx->llq, &get_resp.u.llq, + sizeof(get_resp.u.llq)); + else if (rc == ENA_COM_UNSUPPORTED) + memset(&get_feat_ctx->llq, 0x0, sizeof(get_feat_ctx->llq)); + else + return rc; + + rc = ena_com_get_feature(ena_dev, &get_resp, + ENA_ADMIN_RSS_REDIRECTION_TABLE_CONFIG, 0); + if (!rc) + memcpy(&get_feat_ctx->ind_table, &get_resp.u.ind_table, + sizeof(get_resp.u.ind_table)); + else if (rc == ENA_COM_UNSUPPORTED) + memset(&get_feat_ctx->ind_table, 0x0, + sizeof(get_feat_ctx->ind_table)); + else + return rc; + return 0; } @@ -1829,8 +2108,8 @@ void ena_com_aenq_intr_handler(struct ena_com_dev *dev, void *data) struct ena_admin_aenq_entry *aenq_e; struct ena_admin_aenq_common_desc *aenq_common; struct ena_com_aenq *aenq = &dev->aenq; - ena_aenq_handler handler_cb; unsigned long long timestamp; + ena_aenq_handler handler_cb; u16 masked_head, processed = 0; u8 phase; @@ -1840,8 +2119,13 @@ void ena_com_aenq_intr_handler(struct ena_com_dev *dev, void *data) aenq_common = &aenq_e->aenq_common_desc; /* Go over all the events */ - while ((aenq_common->flags & ENA_ADMIN_AENQ_COMMON_DESC_PHASE_MASK) == - phase) { + while ((READ_ONCE8(aenq_common->flags) & + ENA_ADMIN_AENQ_COMMON_DESC_PHASE_MASK) == phase) { + /* Make sure the phase bit (ownership) is as expected before + * reading the rest of the descriptor. + */ + dma_rmb(); + timestamp = (unsigned long long)aenq_common->timestamp_low | ((unsigned long long)aenq_common->timestamp_high << 32); ENA_TOUCH(timestamp); /* In case debug is disabled */ @@ -1876,7 +2160,9 @@ void ena_com_aenq_intr_handler(struct ena_com_dev *dev, void *data) /* write the aenq doorbell after all AENQ descriptors were read */ mb(); - ENA_REG_WRITE32(dev->bus, (u32)aenq->head, dev->reg_bar + ENA_REGS_AENQ_HEAD_DB_OFF); + ENA_REG_WRITE32_RELAXED(dev->bus, (u32)aenq->head, + dev->reg_bar + ENA_REGS_AENQ_HEAD_DB_OFF); + mmiowb(); } int ena_com_dev_reset(struct ena_com_dev *ena_dev, @@ -2022,7 +2308,7 @@ int ena_com_get_offload_settings(struct ena_com_dev *ena_dev, struct ena_admin_get_feat_resp resp; ret = ena_com_get_feature(ena_dev, &resp, - ENA_ADMIN_STATELESS_OFFLOAD_CONFIG); + ENA_ADMIN_STATELESS_OFFLOAD_CONFIG, 0); if (unlikely(ret)) { ena_trc_err("Failed to get offload capabilities %d\n", ret); return ret; @@ -2051,7 +2337,7 @@ int ena_com_set_hash_function(struct ena_com_dev *ena_dev) /* Validate hash function is supported */ ret = ena_com_get_feature(ena_dev, &get_resp, - ENA_ADMIN_RSS_HASH_FUNCTION); + ENA_ADMIN_RSS_HASH_FUNCTION, 0); if (unlikely(ret)) return ret; @@ -2111,7 +2397,7 @@ int ena_com_fill_hash_function(struct ena_com_dev *ena_dev, rc = ena_com_get_feature_ex(ena_dev, &get_resp, ENA_ADMIN_RSS_HASH_FUNCTION, rss->hash_key_dma_addr, - sizeof(*rss->hash_key)); + sizeof(*rss->hash_key), 0); if (unlikely(rc)) return rc; @@ -2162,7 +2448,7 @@ int ena_com_get_hash_function(struct ena_com_dev *ena_dev, rc = ena_com_get_feature_ex(ena_dev, &get_resp, ENA_ADMIN_RSS_HASH_FUNCTION, rss->hash_key_dma_addr, - sizeof(*rss->hash_key)); + sizeof(*rss->hash_key), 0); if (unlikely(rc)) return rc; @@ -2187,7 +2473,7 @@ int ena_com_get_hash_ctrl(struct ena_com_dev *ena_dev, rc = ena_com_get_feature_ex(ena_dev, &get_resp, ENA_ADMIN_RSS_HASH_INPUT, rss->hash_ctrl_dma_addr, - sizeof(*rss->hash_ctrl)); + sizeof(*rss->hash_ctrl), 0); if (unlikely(rc)) return rc; @@ -2423,7 +2709,7 @@ int ena_com_indirect_table_get(struct ena_com_dev *ena_dev, u32 *ind_tbl) rc = ena_com_get_feature_ex(ena_dev, &get_resp, ENA_ADMIN_RSS_REDIRECTION_TABLE_CONFIG, rss->rss_ind_tbl_dma_addr, - tbl_size); + tbl_size, 0); if (unlikely(rc)) return rc; @@ -2490,6 +2776,10 @@ int ena_com_allocate_host_info(struct ena_com_dev *ena_dev) if (unlikely(!host_attr->host_info)) return ENA_COM_NO_MEM; + host_attr->host_info->ena_spec_version = ((ENA_COMMON_SPEC_VERSION_MAJOR << + ENA_REGS_VERSION_MAJOR_VERSION_SHIFT) | + (ENA_COMMON_SPEC_VERSION_MINOR)); + return 0; } @@ -2642,7 +2932,7 @@ int ena_com_init_interrupt_moderation(struct ena_com_dev *ena_dev) int rc; rc = ena_com_get_feature(ena_dev, &get_resp, - ENA_ADMIN_INTERRUPT_MODERATION); + ENA_ADMIN_INTERRUPT_MODERATION, 0); if (rc) { if (rc == ENA_COM_UNSUPPORTED) { @@ -2768,3 +3058,34 @@ void ena_com_get_intr_moderation_entry(struct ena_com_dev *ena_dev, intr_moder_tbl[level].pkts_per_interval; entry->bytes_per_interval = intr_moder_tbl[level].bytes_per_interval; } + +int ena_com_config_dev_mode(struct ena_com_dev *ena_dev, + struct ena_admin_feature_llq_desc *llq_features, + struct ena_llq_configurations *llq_default_cfg) +{ + int rc; + int size; + + if (!llq_features->max_llq_num) { + ena_dev->tx_mem_queue_type = ENA_ADMIN_PLACEMENT_POLICY_HOST; + return 0; + } + + rc = ena_com_config_llq_info(ena_dev, llq_features, llq_default_cfg); + if (rc) + return rc; + + /* Validate the descriptor is not too big */ + size = ena_dev->tx_max_header_size; + size += ena_dev->llq_info.descs_num_before_header * + sizeof(struct ena_eth_io_tx_desc); + + if (unlikely(ena_dev->llq_info.desc_list_entry_size < size)) { + ena_trc_err("the size of the LLQ entry is smaller than needed\n"); + return ENA_COM_INVAL; + } + + ena_dev->tx_mem_queue_type = ENA_ADMIN_PLACEMENT_POLICY_DEV; + + return 0; +} diff --git a/drivers/net/ena/base/ena_com.h b/drivers/net/ena/base/ena_com.h index f58cd86a8..fd480f428 100644 --- a/drivers/net/ena/base/ena_com.h +++ b/drivers/net/ena/base/ena_com.h @@ -80,12 +80,13 @@ #define ENA_INTR_INITIAL_RX_INTERVAL_USECS 4 #define ENA_INTR_DELAY_OLD_VALUE_WEIGHT 6 #define ENA_INTR_DELAY_NEW_VALUE_WEIGHT 4 - #define ENA_INTR_MODER_LEVEL_STRIDE 1 #define ENA_INTR_BYTE_COUNT_NOT_SUPPORTED 0xFFFFFF #define ENA_HW_HINTS_NO_TIMEOUT 0xFFFF +#define ENA_FEATURE_MAX_QUEUE_EXT_VER 1 + enum ena_intr_moder_level { ENA_INTR_MODER_LOWEST = 0, ENA_INTR_MODER_LOW, @@ -95,6 +96,14 @@ enum ena_intr_moder_level { ENA_INTR_MAX_NUM_OF_LEVELS, }; +struct ena_llq_configurations { + enum ena_admin_llq_header_location llq_header_location; + enum ena_admin_llq_ring_entry_size llq_ring_entry_size; + enum ena_admin_llq_stride_ctrl llq_stride_ctrl; + enum ena_admin_llq_num_descs_before_header llq_num_decs_before_header; + u16 llq_ring_entry_size_value; +}; + struct ena_intr_moder_entry { unsigned int intr_moder_interval; unsigned int pkts_per_interval; @@ -130,6 +139,17 @@ struct ena_com_tx_meta { u16 l4_hdr_len; /* In words */ }; +struct ena_com_llq_info { + bool inline_header; + u16 header_location_ctrl; + u16 desc_stride_ctrl; + u16 desc_list_entry_size_ctrl; + u16 desc_list_entry_size; + u16 descs_num_before_header; + u16 descs_per_entry; + u16 max_entries_in_tx_burst; +}; + struct ena_com_io_cq { struct ena_com_io_desc_addr cdesc_addr; void *bus; @@ -168,6 +188,20 @@ struct ena_com_io_cq { } ____cacheline_aligned; +struct ena_com_io_bounce_buffer_control { + u8 *base_buffer; + u16 next_to_use; + u16 buffer_size; + u16 buffers_num; /* Must be a power of 2 */ +}; + +/* This struct is to keep tracking the current location of the next llq entry */ +struct ena_com_llq_pkt_ctrl { + u8 *curr_bounce_buf; + u16 idx; + u16 descs_left_in_line; +}; + struct ena_com_io_sq { struct ena_com_io_desc_addr desc_addr; void *bus; @@ -180,6 +214,9 @@ struct ena_com_io_sq { u32 msix_vector; struct ena_com_tx_meta cached_tx_meta; + struct ena_com_llq_info llq_info; + struct ena_com_llq_pkt_ctrl llq_buf_ctrl; + struct ena_com_io_bounce_buffer_control bounce_buf_ctrl; u16 q_depth; u16 qid; @@ -187,10 +224,12 @@ struct ena_com_io_sq { u16 idx; u16 tail; u16 next_to_comp; + u16 llq_last_copy_tail; u32 tx_max_header_size; u8 phase; u8 desc_entry_size; u8 dma_addr_bits; + u16 entries_in_tx_burst_left; } ____cacheline_aligned; struct ena_com_admin_cq { @@ -308,6 +347,13 @@ struct ena_host_attribute { ena_mem_handle_t host_info_dma_handle; }; +struct ena_extra_properties_strings { + u8 *virt_addr; + dma_addr_t dma_addr; + ena_mem_handle_t dma_handle; + u32 size; +}; + /* Each ena_dev is a PCI function. */ struct ena_com_dev { struct ena_com_admin_queue admin_queue; @@ -335,14 +381,20 @@ struct ena_com_dev { u16 intr_delay_resolution; u32 intr_moder_tx_interval; struct ena_intr_moder_entry *intr_moder_tbl; + + struct ena_com_llq_info llq_info; + struct ena_extra_properties_strings extra_properties_strings; }; struct ena_com_dev_get_features_ctx { struct ena_admin_queue_feature_desc max_queues; + struct ena_admin_queue_ext_feature_desc max_queue_ext; struct ena_admin_device_attr_feature_desc dev_attr; struct ena_admin_feature_aenq_desc aenq; struct ena_admin_feature_offload_desc offload; struct ena_admin_ena_hw_hints hw_hints; + struct ena_admin_feature_llq_desc llq; + struct ena_admin_feature_rss_ind_table ind_table; }; struct ena_com_create_io_ctx { @@ -401,8 +453,6 @@ void ena_com_mmio_reg_read_request_destroy(struct ena_com_dev *ena_dev); /* ena_com_admin_init - Init the admin and the async queues * @ena_dev: ENA communication layer struct * @aenq_handlers: Those handlers to be called upon event. - * @init_spinlock: Indicate if this method should init the admin spinlock or - * the spinlock was init before (for example, in a case of FLR). * * Initialize the admin submission and completion queues. * Initialize the asynchronous events notification queues. @@ -410,8 +460,7 @@ void ena_com_mmio_reg_read_request_destroy(struct ena_com_dev *ena_dev); * @return - 0 on success, negative value on failure. */ int ena_com_admin_init(struct ena_com_dev *ena_dev, - struct ena_aenq_handlers *aenq_handlers, - bool init_spinlock); + struct ena_aenq_handlers *aenq_handlers); /* ena_com_admin_destroy - Destroy the admin and the async events queues. * @ena_dev: ENA communication layer struct @@ -561,6 +610,31 @@ int ena_com_validate_version(struct ena_com_dev *ena_dev); int ena_com_get_link_params(struct ena_com_dev *ena_dev, struct ena_admin_get_feat_resp *resp); +/* ena_com_extra_properties_strings_init - Initialize the extra properties strings buffer. + * @ena_dev: ENA communication layer struct + * + * Initialize the extra properties strings buffer. + */ +int ena_com_extra_properties_strings_init(struct ena_com_dev *ena_dev); + +/* ena_com_delete_extra_properties_strings - Free the extra properties strings buffer. + * @ena_dev: ENA communication layer struct + * + * Free the allocated extra properties strings buffer. + */ +void ena_com_delete_extra_properties_strings(struct ena_com_dev *ena_dev); + +/* ena_com_get_extra_properties_flags - Retrieve extra properties flags. + * @ena_dev: ENA communication layer struct + * @resp: Extra properties flags. + * + * Retrieve the extra properties flags. + * + * @return - 0 on Success negative value otherwise. + */ +int ena_com_get_extra_properties_flags(struct ena_com_dev *ena_dev, + struct ena_admin_get_feat_resp *resp); + /* ena_com_get_dma_width - Retrieve physical dma address width the device * supports. * @ena_dev: ENA communication layer struct @@ -939,6 +1013,16 @@ void ena_com_get_intr_moderation_entry(struct ena_com_dev *ena_dev, enum ena_intr_moder_level level, struct ena_intr_moder_entry *entry); + +/* ena_com_config_dev_mode - Configure the placement policy of the device. + * @ena_dev: ENA communication layer struct + * @llq_features: LLQ feature descriptor, retrieve via ena_com_get_dev_attr_feat. + * @ena_llq_config: The default driver LLQ parameters configurations + */ +int ena_com_config_dev_mode(struct ena_com_dev *ena_dev, + struct ena_admin_feature_llq_desc *llq_features, + struct ena_llq_configurations *llq_default_config); + static inline bool ena_com_get_adaptive_moderation_enabled(struct ena_com_dev *ena_dev) { return ena_dev->adaptive_coalescing; @@ -1048,6 +1132,23 @@ static inline void ena_com_update_intr_reg(struct ena_eth_io_intr_reg *intr_reg, intr_reg->intr_control |= ENA_ETH_IO_INTR_REG_INTR_UNMASK_MASK; } +static inline u8 *ena_com_get_next_bounce_buffer(struct ena_com_io_bounce_buffer_control *bounce_buf_ctrl) +{ + u16 size, buffers_num; + u8 *buf; + + size = bounce_buf_ctrl->buffer_size; + buffers_num = bounce_buf_ctrl->buffers_num; + + buf = bounce_buf_ctrl->base_buffer + + (bounce_buf_ctrl->next_to_use++ & (buffers_num - 1)) * size; + + prefetch(bounce_buf_ctrl->base_buffer + + (bounce_buf_ctrl->next_to_use & (buffers_num - 1)) * size); + + return buf; +} + #if defined(__cplusplus) } #endif /* __cplusplus */ diff --git a/drivers/net/ena/base/ena_defs/ena_admin_defs.h b/drivers/net/ena/base/ena_defs/ena_admin_defs.h index 04d4e9a59..d97be14a4 100644 --- a/drivers/net/ena/base/ena_defs/ena_admin_defs.h +++ b/drivers/net/ena/base/ena_defs/ena_admin_defs.h @@ -30,119 +30,89 @@ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ - #ifndef _ENA_ADMIN_H_ #define _ENA_ADMIN_H_ -enum ena_admin_aq_opcode { - ENA_ADMIN_CREATE_SQ = 1, - - ENA_ADMIN_DESTROY_SQ = 2, - - ENA_ADMIN_CREATE_CQ = 3, - - ENA_ADMIN_DESTROY_CQ = 4, - - ENA_ADMIN_GET_FEATURE = 8, +#define ENA_ADMIN_EXTRA_PROPERTIES_STRING_LEN 32 +#define ENA_ADMIN_EXTRA_PROPERTIES_COUNT 32 - ENA_ADMIN_SET_FEATURE = 9, - - ENA_ADMIN_GET_STATS = 11, +enum ena_admin_aq_opcode { + ENA_ADMIN_CREATE_SQ = 1, + ENA_ADMIN_DESTROY_SQ = 2, + ENA_ADMIN_CREATE_CQ = 3, + ENA_ADMIN_DESTROY_CQ = 4, + ENA_ADMIN_GET_FEATURE = 8, + ENA_ADMIN_SET_FEATURE = 9, + ENA_ADMIN_GET_STATS = 11, }; enum ena_admin_aq_completion_status { - ENA_ADMIN_SUCCESS = 0, - - ENA_ADMIN_RESOURCE_ALLOCATION_FAILURE = 1, - - ENA_ADMIN_BAD_OPCODE = 2, - - ENA_ADMIN_UNSUPPORTED_OPCODE = 3, - - ENA_ADMIN_MALFORMED_REQUEST = 4, - + ENA_ADMIN_SUCCESS = 0, + ENA_ADMIN_RESOURCE_ALLOCATION_FAILURE = 1, + ENA_ADMIN_BAD_OPCODE = 2, + ENA_ADMIN_UNSUPPORTED_OPCODE = 3, + ENA_ADMIN_MALFORMED_REQUEST = 4, /* Additional status is provided in ACQ entry extended_status */ - ENA_ADMIN_ILLEGAL_PARAMETER = 5, - - ENA_ADMIN_UNKNOWN_ERROR = 6, + ENA_ADMIN_ILLEGAL_PARAMETER = 5, + ENA_ADMIN_UNKNOWN_ERROR = 6, + ENA_ADMIN_RESOURCE_BUSY = 7, }; enum ena_admin_aq_feature_id { - ENA_ADMIN_DEVICE_ATTRIBUTES = 1, - - ENA_ADMIN_MAX_QUEUES_NUM = 2, - - ENA_ADMIN_HW_HINTS = 3, - - ENA_ADMIN_RSS_HASH_FUNCTION = 10, - - ENA_ADMIN_STATELESS_OFFLOAD_CONFIG = 11, - - ENA_ADMIN_RSS_REDIRECTION_TABLE_CONFIG = 12, - - ENA_ADMIN_MTU = 14, - - ENA_ADMIN_RSS_HASH_INPUT = 18, - - ENA_ADMIN_INTERRUPT_MODERATION = 20, - - ENA_ADMIN_AENQ_CONFIG = 26, - - ENA_ADMIN_LINK_CONFIG = 27, - - ENA_ADMIN_HOST_ATTR_CONFIG = 28, - - ENA_ADMIN_FEATURES_OPCODE_NUM = 32, + ENA_ADMIN_DEVICE_ATTRIBUTES = 1, + ENA_ADMIN_MAX_QUEUES_NUM = 2, + ENA_ADMIN_HW_HINTS = 3, + ENA_ADMIN_LLQ = 4, + ENA_ADMIN_EXTRA_PROPERTIES_STRINGS = 5, + ENA_ADMIN_EXTRA_PROPERTIES_FLAGS = 6, + ENA_ADMIN_MAX_QUEUES_EXT = 7, + ENA_ADMIN_RSS_HASH_FUNCTION = 10, + ENA_ADMIN_STATELESS_OFFLOAD_CONFIG = 11, + ENA_ADMIN_RSS_REDIRECTION_TABLE_CONFIG = 12, + ENA_ADMIN_MTU = 14, + ENA_ADMIN_RSS_HASH_INPUT = 18, + ENA_ADMIN_INTERRUPT_MODERATION = 20, + ENA_ADMIN_AENQ_CONFIG = 26, + ENA_ADMIN_LINK_CONFIG = 27, + ENA_ADMIN_HOST_ATTR_CONFIG = 28, + ENA_ADMIN_FEATURES_OPCODE_NUM = 32, }; enum ena_admin_placement_policy_type { /* descriptors and headers are in host memory */ - ENA_ADMIN_PLACEMENT_POLICY_HOST = 1, - + ENA_ADMIN_PLACEMENT_POLICY_HOST = 1, /* descriptors and headers are in device memory (a.k.a Low Latency * Queue) */ - ENA_ADMIN_PLACEMENT_POLICY_DEV = 3, + ENA_ADMIN_PLACEMENT_POLICY_DEV = 3, }; enum ena_admin_link_types { - ENA_ADMIN_LINK_SPEED_1G = 0x1, - - ENA_ADMIN_LINK_SPEED_2_HALF_G = 0x2, - - ENA_ADMIN_LINK_SPEED_5G = 0x4, - - ENA_ADMIN_LINK_SPEED_10G = 0x8, - - ENA_ADMIN_LINK_SPEED_25G = 0x10, - - ENA_ADMIN_LINK_SPEED_40G = 0x20, - - ENA_ADMIN_LINK_SPEED_50G = 0x40, - - ENA_ADMIN_LINK_SPEED_100G = 0x80, - - ENA_ADMIN_LINK_SPEED_200G = 0x100, - - ENA_ADMIN_LINK_SPEED_400G = 0x200, + ENA_ADMIN_LINK_SPEED_1G = 0x1, + ENA_ADMIN_LINK_SPEED_2_HALF_G = 0x2, + ENA_ADMIN_LINK_SPEED_5G = 0x4, + ENA_ADMIN_LINK_SPEED_10G = 0x8, + ENA_ADMIN_LINK_SPEED_25G = 0x10, + ENA_ADMIN_LINK_SPEED_40G = 0x20, + ENA_ADMIN_LINK_SPEED_50G = 0x40, + ENA_ADMIN_LINK_SPEED_100G = 0x80, + ENA_ADMIN_LINK_SPEED_200G = 0x100, + ENA_ADMIN_LINK_SPEED_400G = 0x200, }; enum ena_admin_completion_policy_type { /* completion queue entry for each sq descriptor */ - ENA_ADMIN_COMPLETION_POLICY_DESC = 0, - + ENA_ADMIN_COMPLETION_POLICY_DESC = 0, /* completion queue entry upon request in sq descriptor */ - ENA_ADMIN_COMPLETION_POLICY_DESC_ON_DEMAND = 1, - + ENA_ADMIN_COMPLETION_POLICY_DESC_ON_DEMAND = 1, /* current queue head pointer is updated in OS memory upon sq * descriptor request */ - ENA_ADMIN_COMPLETION_POLICY_HEAD_ON_DEMAND = 2, - + ENA_ADMIN_COMPLETION_POLICY_HEAD_ON_DEMAND = 2, /* current queue head pointer is updated in OS memory for each sq * descriptor */ - ENA_ADMIN_COMPLETION_POLICY_HEAD = 3, + ENA_ADMIN_COMPLETION_POLICY_HEAD = 3, }; /* basic stats return ena_admin_basic_stats while extanded stats return a @@ -150,15 +120,13 @@ enum ena_admin_completion_policy_type { * device id */ enum ena_admin_get_stats_type { - ENA_ADMIN_GET_STATS_TYPE_BASIC = 0, - - ENA_ADMIN_GET_STATS_TYPE_EXTENDED = 1, + ENA_ADMIN_GET_STATS_TYPE_BASIC = 0, + ENA_ADMIN_GET_STATS_TYPE_EXTENDED = 1, }; enum ena_admin_get_stats_scope { - ENA_ADMIN_SPECIFIC_QUEUE = 0, - - ENA_ADMIN_ETH_TRAFFIC = 1, + ENA_ADMIN_SPECIFIC_QUEUE = 0, + ENA_ADMIN_ETH_TRAFFIC = 1, }; struct ena_admin_aq_common_desc { @@ -229,7 +197,9 @@ struct ena_admin_acq_common_desc { uint16_t extended_status; - /* serves as a hint what AQ entries can be revoked */ + /* indicates to the driver which AQ entry has been consumed by the + * device and could be reused + */ uint16_t sq_head_indx; }; @@ -298,9 +268,8 @@ struct ena_admin_aq_create_sq_cmd { }; enum ena_admin_sq_direction { - ENA_ADMIN_SQ_DIRECTION_TX = 1, - - ENA_ADMIN_SQ_DIRECTION_RX = 2, + ENA_ADMIN_SQ_DIRECTION_TX = 1, + ENA_ADMIN_SQ_DIRECTION_RX = 2, }; struct ena_admin_acq_create_sq_resp_desc { @@ -458,7 +427,13 @@ struct ena_admin_get_set_feature_common_desc { /* as appears in ena_admin_aq_feature_id */ uint8_t feature_id; - uint16_t reserved16; + /* The driver specifies the max feature version it supports and the + * device responds with the currently supported feature version. The + * field is zero based + */ + uint8_t feature_version; + + uint8_t reserved8; }; struct ena_admin_device_attr_feature_desc { @@ -485,8 +460,118 @@ struct ena_admin_device_attr_feature_desc { uint32_t max_mtu; }; +enum ena_admin_llq_header_location { + /* header is in descriptor list */ + ENA_ADMIN_INLINE_HEADER = 1, + /* header in a separate ring, implies 16B descriptor list entry */ + ENA_ADMIN_HEADER_RING = 2, +}; + +enum ena_admin_llq_ring_entry_size { + ENA_ADMIN_LIST_ENTRY_SIZE_128B = 1, + ENA_ADMIN_LIST_ENTRY_SIZE_192B = 2, + ENA_ADMIN_LIST_ENTRY_SIZE_256B = 4, +}; + +enum ena_admin_llq_num_descs_before_header { + ENA_ADMIN_LLQ_NUM_DESCS_BEFORE_HEADER_0 = 0, + ENA_ADMIN_LLQ_NUM_DESCS_BEFORE_HEADER_1 = 1, + ENA_ADMIN_LLQ_NUM_DESCS_BEFORE_HEADER_2 = 2, + ENA_ADMIN_LLQ_NUM_DESCS_BEFORE_HEADER_4 = 4, + ENA_ADMIN_LLQ_NUM_DESCS_BEFORE_HEADER_8 = 8, +}; + +/* packet descriptor list entry always starts with one or more descriptors, + * followed by a header. The rest of the descriptors are located in the + * beginning of the subsequent entry. Stride refers to how the rest of the + * descriptors are placed. This field is relevant only for inline header + * mode + */ +enum ena_admin_llq_stride_ctrl { + ENA_ADMIN_SINGLE_DESC_PER_ENTRY = 1, + ENA_ADMIN_MULTIPLE_DESCS_PER_ENTRY = 2, +}; + +struct ena_admin_feature_llq_desc { + uint32_t max_llq_num; + + uint32_t max_llq_depth; + + /* specify the header locations the device supports. bitfield of + * enum ena_admin_llq_header_location. + */ + uint16_t header_location_ctrl_supported; + + /* the header location the driver selected to use. */ + uint16_t header_location_ctrl_enabled; + + /* if inline header is specified - this is the size of descriptor + * list entry. If header in a separate ring is specified - this is + * the size of header ring entry. bitfield of enum + * ena_admin_llq_ring_entry_size. specify the entry sizes the device + * supports + */ + uint16_t entry_size_ctrl_supported; + + /* the entry size the driver selected to use. */ + uint16_t entry_size_ctrl_enabled; + + /* valid only if inline header is specified. First entry associated + * with the packet includes descriptors and header. Rest of the + * entries occupied by descriptors. This parameter defines the max + * number of descriptors precedding the header in the first entry. + * The field is bitfield of enum + * ena_admin_llq_num_descs_before_header and specify the values the + * device supports + */ + uint16_t desc_num_before_header_supported; + + /* the desire field the driver selected to use */ + uint16_t desc_num_before_header_enabled; + + /* valid only if inline was chosen. bitfield of enum + * ena_admin_llq_stride_ctrl + */ + uint16_t descriptors_stride_ctrl_supported; + + /* the stride control the driver selected to use */ + uint16_t descriptors_stride_ctrl_enabled; + + /* Maximum size in bytes taken by llq entries in a single tx burst. + * Set to 0 when there is no such limit. + */ + uint32_t max_tx_burst_size; +}; + +struct ena_admin_queue_ext_feature_fields { + uint32_t max_tx_sq_num; + + uint32_t max_tx_cq_num; + + uint32_t max_rx_sq_num; + + uint32_t max_rx_cq_num; + + uint32_t max_tx_sq_depth; + + uint32_t max_tx_cq_depth; + + uint32_t max_rx_sq_depth; + + uint32_t max_rx_cq_depth; + + uint32_t max_tx_header_size; + + /* Maximum Descriptors number, including meta descriptor, allowed for + * a single Tx packet + */ + uint16_t max_per_packet_tx_descs; + + /* Maximum Descriptors number allowed for a single Rx packet */ + uint16_t max_per_packet_rx_descs; +}; + struct ena_admin_queue_feature_desc { - /* including LLQs */ uint32_t max_sq_num; uint32_t max_sq_depth; @@ -495,9 +580,9 @@ struct ena_admin_queue_feature_desc { uint32_t max_cq_depth; - uint32_t max_llq_num; + uint32_t max_legacy_llq_num; - uint32_t max_llq_depth; + uint32_t max_legacy_llq_depth; uint32_t max_header_size; @@ -515,6 +600,14 @@ struct ena_admin_set_feature_mtu_desc { uint32_t mtu; }; +struct ena_admin_get_extra_properties_strings_desc { + uint32_t count; +}; + +struct ena_admin_get_extra_properties_flags_desc { + uint32_t flags; +}; + struct ena_admin_set_feature_host_attr_desc { /* host OS info base address in OS memory. host info is 4KB of * physically contiguous @@ -585,9 +678,8 @@ struct ena_admin_feature_offload_desc { }; enum ena_admin_hash_functions { - ENA_ADMIN_TOEPLITZ = 1, - - ENA_ADMIN_CRC32 = 2, + ENA_ADMIN_TOEPLITZ = 1, + ENA_ADMIN_CRC32 = 2, }; struct ena_admin_feature_rss_flow_hash_control { @@ -613,50 +705,35 @@ struct ena_admin_feature_rss_flow_hash_function { /* RSS flow hash protocols */ enum ena_admin_flow_hash_proto { - ENA_ADMIN_RSS_TCP4 = 0, - - ENA_ADMIN_RSS_UDP4 = 1, - - ENA_ADMIN_RSS_TCP6 = 2, - - ENA_ADMIN_RSS_UDP6 = 3, - - ENA_ADMIN_RSS_IP4 = 4, - - ENA_ADMIN_RSS_IP6 = 5, - - ENA_ADMIN_RSS_IP4_FRAG = 6, - - ENA_ADMIN_RSS_NOT_IP = 7, - + ENA_ADMIN_RSS_TCP4 = 0, + ENA_ADMIN_RSS_UDP4 = 1, + ENA_ADMIN_RSS_TCP6 = 2, + ENA_ADMIN_RSS_UDP6 = 3, + ENA_ADMIN_RSS_IP4 = 4, + ENA_ADMIN_RSS_IP6 = 5, + ENA_ADMIN_RSS_IP4_FRAG = 6, + ENA_ADMIN_RSS_NOT_IP = 7, /* TCPv6 with extension header */ - ENA_ADMIN_RSS_TCP6_EX = 8, - + ENA_ADMIN_RSS_TCP6_EX = 8, /* IPv6 with extension header */ - ENA_ADMIN_RSS_IP6_EX = 9, - - ENA_ADMIN_RSS_PROTO_NUM = 16, + ENA_ADMIN_RSS_IP6_EX = 9, + ENA_ADMIN_RSS_PROTO_NUM = 16, }; /* RSS flow hash fields */ enum ena_admin_flow_hash_fields { /* Ethernet Dest Addr */ - ENA_ADMIN_RSS_L2_DA = BIT(0), - + ENA_ADMIN_RSS_L2_DA = BIT(0), /* Ethernet Src Addr */ - ENA_ADMIN_RSS_L2_SA = BIT(1), - + ENA_ADMIN_RSS_L2_SA = BIT(1), /* ipv4/6 Dest Addr */ - ENA_ADMIN_RSS_L3_DA = BIT(2), - + ENA_ADMIN_RSS_L3_DA = BIT(2), /* ipv4/6 Src Addr */ - ENA_ADMIN_RSS_L3_SA = BIT(3), - + ENA_ADMIN_RSS_L3_SA = BIT(3), /* tcp/udp Dest Port */ - ENA_ADMIN_RSS_L4_DP = BIT(4), - + ENA_ADMIN_RSS_L4_DP = BIT(4), /* tcp/udp Src Port */ - ENA_ADMIN_RSS_L4_SP = BIT(5), + ENA_ADMIN_RSS_L4_SP = BIT(5), }; struct ena_admin_proto_input { @@ -695,15 +772,13 @@ struct ena_admin_feature_rss_flow_hash_input { }; enum ena_admin_os_type { - ENA_ADMIN_OS_LINUX = 1, - - ENA_ADMIN_OS_WIN = 2, - - ENA_ADMIN_OS_DPDK = 3, - - ENA_ADMIN_OS_FREEBSD = 4, - - ENA_ADMIN_OS_IPXE = 5, + ENA_ADMIN_OS_LINUX = 1, + ENA_ADMIN_OS_WIN = 2, + ENA_ADMIN_OS_DPDK = 3, + ENA_ADMIN_OS_FREEBSD = 4, + ENA_ADMIN_OS_IPXE = 5, + ENA_ADMIN_OS_ESXI = 6, + ENA_ADMIN_OS_GROUPS_NUM = 6, }; struct ena_admin_host_info { @@ -725,11 +800,27 @@ struct ena_admin_host_info { /* 7:0 : major * 15:8 : minor * 23:16 : sub_minor + * 31:24 : module_type */ uint32_t driver_version; /* features bitmap */ - uint32_t supported_network_features[4]; + uint32_t supported_network_features[2]; + + /* ENA spec version of driver */ + uint16_t ena_spec_version; + + /* ENA device's Bus, Device and Function + * 2:0 : function + * 7:3 : device + * 15:8 : bus + */ + uint16_t bdf; + + /* Number of CPUs */ + uint16_t num_cpus; + + uint16_t reserved; }; struct ena_admin_rss_ind_table_entry { @@ -748,7 +839,12 @@ struct ena_admin_feature_rss_ind_table { /* table size (2^size) */ uint16_t size; - uint16_t reserved; + /* 0 : one_entry_update - The FW supports setting a + * single RSS table entry + */ + uint8_t flags; + + uint8_t reserved; /* index of the inline entry. 0xFFFFFFFF means invalid */ uint32_t inline_index; @@ -794,6 +890,19 @@ struct ena_admin_get_feat_cmd { uint32_t raw[11]; }; +struct ena_admin_queue_ext_feature_desc { + /* version */ + uint8_t version; + + uint8_t reserved1[3]; + + union { + struct ena_admin_queue_ext_feature_fields max_queue_ext; + + uint32_t raw[10]; + } ; +}; + struct ena_admin_get_feat_resp { struct ena_admin_acq_common_desc acq_common_desc; @@ -802,8 +911,12 @@ struct ena_admin_get_feat_resp { struct ena_admin_device_attr_feature_desc dev_attr; + struct ena_admin_feature_llq_desc llq; + struct ena_admin_queue_feature_desc max_queue; + struct ena_admin_queue_ext_feature_desc max_queue_ext; + struct ena_admin_feature_aenq_desc aenq; struct ena_admin_get_feature_link_desc link; @@ -819,6 +932,10 @@ struct ena_admin_get_feat_resp { struct ena_admin_feature_intr_moder_desc intr_moderation; struct ena_admin_ena_hw_hints hw_hints; + + struct ena_admin_get_extra_properties_strings_desc extra_properties_strings; + + struct ena_admin_get_extra_properties_flags_desc extra_properties_flags; } u; }; @@ -849,6 +966,9 @@ struct ena_admin_set_feat_cmd { /* rss indirection table */ struct ena_admin_feature_rss_ind_table ind_table; + + /* LLQ configuration */ + struct ena_admin_feature_llq_desc llq; } u; }; @@ -865,7 +985,9 @@ struct ena_admin_aenq_common_desc { uint16_t syndrom; - /* 0 : phase */ + /* 0 : phase + * 7:1 : reserved - MBZ + */ uint8_t flags; uint8_t reserved1[3]; @@ -877,25 +999,18 @@ struct ena_admin_aenq_common_desc { /* asynchronous event notification groups */ enum ena_admin_aenq_group { - ENA_ADMIN_LINK_CHANGE = 0, - - ENA_ADMIN_FATAL_ERROR = 1, - - ENA_ADMIN_WARNING = 2, - - ENA_ADMIN_NOTIFICATION = 3, - - ENA_ADMIN_KEEP_ALIVE = 4, - - ENA_ADMIN_AENQ_GROUPS_NUM = 5, + ENA_ADMIN_LINK_CHANGE = 0, + ENA_ADMIN_FATAL_ERROR = 1, + ENA_ADMIN_WARNING = 2, + ENA_ADMIN_NOTIFICATION = 3, + ENA_ADMIN_KEEP_ALIVE = 4, + ENA_ADMIN_AENQ_GROUPS_NUM = 5, }; enum ena_admin_aenq_notification_syndrom { - ENA_ADMIN_SUSPEND = 0, - - ENA_ADMIN_RESUME = 1, - - ENA_ADMIN_UPDATE_HINTS = 2, + ENA_ADMIN_SUSPEND = 0, + ENA_ADMIN_RESUME = 1, + ENA_ADMIN_UPDATE_HINTS = 2, }; struct ena_admin_aenq_entry { @@ -930,27 +1045,27 @@ struct ena_admin_ena_mmio_req_read_less_resp { }; /* aq_common_desc */ -#define ENA_ADMIN_AQ_COMMON_DESC_COMMAND_ID_MASK GENMASK(11, 0) -#define ENA_ADMIN_AQ_COMMON_DESC_PHASE_MASK BIT(0) -#define ENA_ADMIN_AQ_COMMON_DESC_CTRL_DATA_SHIFT 1 -#define ENA_ADMIN_AQ_COMMON_DESC_CTRL_DATA_MASK BIT(1) -#define ENA_ADMIN_AQ_COMMON_DESC_CTRL_DATA_INDIRECT_SHIFT 2 -#define ENA_ADMIN_AQ_COMMON_DESC_CTRL_DATA_INDIRECT_MASK BIT(2) +#define ENA_ADMIN_AQ_COMMON_DESC_COMMAND_ID_MASK GENMASK(11, 0) +#define ENA_ADMIN_AQ_COMMON_DESC_PHASE_MASK BIT(0) +#define ENA_ADMIN_AQ_COMMON_DESC_CTRL_DATA_SHIFT 1 +#define ENA_ADMIN_AQ_COMMON_DESC_CTRL_DATA_MASK BIT(1) +#define ENA_ADMIN_AQ_COMMON_DESC_CTRL_DATA_INDIRECT_SHIFT 2 +#define ENA_ADMIN_AQ_COMMON_DESC_CTRL_DATA_INDIRECT_MASK BIT(2) /* sq */ -#define ENA_ADMIN_SQ_SQ_DIRECTION_SHIFT 5 -#define ENA_ADMIN_SQ_SQ_DIRECTION_MASK GENMASK(7, 5) +#define ENA_ADMIN_SQ_SQ_DIRECTION_SHIFT 5 +#define ENA_ADMIN_SQ_SQ_DIRECTION_MASK GENMASK(7, 5) /* acq_common_desc */ -#define ENA_ADMIN_ACQ_COMMON_DESC_COMMAND_ID_MASK GENMASK(11, 0) -#define ENA_ADMIN_ACQ_COMMON_DESC_PHASE_MASK BIT(0) +#define ENA_ADMIN_ACQ_COMMON_DESC_COMMAND_ID_MASK GENMASK(11, 0) +#define ENA_ADMIN_ACQ_COMMON_DESC_PHASE_MASK BIT(0) /* aq_create_sq_cmd */ -#define ENA_ADMIN_AQ_CREATE_SQ_CMD_SQ_DIRECTION_SHIFT 5 -#define ENA_ADMIN_AQ_CREATE_SQ_CMD_SQ_DIRECTION_MASK GENMASK(7, 5) -#define ENA_ADMIN_AQ_CREATE_SQ_CMD_PLACEMENT_POLICY_MASK GENMASK(3, 0) -#define ENA_ADMIN_AQ_CREATE_SQ_CMD_COMPLETION_POLICY_SHIFT 4 -#define ENA_ADMIN_AQ_CREATE_SQ_CMD_COMPLETION_POLICY_MASK GENMASK(6, 4) +#define ENA_ADMIN_AQ_CREATE_SQ_CMD_SQ_DIRECTION_SHIFT 5 +#define ENA_ADMIN_AQ_CREATE_SQ_CMD_SQ_DIRECTION_MASK GENMASK(7, 5) +#define ENA_ADMIN_AQ_CREATE_SQ_CMD_PLACEMENT_POLICY_MASK GENMASK(3, 0) +#define ENA_ADMIN_AQ_CREATE_SQ_CMD_COMPLETION_POLICY_SHIFT 4 +#define ENA_ADMIN_AQ_CREATE_SQ_CMD_COMPLETION_POLICY_MASK GENMASK(6, 4) #define ENA_ADMIN_AQ_CREATE_SQ_CMD_IS_PHYSICALLY_CONTIGUOUS_MASK BIT(0) /* aq_create_cq_cmd */ @@ -959,12 +1074,12 @@ struct ena_admin_ena_mmio_req_read_less_resp { #define ENA_ADMIN_AQ_CREATE_CQ_CMD_CQ_ENTRY_SIZE_WORDS_MASK GENMASK(4, 0) /* get_set_feature_common_desc */ -#define ENA_ADMIN_GET_SET_FEATURE_COMMON_DESC_SELECT_MASK GENMASK(1, 0) +#define ENA_ADMIN_GET_SET_FEATURE_COMMON_DESC_SELECT_MASK GENMASK(1, 0) /* get_feature_link_desc */ -#define ENA_ADMIN_GET_FEATURE_LINK_DESC_AUTONEG_MASK BIT(0) -#define ENA_ADMIN_GET_FEATURE_LINK_DESC_DUPLEX_SHIFT 1 -#define ENA_ADMIN_GET_FEATURE_LINK_DESC_DUPLEX_MASK BIT(1) +#define ENA_ADMIN_GET_FEATURE_LINK_DESC_AUTONEG_MASK BIT(0) +#define ENA_ADMIN_GET_FEATURE_LINK_DESC_DUPLEX_SHIFT 1 +#define ENA_ADMIN_GET_FEATURE_LINK_DESC_DUPLEX_MASK BIT(1) /* feature_offload_desc */ #define ENA_ADMIN_FEATURE_OFFLOAD_DESC_TX_L3_CSUM_IPV4_MASK BIT(0) @@ -976,19 +1091,19 @@ struct ena_admin_ena_mmio_req_read_less_resp { #define ENA_ADMIN_FEATURE_OFFLOAD_DESC_TX_L4_IPV6_CSUM_PART_MASK BIT(3) #define ENA_ADMIN_FEATURE_OFFLOAD_DESC_TX_L4_IPV6_CSUM_FULL_SHIFT 4 #define ENA_ADMIN_FEATURE_OFFLOAD_DESC_TX_L4_IPV6_CSUM_FULL_MASK BIT(4) -#define ENA_ADMIN_FEATURE_OFFLOAD_DESC_TSO_IPV4_SHIFT 5 -#define ENA_ADMIN_FEATURE_OFFLOAD_DESC_TSO_IPV4_MASK BIT(5) -#define ENA_ADMIN_FEATURE_OFFLOAD_DESC_TSO_IPV6_SHIFT 6 -#define ENA_ADMIN_FEATURE_OFFLOAD_DESC_TSO_IPV6_MASK BIT(6) -#define ENA_ADMIN_FEATURE_OFFLOAD_DESC_TSO_ECN_SHIFT 7 -#define ENA_ADMIN_FEATURE_OFFLOAD_DESC_TSO_ECN_MASK BIT(7) +#define ENA_ADMIN_FEATURE_OFFLOAD_DESC_TSO_IPV4_SHIFT 5 +#define ENA_ADMIN_FEATURE_OFFLOAD_DESC_TSO_IPV4_MASK BIT(5) +#define ENA_ADMIN_FEATURE_OFFLOAD_DESC_TSO_IPV6_SHIFT 6 +#define ENA_ADMIN_FEATURE_OFFLOAD_DESC_TSO_IPV6_MASK BIT(6) +#define ENA_ADMIN_FEATURE_OFFLOAD_DESC_TSO_ECN_SHIFT 7 +#define ENA_ADMIN_FEATURE_OFFLOAD_DESC_TSO_ECN_MASK BIT(7) #define ENA_ADMIN_FEATURE_OFFLOAD_DESC_RX_L3_CSUM_IPV4_MASK BIT(0) #define ENA_ADMIN_FEATURE_OFFLOAD_DESC_RX_L4_IPV4_CSUM_SHIFT 1 #define ENA_ADMIN_FEATURE_OFFLOAD_DESC_RX_L4_IPV4_CSUM_MASK BIT(1) #define ENA_ADMIN_FEATURE_OFFLOAD_DESC_RX_L4_IPV6_CSUM_SHIFT 2 #define ENA_ADMIN_FEATURE_OFFLOAD_DESC_RX_L4_IPV6_CSUM_MASK BIT(2) -#define ENA_ADMIN_FEATURE_OFFLOAD_DESC_RX_HASH_SHIFT 3 -#define ENA_ADMIN_FEATURE_OFFLOAD_DESC_RX_HASH_MASK BIT(3) +#define ENA_ADMIN_FEATURE_OFFLOAD_DESC_RX_HASH_SHIFT 3 +#define ENA_ADMIN_FEATURE_OFFLOAD_DESC_RX_HASH_MASK BIT(3) /* feature_rss_flow_hash_function */ #define ENA_ADMIN_FEATURE_RSS_FLOW_HASH_FUNCTION_FUNCS_MASK GENMASK(7, 0) @@ -996,28 +1111,38 @@ struct ena_admin_ena_mmio_req_read_less_resp { /* feature_rss_flow_hash_input */ #define ENA_ADMIN_FEATURE_RSS_FLOW_HASH_INPUT_L3_SORT_SHIFT 1 -#define ENA_ADMIN_FEATURE_RSS_FLOW_HASH_INPUT_L3_SORT_MASK BIT(1) +#define ENA_ADMIN_FEATURE_RSS_FLOW_HASH_INPUT_L3_SORT_MASK BIT(1) #define ENA_ADMIN_FEATURE_RSS_FLOW_HASH_INPUT_L4_SORT_SHIFT 2 -#define ENA_ADMIN_FEATURE_RSS_FLOW_HASH_INPUT_L4_SORT_MASK BIT(2) +#define ENA_ADMIN_FEATURE_RSS_FLOW_HASH_INPUT_L4_SORT_MASK BIT(2) #define ENA_ADMIN_FEATURE_RSS_FLOW_HASH_INPUT_ENABLE_L3_SORT_SHIFT 1 #define ENA_ADMIN_FEATURE_RSS_FLOW_HASH_INPUT_ENABLE_L3_SORT_MASK BIT(1) #define ENA_ADMIN_FEATURE_RSS_FLOW_HASH_INPUT_ENABLE_L4_SORT_SHIFT 2 #define ENA_ADMIN_FEATURE_RSS_FLOW_HASH_INPUT_ENABLE_L4_SORT_MASK BIT(2) /* host_info */ -#define ENA_ADMIN_HOST_INFO_MAJOR_MASK GENMASK(7, 0) -#define ENA_ADMIN_HOST_INFO_MINOR_SHIFT 8 -#define ENA_ADMIN_HOST_INFO_MINOR_MASK GENMASK(15, 8) -#define ENA_ADMIN_HOST_INFO_SUB_MINOR_SHIFT 16 -#define ENA_ADMIN_HOST_INFO_SUB_MINOR_MASK GENMASK(23, 16) +#define ENA_ADMIN_HOST_INFO_MAJOR_MASK GENMASK(7, 0) +#define ENA_ADMIN_HOST_INFO_MINOR_SHIFT 8 +#define ENA_ADMIN_HOST_INFO_MINOR_MASK GENMASK(15, 8) +#define ENA_ADMIN_HOST_INFO_SUB_MINOR_SHIFT 16 +#define ENA_ADMIN_HOST_INFO_SUB_MINOR_MASK GENMASK(23, 16) +#define ENA_ADMIN_HOST_INFO_MODULE_TYPE_SHIFT 24 +#define ENA_ADMIN_HOST_INFO_MODULE_TYPE_MASK GENMASK(31, 24) +#define ENA_ADMIN_HOST_INFO_FUNCTION_MASK GENMASK(2, 0) +#define ENA_ADMIN_HOST_INFO_DEVICE_SHIFT 3 +#define ENA_ADMIN_HOST_INFO_DEVICE_MASK GENMASK(7, 3) +#define ENA_ADMIN_HOST_INFO_BUS_SHIFT 8 +#define ENA_ADMIN_HOST_INFO_BUS_MASK GENMASK(15, 8) + +/* feature_rss_ind_table */ +#define ENA_ADMIN_FEATURE_RSS_IND_TABLE_ONE_ENTRY_UPDATE_MASK BIT(0) /* aenq_common_desc */ -#define ENA_ADMIN_AENQ_COMMON_DESC_PHASE_MASK BIT(0) +#define ENA_ADMIN_AENQ_COMMON_DESC_PHASE_MASK BIT(0) /* aenq_link_change_desc */ -#define ENA_ADMIN_AENQ_LINK_CHANGE_DESC_LINK_STATUS_MASK BIT(0) +#define ENA_ADMIN_AENQ_LINK_CHANGE_DESC_LINK_STATUS_MASK BIT(0) -#if !defined(ENA_DEFS_LINUX_MAINLINE) +#if !defined(DEFS_LINUX_MAINLINE) static inline uint16_t get_ena_admin_aq_common_desc_command_id(const struct ena_admin_aq_common_desc *p) { return p->command_id & ENA_ADMIN_AQ_COMMON_DESC_COMMAND_ID_MASK; @@ -1388,6 +1513,56 @@ static inline void set_ena_admin_host_info_sub_minor(struct ena_admin_host_info p->driver_version |= (val << ENA_ADMIN_HOST_INFO_SUB_MINOR_SHIFT) & ENA_ADMIN_HOST_INFO_SUB_MINOR_MASK; } +static inline uint32_t get_ena_admin_host_info_module_type(const struct ena_admin_host_info *p) +{ + return (p->driver_version & ENA_ADMIN_HOST_INFO_MODULE_TYPE_MASK) >> ENA_ADMIN_HOST_INFO_MODULE_TYPE_SHIFT; +} + +static inline void set_ena_admin_host_info_module_type(struct ena_admin_host_info *p, uint32_t val) +{ + p->driver_version |= (val << ENA_ADMIN_HOST_INFO_MODULE_TYPE_SHIFT) & ENA_ADMIN_HOST_INFO_MODULE_TYPE_MASK; +} + +static inline uint16_t get_ena_admin_host_info_function(const struct ena_admin_host_info *p) +{ + return p->bdf & ENA_ADMIN_HOST_INFO_FUNCTION_MASK; +} + +static inline void set_ena_admin_host_info_function(struct ena_admin_host_info *p, uint16_t val) +{ + p->bdf |= val & ENA_ADMIN_HOST_INFO_FUNCTION_MASK; +} + +static inline uint16_t get_ena_admin_host_info_device(const struct ena_admin_host_info *p) +{ + return (p->bdf & ENA_ADMIN_HOST_INFO_DEVICE_MASK) >> ENA_ADMIN_HOST_INFO_DEVICE_SHIFT; +} + +static inline void set_ena_admin_host_info_device(struct ena_admin_host_info *p, uint16_t val) +{ + p->bdf |= (val << ENA_ADMIN_HOST_INFO_DEVICE_SHIFT) & ENA_ADMIN_HOST_INFO_DEVICE_MASK; +} + +static inline uint16_t get_ena_admin_host_info_bus(const struct ena_admin_host_info *p) +{ + return (p->bdf & ENA_ADMIN_HOST_INFO_BUS_MASK) >> ENA_ADMIN_HOST_INFO_BUS_SHIFT; +} + +static inline void set_ena_admin_host_info_bus(struct ena_admin_host_info *p, uint16_t val) +{ + p->bdf |= (val << ENA_ADMIN_HOST_INFO_BUS_SHIFT) & ENA_ADMIN_HOST_INFO_BUS_MASK; +} + +static inline uint8_t get_ena_admin_feature_rss_ind_table_one_entry_update(const struct ena_admin_feature_rss_ind_table *p) +{ + return p->flags & ENA_ADMIN_FEATURE_RSS_IND_TABLE_ONE_ENTRY_UPDATE_MASK; +} + +static inline void set_ena_admin_feature_rss_ind_table_one_entry_update(struct ena_admin_feature_rss_ind_table *p, uint8_t val) +{ + p->flags |= val & ENA_ADMIN_FEATURE_RSS_IND_TABLE_ONE_ENTRY_UPDATE_MASK; +} + static inline uint8_t get_ena_admin_aenq_common_desc_phase(const struct ena_admin_aenq_common_desc *p) { return p->flags & ENA_ADMIN_AENQ_COMMON_DESC_PHASE_MASK; @@ -1408,5 +1583,5 @@ static inline void set_ena_admin_aenq_link_change_desc_link_status(struct ena_ad p->flags |= val & ENA_ADMIN_AENQ_LINK_CHANGE_DESC_LINK_STATUS_MASK; } -#endif /* !defined(ENA_DEFS_LINUX_MAINLINE) */ +#endif /* !defined(DEFS_LINUX_MAINLINE) */ #endif /*_ENA_ADMIN_H_ */ diff --git a/drivers/net/ena/base/ena_defs/ena_common_defs.h b/drivers/net/ena/base/ena_defs/ena_common_defs.h index 072e6c1f1..e02334072 100644 --- a/drivers/net/ena/base/ena_defs/ena_common_defs.h +++ b/drivers/net/ena/base/ena_defs/ena_common_defs.h @@ -30,12 +30,11 @@ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ - #ifndef _ENA_COMMON_H_ #define _ENA_COMMON_H_ -#define ENA_COMMON_SPEC_VERSION_MAJOR 0 /* */ -#define ENA_COMMON_SPEC_VERSION_MINOR 10 /* */ +#define ENA_COMMON_SPEC_VERSION_MAJOR 2 +#define ENA_COMMON_SPEC_VERSION_MINOR 0 /* ENA operates with 48-bit memory addresses. ena_mem_addr_t */ struct ena_common_mem_addr { diff --git a/drivers/net/ena/base/ena_defs/ena_eth_io_defs.h b/drivers/net/ena/base/ena_defs/ena_eth_io_defs.h index 4cf0b205b..d060d8be3 100644 --- a/drivers/net/ena/base/ena_defs/ena_eth_io_defs.h +++ b/drivers/net/ena/base/ena_defs/ena_eth_io_defs.h @@ -30,30 +30,22 @@ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ - #ifndef _ENA_ETH_IO_H_ #define _ENA_ETH_IO_H_ enum ena_eth_io_l3_proto_index { - ENA_ETH_IO_L3_PROTO_UNKNOWN = 0, - - ENA_ETH_IO_L3_PROTO_IPV4 = 8, - - ENA_ETH_IO_L3_PROTO_IPV6 = 11, - - ENA_ETH_IO_L3_PROTO_FCOE = 21, - - ENA_ETH_IO_L3_PROTO_ROCE = 22, + ENA_ETH_IO_L3_PROTO_UNKNOWN = 0, + ENA_ETH_IO_L3_PROTO_IPV4 = 8, + ENA_ETH_IO_L3_PROTO_IPV6 = 11, + ENA_ETH_IO_L3_PROTO_FCOE = 21, + ENA_ETH_IO_L3_PROTO_ROCE = 22, }; enum ena_eth_io_l4_proto_index { - ENA_ETH_IO_L4_PROTO_UNKNOWN = 0, - - ENA_ETH_IO_L4_PROTO_TCP = 12, - - ENA_ETH_IO_L4_PROTO_UDP = 13, - - ENA_ETH_IO_L4_PROTO_ROUTEABLE_ROCE = 23, + ENA_ETH_IO_L4_PROTO_UNKNOWN = 0, + ENA_ETH_IO_L4_PROTO_TCP = 12, + ENA_ETH_IO_L4_PROTO_UDP = 13, + ENA_ETH_IO_L4_PROTO_ROUTEABLE_ROCE = 23, }; struct ena_eth_io_tx_desc { @@ -244,9 +236,13 @@ struct ena_eth_io_rx_cdesc_base { * checksum error detected, or, the controller didn't * validate the checksum. This bit is valid only when * l4_proto_idx indicates TCP/UDP packet, and, - * ipv4_frag is not set + * ipv4_frag is not set. This bit is valid only when + * l4_csum_checked below is set. * 15 : ipv4_frag - Indicates IPv4 fragmented packet - * 23:16 : reserved16 + * 16 : l4_csum_checked - L4 checksum was verified + * (could be OK or error), when cleared the status of + * checksum is unknown + * 23:17 : reserved16 * 24 : phase * 25 : l3_csum2 - second checksum engine result * 26 : first - Indicates first descriptor in @@ -305,117 +301,119 @@ struct ena_eth_io_numa_node_cfg_reg { }; /* tx_desc */ -#define ENA_ETH_IO_TX_DESC_LENGTH_MASK GENMASK(15, 0) -#define ENA_ETH_IO_TX_DESC_REQ_ID_HI_SHIFT 16 -#define ENA_ETH_IO_TX_DESC_REQ_ID_HI_MASK GENMASK(21, 16) -#define ENA_ETH_IO_TX_DESC_META_DESC_SHIFT 23 -#define ENA_ETH_IO_TX_DESC_META_DESC_MASK BIT(23) -#define ENA_ETH_IO_TX_DESC_PHASE_SHIFT 24 -#define ENA_ETH_IO_TX_DESC_PHASE_MASK BIT(24) -#define ENA_ETH_IO_TX_DESC_FIRST_SHIFT 26 -#define ENA_ETH_IO_TX_DESC_FIRST_MASK BIT(26) -#define ENA_ETH_IO_TX_DESC_LAST_SHIFT 27 -#define ENA_ETH_IO_TX_DESC_LAST_MASK BIT(27) -#define ENA_ETH_IO_TX_DESC_COMP_REQ_SHIFT 28 -#define ENA_ETH_IO_TX_DESC_COMP_REQ_MASK BIT(28) -#define ENA_ETH_IO_TX_DESC_L3_PROTO_IDX_MASK GENMASK(3, 0) -#define ENA_ETH_IO_TX_DESC_DF_SHIFT 4 -#define ENA_ETH_IO_TX_DESC_DF_MASK BIT(4) -#define ENA_ETH_IO_TX_DESC_TSO_EN_SHIFT 7 -#define ENA_ETH_IO_TX_DESC_TSO_EN_MASK BIT(7) -#define ENA_ETH_IO_TX_DESC_L4_PROTO_IDX_SHIFT 8 -#define ENA_ETH_IO_TX_DESC_L4_PROTO_IDX_MASK GENMASK(12, 8) -#define ENA_ETH_IO_TX_DESC_L3_CSUM_EN_SHIFT 13 -#define ENA_ETH_IO_TX_DESC_L3_CSUM_EN_MASK BIT(13) -#define ENA_ETH_IO_TX_DESC_L4_CSUM_EN_SHIFT 14 -#define ENA_ETH_IO_TX_DESC_L4_CSUM_EN_MASK BIT(14) -#define ENA_ETH_IO_TX_DESC_ETHERNET_FCS_DIS_SHIFT 15 -#define ENA_ETH_IO_TX_DESC_ETHERNET_FCS_DIS_MASK BIT(15) -#define ENA_ETH_IO_TX_DESC_L4_CSUM_PARTIAL_SHIFT 17 -#define ENA_ETH_IO_TX_DESC_L4_CSUM_PARTIAL_MASK BIT(17) -#define ENA_ETH_IO_TX_DESC_REQ_ID_LO_SHIFT 22 -#define ENA_ETH_IO_TX_DESC_REQ_ID_LO_MASK GENMASK(31, 22) -#define ENA_ETH_IO_TX_DESC_ADDR_HI_MASK GENMASK(15, 0) -#define ENA_ETH_IO_TX_DESC_HEADER_LENGTH_SHIFT 24 -#define ENA_ETH_IO_TX_DESC_HEADER_LENGTH_MASK GENMASK(31, 24) +#define ENA_ETH_IO_TX_DESC_LENGTH_MASK GENMASK(15, 0) +#define ENA_ETH_IO_TX_DESC_REQ_ID_HI_SHIFT 16 +#define ENA_ETH_IO_TX_DESC_REQ_ID_HI_MASK GENMASK(21, 16) +#define ENA_ETH_IO_TX_DESC_META_DESC_SHIFT 23 +#define ENA_ETH_IO_TX_DESC_META_DESC_MASK BIT(23) +#define ENA_ETH_IO_TX_DESC_PHASE_SHIFT 24 +#define ENA_ETH_IO_TX_DESC_PHASE_MASK BIT(24) +#define ENA_ETH_IO_TX_DESC_FIRST_SHIFT 26 +#define ENA_ETH_IO_TX_DESC_FIRST_MASK BIT(26) +#define ENA_ETH_IO_TX_DESC_LAST_SHIFT 27 +#define ENA_ETH_IO_TX_DESC_LAST_MASK BIT(27) +#define ENA_ETH_IO_TX_DESC_COMP_REQ_SHIFT 28 +#define ENA_ETH_IO_TX_DESC_COMP_REQ_MASK BIT(28) +#define ENA_ETH_IO_TX_DESC_L3_PROTO_IDX_MASK GENMASK(3, 0) +#define ENA_ETH_IO_TX_DESC_DF_SHIFT 4 +#define ENA_ETH_IO_TX_DESC_DF_MASK BIT(4) +#define ENA_ETH_IO_TX_DESC_TSO_EN_SHIFT 7 +#define ENA_ETH_IO_TX_DESC_TSO_EN_MASK BIT(7) +#define ENA_ETH_IO_TX_DESC_L4_PROTO_IDX_SHIFT 8 +#define ENA_ETH_IO_TX_DESC_L4_PROTO_IDX_MASK GENMASK(12, 8) +#define ENA_ETH_IO_TX_DESC_L3_CSUM_EN_SHIFT 13 +#define ENA_ETH_IO_TX_DESC_L3_CSUM_EN_MASK BIT(13) +#define ENA_ETH_IO_TX_DESC_L4_CSUM_EN_SHIFT 14 +#define ENA_ETH_IO_TX_DESC_L4_CSUM_EN_MASK BIT(14) +#define ENA_ETH_IO_TX_DESC_ETHERNET_FCS_DIS_SHIFT 15 +#define ENA_ETH_IO_TX_DESC_ETHERNET_FCS_DIS_MASK BIT(15) +#define ENA_ETH_IO_TX_DESC_L4_CSUM_PARTIAL_SHIFT 17 +#define ENA_ETH_IO_TX_DESC_L4_CSUM_PARTIAL_MASK BIT(17) +#define ENA_ETH_IO_TX_DESC_REQ_ID_LO_SHIFT 22 +#define ENA_ETH_IO_TX_DESC_REQ_ID_LO_MASK GENMASK(31, 22) +#define ENA_ETH_IO_TX_DESC_ADDR_HI_MASK GENMASK(15, 0) +#define ENA_ETH_IO_TX_DESC_HEADER_LENGTH_SHIFT 24 +#define ENA_ETH_IO_TX_DESC_HEADER_LENGTH_MASK GENMASK(31, 24) /* tx_meta_desc */ -#define ENA_ETH_IO_TX_META_DESC_REQ_ID_LO_MASK GENMASK(9, 0) -#define ENA_ETH_IO_TX_META_DESC_EXT_VALID_SHIFT 14 -#define ENA_ETH_IO_TX_META_DESC_EXT_VALID_MASK BIT(14) -#define ENA_ETH_IO_TX_META_DESC_MSS_HI_SHIFT 16 -#define ENA_ETH_IO_TX_META_DESC_MSS_HI_MASK GENMASK(19, 16) -#define ENA_ETH_IO_TX_META_DESC_ETH_META_TYPE_SHIFT 20 -#define ENA_ETH_IO_TX_META_DESC_ETH_META_TYPE_MASK BIT(20) -#define ENA_ETH_IO_TX_META_DESC_META_STORE_SHIFT 21 -#define ENA_ETH_IO_TX_META_DESC_META_STORE_MASK BIT(21) -#define ENA_ETH_IO_TX_META_DESC_META_DESC_SHIFT 23 -#define ENA_ETH_IO_TX_META_DESC_META_DESC_MASK BIT(23) -#define ENA_ETH_IO_TX_META_DESC_PHASE_SHIFT 24 -#define ENA_ETH_IO_TX_META_DESC_PHASE_MASK BIT(24) -#define ENA_ETH_IO_TX_META_DESC_FIRST_SHIFT 26 -#define ENA_ETH_IO_TX_META_DESC_FIRST_MASK BIT(26) -#define ENA_ETH_IO_TX_META_DESC_LAST_SHIFT 27 -#define ENA_ETH_IO_TX_META_DESC_LAST_MASK BIT(27) -#define ENA_ETH_IO_TX_META_DESC_COMP_REQ_SHIFT 28 -#define ENA_ETH_IO_TX_META_DESC_COMP_REQ_MASK BIT(28) -#define ENA_ETH_IO_TX_META_DESC_REQ_ID_HI_MASK GENMASK(5, 0) -#define ENA_ETH_IO_TX_META_DESC_L3_HDR_LEN_MASK GENMASK(7, 0) -#define ENA_ETH_IO_TX_META_DESC_L3_HDR_OFF_SHIFT 8 -#define ENA_ETH_IO_TX_META_DESC_L3_HDR_OFF_MASK GENMASK(15, 8) -#define ENA_ETH_IO_TX_META_DESC_L4_HDR_LEN_IN_WORDS_SHIFT 16 -#define ENA_ETH_IO_TX_META_DESC_L4_HDR_LEN_IN_WORDS_MASK GENMASK(21, 16) -#define ENA_ETH_IO_TX_META_DESC_MSS_LO_SHIFT 22 -#define ENA_ETH_IO_TX_META_DESC_MSS_LO_MASK GENMASK(31, 22) +#define ENA_ETH_IO_TX_META_DESC_REQ_ID_LO_MASK GENMASK(9, 0) +#define ENA_ETH_IO_TX_META_DESC_EXT_VALID_SHIFT 14 +#define ENA_ETH_IO_TX_META_DESC_EXT_VALID_MASK BIT(14) +#define ENA_ETH_IO_TX_META_DESC_MSS_HI_SHIFT 16 +#define ENA_ETH_IO_TX_META_DESC_MSS_HI_MASK GENMASK(19, 16) +#define ENA_ETH_IO_TX_META_DESC_ETH_META_TYPE_SHIFT 20 +#define ENA_ETH_IO_TX_META_DESC_ETH_META_TYPE_MASK BIT(20) +#define ENA_ETH_IO_TX_META_DESC_META_STORE_SHIFT 21 +#define ENA_ETH_IO_TX_META_DESC_META_STORE_MASK BIT(21) +#define ENA_ETH_IO_TX_META_DESC_META_DESC_SHIFT 23 +#define ENA_ETH_IO_TX_META_DESC_META_DESC_MASK BIT(23) +#define ENA_ETH_IO_TX_META_DESC_PHASE_SHIFT 24 +#define ENA_ETH_IO_TX_META_DESC_PHASE_MASK BIT(24) +#define ENA_ETH_IO_TX_META_DESC_FIRST_SHIFT 26 +#define ENA_ETH_IO_TX_META_DESC_FIRST_MASK BIT(26) +#define ENA_ETH_IO_TX_META_DESC_LAST_SHIFT 27 +#define ENA_ETH_IO_TX_META_DESC_LAST_MASK BIT(27) +#define ENA_ETH_IO_TX_META_DESC_COMP_REQ_SHIFT 28 +#define ENA_ETH_IO_TX_META_DESC_COMP_REQ_MASK BIT(28) +#define ENA_ETH_IO_TX_META_DESC_REQ_ID_HI_MASK GENMASK(5, 0) +#define ENA_ETH_IO_TX_META_DESC_L3_HDR_LEN_MASK GENMASK(7, 0) +#define ENA_ETH_IO_TX_META_DESC_L3_HDR_OFF_SHIFT 8 +#define ENA_ETH_IO_TX_META_DESC_L3_HDR_OFF_MASK GENMASK(15, 8) +#define ENA_ETH_IO_TX_META_DESC_L4_HDR_LEN_IN_WORDS_SHIFT 16 +#define ENA_ETH_IO_TX_META_DESC_L4_HDR_LEN_IN_WORDS_MASK GENMASK(21, 16) +#define ENA_ETH_IO_TX_META_DESC_MSS_LO_SHIFT 22 +#define ENA_ETH_IO_TX_META_DESC_MSS_LO_MASK GENMASK(31, 22) /* tx_cdesc */ -#define ENA_ETH_IO_TX_CDESC_PHASE_MASK BIT(0) +#define ENA_ETH_IO_TX_CDESC_PHASE_MASK BIT(0) /* rx_desc */ -#define ENA_ETH_IO_RX_DESC_PHASE_MASK BIT(0) -#define ENA_ETH_IO_RX_DESC_FIRST_SHIFT 2 -#define ENA_ETH_IO_RX_DESC_FIRST_MASK BIT(2) -#define ENA_ETH_IO_RX_DESC_LAST_SHIFT 3 -#define ENA_ETH_IO_RX_DESC_LAST_MASK BIT(3) -#define ENA_ETH_IO_RX_DESC_COMP_REQ_SHIFT 4 -#define ENA_ETH_IO_RX_DESC_COMP_REQ_MASK BIT(4) +#define ENA_ETH_IO_RX_DESC_PHASE_MASK BIT(0) +#define ENA_ETH_IO_RX_DESC_FIRST_SHIFT 2 +#define ENA_ETH_IO_RX_DESC_FIRST_MASK BIT(2) +#define ENA_ETH_IO_RX_DESC_LAST_SHIFT 3 +#define ENA_ETH_IO_RX_DESC_LAST_MASK BIT(3) +#define ENA_ETH_IO_RX_DESC_COMP_REQ_SHIFT 4 +#define ENA_ETH_IO_RX_DESC_COMP_REQ_MASK BIT(4) /* rx_cdesc_base */ -#define ENA_ETH_IO_RX_CDESC_BASE_L3_PROTO_IDX_MASK GENMASK(4, 0) -#define ENA_ETH_IO_RX_CDESC_BASE_SRC_VLAN_CNT_SHIFT 5 -#define ENA_ETH_IO_RX_CDESC_BASE_SRC_VLAN_CNT_MASK GENMASK(6, 5) -#define ENA_ETH_IO_RX_CDESC_BASE_L4_PROTO_IDX_SHIFT 8 -#define ENA_ETH_IO_RX_CDESC_BASE_L4_PROTO_IDX_MASK GENMASK(12, 8) -#define ENA_ETH_IO_RX_CDESC_BASE_L3_CSUM_ERR_SHIFT 13 -#define ENA_ETH_IO_RX_CDESC_BASE_L3_CSUM_ERR_MASK BIT(13) -#define ENA_ETH_IO_RX_CDESC_BASE_L4_CSUM_ERR_SHIFT 14 -#define ENA_ETH_IO_RX_CDESC_BASE_L4_CSUM_ERR_MASK BIT(14) -#define ENA_ETH_IO_RX_CDESC_BASE_IPV4_FRAG_SHIFT 15 -#define ENA_ETH_IO_RX_CDESC_BASE_IPV4_FRAG_MASK BIT(15) -#define ENA_ETH_IO_RX_CDESC_BASE_PHASE_SHIFT 24 -#define ENA_ETH_IO_RX_CDESC_BASE_PHASE_MASK BIT(24) -#define ENA_ETH_IO_RX_CDESC_BASE_L3_CSUM2_SHIFT 25 -#define ENA_ETH_IO_RX_CDESC_BASE_L3_CSUM2_MASK BIT(25) -#define ENA_ETH_IO_RX_CDESC_BASE_FIRST_SHIFT 26 -#define ENA_ETH_IO_RX_CDESC_BASE_FIRST_MASK BIT(26) -#define ENA_ETH_IO_RX_CDESC_BASE_LAST_SHIFT 27 -#define ENA_ETH_IO_RX_CDESC_BASE_LAST_MASK BIT(27) -#define ENA_ETH_IO_RX_CDESC_BASE_BUFFER_SHIFT 30 -#define ENA_ETH_IO_RX_CDESC_BASE_BUFFER_MASK BIT(30) +#define ENA_ETH_IO_RX_CDESC_BASE_L3_PROTO_IDX_MASK GENMASK(4, 0) +#define ENA_ETH_IO_RX_CDESC_BASE_SRC_VLAN_CNT_SHIFT 5 +#define ENA_ETH_IO_RX_CDESC_BASE_SRC_VLAN_CNT_MASK GENMASK(6, 5) +#define ENA_ETH_IO_RX_CDESC_BASE_L4_PROTO_IDX_SHIFT 8 +#define ENA_ETH_IO_RX_CDESC_BASE_L4_PROTO_IDX_MASK GENMASK(12, 8) +#define ENA_ETH_IO_RX_CDESC_BASE_L3_CSUM_ERR_SHIFT 13 +#define ENA_ETH_IO_RX_CDESC_BASE_L3_CSUM_ERR_MASK BIT(13) +#define ENA_ETH_IO_RX_CDESC_BASE_L4_CSUM_ERR_SHIFT 14 +#define ENA_ETH_IO_RX_CDESC_BASE_L4_CSUM_ERR_MASK BIT(14) +#define ENA_ETH_IO_RX_CDESC_BASE_IPV4_FRAG_SHIFT 15 +#define ENA_ETH_IO_RX_CDESC_BASE_IPV4_FRAG_MASK BIT(15) +#define ENA_ETH_IO_RX_CDESC_BASE_L4_CSUM_CHECKED_SHIFT 16 +#define ENA_ETH_IO_RX_CDESC_BASE_L4_CSUM_CHECKED_MASK BIT(16) +#define ENA_ETH_IO_RX_CDESC_BASE_PHASE_SHIFT 24 +#define ENA_ETH_IO_RX_CDESC_BASE_PHASE_MASK BIT(24) +#define ENA_ETH_IO_RX_CDESC_BASE_L3_CSUM2_SHIFT 25 +#define ENA_ETH_IO_RX_CDESC_BASE_L3_CSUM2_MASK BIT(25) +#define ENA_ETH_IO_RX_CDESC_BASE_FIRST_SHIFT 26 +#define ENA_ETH_IO_RX_CDESC_BASE_FIRST_MASK BIT(26) +#define ENA_ETH_IO_RX_CDESC_BASE_LAST_SHIFT 27 +#define ENA_ETH_IO_RX_CDESC_BASE_LAST_MASK BIT(27) +#define ENA_ETH_IO_RX_CDESC_BASE_BUFFER_SHIFT 30 +#define ENA_ETH_IO_RX_CDESC_BASE_BUFFER_MASK BIT(30) /* intr_reg */ -#define ENA_ETH_IO_INTR_REG_RX_INTR_DELAY_MASK GENMASK(14, 0) -#define ENA_ETH_IO_INTR_REG_TX_INTR_DELAY_SHIFT 15 -#define ENA_ETH_IO_INTR_REG_TX_INTR_DELAY_MASK GENMASK(29, 15) -#define ENA_ETH_IO_INTR_REG_INTR_UNMASK_SHIFT 30 -#define ENA_ETH_IO_INTR_REG_INTR_UNMASK_MASK BIT(30) +#define ENA_ETH_IO_INTR_REG_RX_INTR_DELAY_MASK GENMASK(14, 0) +#define ENA_ETH_IO_INTR_REG_TX_INTR_DELAY_SHIFT 15 +#define ENA_ETH_IO_INTR_REG_TX_INTR_DELAY_MASK GENMASK(29, 15) +#define ENA_ETH_IO_INTR_REG_INTR_UNMASK_SHIFT 30 +#define ENA_ETH_IO_INTR_REG_INTR_UNMASK_MASK BIT(30) /* numa_node_cfg_reg */ -#define ENA_ETH_IO_NUMA_NODE_CFG_REG_NUMA_MASK GENMASK(7, 0) -#define ENA_ETH_IO_NUMA_NODE_CFG_REG_ENABLED_SHIFT 31 -#define ENA_ETH_IO_NUMA_NODE_CFG_REG_ENABLED_MASK BIT(31) +#define ENA_ETH_IO_NUMA_NODE_CFG_REG_NUMA_MASK GENMASK(7, 0) +#define ENA_ETH_IO_NUMA_NODE_CFG_REG_ENABLED_SHIFT 31 +#define ENA_ETH_IO_NUMA_NODE_CFG_REG_ENABLED_MASK BIT(31) -#if !defined(ENA_DEFS_LINUX_MAINLINE) +#if !defined(DEFS_LINUX_MAINLINE) static inline uint32_t get_ena_eth_io_tx_desc_length(const struct ena_eth_io_tx_desc *p) { return p->len_ctrl & ENA_ETH_IO_TX_DESC_LENGTH_MASK; @@ -856,6 +854,16 @@ static inline void set_ena_eth_io_rx_cdesc_base_ipv4_frag(struct ena_eth_io_rx_c p->status |= (val << ENA_ETH_IO_RX_CDESC_BASE_IPV4_FRAG_SHIFT) & ENA_ETH_IO_RX_CDESC_BASE_IPV4_FRAG_MASK; } +static inline uint32_t get_ena_eth_io_rx_cdesc_base_l4_csum_checked(const struct ena_eth_io_rx_cdesc_base *p) +{ + return (p->status & ENA_ETH_IO_RX_CDESC_BASE_L4_CSUM_CHECKED_MASK) >> ENA_ETH_IO_RX_CDESC_BASE_L4_CSUM_CHECKED_SHIFT; +} + +static inline void set_ena_eth_io_rx_cdesc_base_l4_csum_checked(struct ena_eth_io_rx_cdesc_base *p, uint32_t val) +{ + p->status |= (val << ENA_ETH_IO_RX_CDESC_BASE_L4_CSUM_CHECKED_SHIFT) & ENA_ETH_IO_RX_CDESC_BASE_L4_CSUM_CHECKED_MASK; +} + static inline uint32_t get_ena_eth_io_rx_cdesc_base_phase(const struct ena_eth_io_rx_cdesc_base *p) { return (p->status & ENA_ETH_IO_RX_CDESC_BASE_PHASE_MASK) >> ENA_ETH_IO_RX_CDESC_BASE_PHASE_SHIFT; @@ -956,5 +964,5 @@ static inline void set_ena_eth_io_numa_node_cfg_reg_enabled(struct ena_eth_io_nu p->numa_cfg |= (val << ENA_ETH_IO_NUMA_NODE_CFG_REG_ENABLED_SHIFT) & ENA_ETH_IO_NUMA_NODE_CFG_REG_ENABLED_MASK; } -#endif /* !defined(ENA_DEFS_LINUX_MAINLINE) */ +#endif /* !defined(DEFS_LINUX_MAINLINE) */ #endif /*_ENA_ETH_IO_H_ */ diff --git a/drivers/net/ena/base/ena_defs/ena_gen_info.h b/drivers/net/ena/base/ena_defs/ena_gen_info.h index e87bcfd88..640d5aa3e 100644 --- a/drivers/net/ena/base/ena_defs/ena_gen_info.h +++ b/drivers/net/ena/base/ena_defs/ena_gen_info.h @@ -30,6 +30,5 @@ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ - -#define ENA_GEN_DATE "Sun Oct 23 12:27:32 IDT 2016" -#define ENA_GEN_COMMIT "79d82fa" +#define ENA_GEN_DATE "Wed Sep 26 13:46:28 DST 2018" +#define ENA_GEN_COMMIT "aac865f" diff --git a/drivers/net/ena/base/ena_defs/ena_includes.h b/drivers/net/ena/base/ena_defs/ena_includes.h index 30a920a8e..5ea312f82 100644 --- a/drivers/net/ena/base/ena_defs/ena_includes.h +++ b/drivers/net/ena/base/ena_defs/ena_includes.h @@ -1,36 +1,3 @@ -/*- -* BSD LICENSE -* -* Copyright (c) 2015-2016 Amazon.com, Inc. or its affiliates. -* All rights reserved. -* -* Redistribution and use in source and binary forms, with or without -* modification, are permitted provided that the following conditions -* are met: -* -* * Redistributions of source code must retain the above copyright -* notice, this list of conditions and the following disclaimer. -* * Redistributions in binary form must reproduce the above copyright -* notice, this list of conditions and the following disclaimer in -* the documentation and/or other materials provided with the -* distribution. -* * Neither the name of copyright holder nor the names of its -* contributors may be used to endorse or promote products derived -* from this software without specific prior written permission. -* -* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS -* "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT -* LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR -* A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT -* OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, -* SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT -* LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, -* DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY -* THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT -* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE -* OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. -*/ - #include "ena_common_defs.h" #include "ena_regs_defs.h" #include "ena_admin_defs.h" diff --git a/drivers/net/ena/base/ena_defs/ena_regs_defs.h b/drivers/net/ena/base/ena_defs/ena_regs_defs.h index b0870f254..13b419830 100644 --- a/drivers/net/ena/base/ena_defs/ena_regs_defs.h +++ b/drivers/net/ena/base/ena_defs/ena_regs_defs.h @@ -30,142 +30,129 @@ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ - #ifndef _ENA_REGS_H_ #define _ENA_REGS_H_ enum ena_regs_reset_reason_types { - ENA_REGS_RESET_NORMAL = 0, - - ENA_REGS_RESET_KEEP_ALIVE_TO = 1, - - ENA_REGS_RESET_ADMIN_TO = 2, - - ENA_REGS_RESET_MISS_TX_CMPL = 3, - - ENA_REGS_RESET_INV_RX_REQ_ID = 4, - - ENA_REGS_RESET_INV_TX_REQ_ID = 5, - - ENA_REGS_RESET_TOO_MANY_RX_DESCS = 6, - - ENA_REGS_RESET_INIT_ERR = 7, - - ENA_REGS_RESET_DRIVER_INVALID_STATE = 8, - - ENA_REGS_RESET_OS_TRIGGER = 9, - - ENA_REGS_RESET_OS_NETDEV_WD = 10, - - ENA_REGS_RESET_SHUTDOWN = 11, - - ENA_REGS_RESET_USER_TRIGGER = 12, - - ENA_REGS_RESET_GENERIC = 13, - - ENA_REGS_RESET_MISS_INTERRUPT = 14, + ENA_REGS_RESET_NORMAL = 0, + ENA_REGS_RESET_KEEP_ALIVE_TO = 1, + ENA_REGS_RESET_ADMIN_TO = 2, + ENA_REGS_RESET_MISS_TX_CMPL = 3, + ENA_REGS_RESET_INV_RX_REQ_ID = 4, + ENA_REGS_RESET_INV_TX_REQ_ID = 5, + ENA_REGS_RESET_TOO_MANY_RX_DESCS = 6, + ENA_REGS_RESET_INIT_ERR = 7, + ENA_REGS_RESET_DRIVER_INVALID_STATE = 8, + ENA_REGS_RESET_OS_TRIGGER = 9, + ENA_REGS_RESET_OS_NETDEV_WD = 10, + ENA_REGS_RESET_SHUTDOWN = 11, + ENA_REGS_RESET_USER_TRIGGER = 12, + ENA_REGS_RESET_GENERIC = 13, + ENA_REGS_RESET_MISS_INTERRUPT = 14, }; /* ena_registers offsets */ -#define ENA_REGS_VERSION_OFF 0x0 -#define ENA_REGS_CONTROLLER_VERSION_OFF 0x4 -#define ENA_REGS_CAPS_OFF 0x8 -#define ENA_REGS_CAPS_EXT_OFF 0xc -#define ENA_REGS_AQ_BASE_LO_OFF 0x10 -#define ENA_REGS_AQ_BASE_HI_OFF 0x14 -#define ENA_REGS_AQ_CAPS_OFF 0x18 -#define ENA_REGS_ACQ_BASE_LO_OFF 0x20 -#define ENA_REGS_ACQ_BASE_HI_OFF 0x24 -#define ENA_REGS_ACQ_CAPS_OFF 0x28 -#define ENA_REGS_AQ_DB_OFF 0x2c -#define ENA_REGS_ACQ_TAIL_OFF 0x30 -#define ENA_REGS_AENQ_CAPS_OFF 0x34 -#define ENA_REGS_AENQ_BASE_LO_OFF 0x38 -#define ENA_REGS_AENQ_BASE_HI_OFF 0x3c -#define ENA_REGS_AENQ_HEAD_DB_OFF 0x40 -#define ENA_REGS_AENQ_TAIL_OFF 0x44 -#define ENA_REGS_INTR_MASK_OFF 0x4c -#define ENA_REGS_DEV_CTL_OFF 0x54 -#define ENA_REGS_DEV_STS_OFF 0x58 -#define ENA_REGS_MMIO_REG_READ_OFF 0x5c -#define ENA_REGS_MMIO_RESP_LO_OFF 0x60 -#define ENA_REGS_MMIO_RESP_HI_OFF 0x64 -#define ENA_REGS_RSS_IND_ENTRY_UPDATE_OFF 0x68 + +/* 0 base */ +#define ENA_REGS_VERSION_OFF 0x0 +#define ENA_REGS_CONTROLLER_VERSION_OFF 0x4 +#define ENA_REGS_CAPS_OFF 0x8 +#define ENA_REGS_CAPS_EXT_OFF 0xc +#define ENA_REGS_AQ_BASE_LO_OFF 0x10 +#define ENA_REGS_AQ_BASE_HI_OFF 0x14 +#define ENA_REGS_AQ_CAPS_OFF 0x18 +#define ENA_REGS_ACQ_BASE_LO_OFF 0x20 +#define ENA_REGS_ACQ_BASE_HI_OFF 0x24 +#define ENA_REGS_ACQ_CAPS_OFF 0x28 +#define ENA_REGS_AQ_DB_OFF 0x2c +#define ENA_REGS_ACQ_TAIL_OFF 0x30 +#define ENA_REGS_AENQ_CAPS_OFF 0x34 +#define ENA_REGS_AENQ_BASE_LO_OFF 0x38 +#define ENA_REGS_AENQ_BASE_HI_OFF 0x3c +#define ENA_REGS_AENQ_HEAD_DB_OFF 0x40 +#define ENA_REGS_AENQ_TAIL_OFF 0x44 +#define ENA_REGS_INTR_MASK_OFF 0x4c +#define ENA_REGS_DEV_CTL_OFF 0x54 +#define ENA_REGS_DEV_STS_OFF 0x58 +#define ENA_REGS_MMIO_REG_READ_OFF 0x5c +#define ENA_REGS_MMIO_RESP_LO_OFF 0x60 +#define ENA_REGS_MMIO_RESP_HI_OFF 0x64 +#define ENA_REGS_RSS_IND_ENTRY_UPDATE_OFF 0x68 /* version register */ -#define ENA_REGS_VERSION_MINOR_VERSION_MASK 0xff -#define ENA_REGS_VERSION_MAJOR_VERSION_SHIFT 8 -#define ENA_REGS_VERSION_MAJOR_VERSION_MASK 0xff00 +#define ENA_REGS_VERSION_MINOR_VERSION_MASK 0xff +#define ENA_REGS_VERSION_MAJOR_VERSION_SHIFT 8 +#define ENA_REGS_VERSION_MAJOR_VERSION_MASK 0xff00 /* controller_version register */ -#define ENA_REGS_CONTROLLER_VERSION_SUBMINOR_VERSION_MASK 0xff -#define ENA_REGS_CONTROLLER_VERSION_MINOR_VERSION_SHIFT 8 -#define ENA_REGS_CONTROLLER_VERSION_MINOR_VERSION_MASK 0xff00 -#define ENA_REGS_CONTROLLER_VERSION_MAJOR_VERSION_SHIFT 16 -#define ENA_REGS_CONTROLLER_VERSION_MAJOR_VERSION_MASK 0xff0000 -#define ENA_REGS_CONTROLLER_VERSION_IMPL_ID_SHIFT 24 -#define ENA_REGS_CONTROLLER_VERSION_IMPL_ID_MASK 0xff000000 +#define ENA_REGS_CONTROLLER_VERSION_SUBMINOR_VERSION_MASK 0xff +#define ENA_REGS_CONTROLLER_VERSION_MINOR_VERSION_SHIFT 8 +#define ENA_REGS_CONTROLLER_VERSION_MINOR_VERSION_MASK 0xff00 +#define ENA_REGS_CONTROLLER_VERSION_MAJOR_VERSION_SHIFT 16 +#define ENA_REGS_CONTROLLER_VERSION_MAJOR_VERSION_MASK 0xff0000 +#define ENA_REGS_CONTROLLER_VERSION_IMPL_ID_SHIFT 24 +#define ENA_REGS_CONTROLLER_VERSION_IMPL_ID_MASK 0xff000000 /* caps register */ -#define ENA_REGS_CAPS_CONTIGUOUS_QUEUE_REQUIRED_MASK 0x1 -#define ENA_REGS_CAPS_RESET_TIMEOUT_SHIFT 1 -#define ENA_REGS_CAPS_RESET_TIMEOUT_MASK 0x3e -#define ENA_REGS_CAPS_DMA_ADDR_WIDTH_SHIFT 8 -#define ENA_REGS_CAPS_DMA_ADDR_WIDTH_MASK 0xff00 -#define ENA_REGS_CAPS_ADMIN_CMD_TO_SHIFT 16 -#define ENA_REGS_CAPS_ADMIN_CMD_TO_MASK 0xf0000 +#define ENA_REGS_CAPS_CONTIGUOUS_QUEUE_REQUIRED_MASK 0x1 +#define ENA_REGS_CAPS_RESET_TIMEOUT_SHIFT 1 +#define ENA_REGS_CAPS_RESET_TIMEOUT_MASK 0x3e +#define ENA_REGS_CAPS_DMA_ADDR_WIDTH_SHIFT 8 +#define ENA_REGS_CAPS_DMA_ADDR_WIDTH_MASK 0xff00 +#define ENA_REGS_CAPS_ADMIN_CMD_TO_SHIFT 16 +#define ENA_REGS_CAPS_ADMIN_CMD_TO_MASK 0xf0000 /* aq_caps register */ -#define ENA_REGS_AQ_CAPS_AQ_DEPTH_MASK 0xffff -#define ENA_REGS_AQ_CAPS_AQ_ENTRY_SIZE_SHIFT 16 -#define ENA_REGS_AQ_CAPS_AQ_ENTRY_SIZE_MASK 0xffff0000 +#define ENA_REGS_AQ_CAPS_AQ_DEPTH_MASK 0xffff +#define ENA_REGS_AQ_CAPS_AQ_ENTRY_SIZE_SHIFT 16 +#define ENA_REGS_AQ_CAPS_AQ_ENTRY_SIZE_MASK 0xffff0000 /* acq_caps register */ -#define ENA_REGS_ACQ_CAPS_ACQ_DEPTH_MASK 0xffff -#define ENA_REGS_ACQ_CAPS_ACQ_ENTRY_SIZE_SHIFT 16 -#define ENA_REGS_ACQ_CAPS_ACQ_ENTRY_SIZE_MASK 0xffff0000 +#define ENA_REGS_ACQ_CAPS_ACQ_DEPTH_MASK 0xffff +#define ENA_REGS_ACQ_CAPS_ACQ_ENTRY_SIZE_SHIFT 16 +#define ENA_REGS_ACQ_CAPS_ACQ_ENTRY_SIZE_MASK 0xffff0000 /* aenq_caps register */ -#define ENA_REGS_AENQ_CAPS_AENQ_DEPTH_MASK 0xffff -#define ENA_REGS_AENQ_CAPS_AENQ_ENTRY_SIZE_SHIFT 16 -#define ENA_REGS_AENQ_CAPS_AENQ_ENTRY_SIZE_MASK 0xffff0000 +#define ENA_REGS_AENQ_CAPS_AENQ_DEPTH_MASK 0xffff +#define ENA_REGS_AENQ_CAPS_AENQ_ENTRY_SIZE_SHIFT 16 +#define ENA_REGS_AENQ_CAPS_AENQ_ENTRY_SIZE_MASK 0xffff0000 /* dev_ctl register */ -#define ENA_REGS_DEV_CTL_DEV_RESET_MASK 0x1 -#define ENA_REGS_DEV_CTL_AQ_RESTART_SHIFT 1 -#define ENA_REGS_DEV_CTL_AQ_RESTART_MASK 0x2 -#define ENA_REGS_DEV_CTL_QUIESCENT_SHIFT 2 -#define ENA_REGS_DEV_CTL_QUIESCENT_MASK 0x4 -#define ENA_REGS_DEV_CTL_IO_RESUME_SHIFT 3 -#define ENA_REGS_DEV_CTL_IO_RESUME_MASK 0x8 -#define ENA_REGS_DEV_CTL_RESET_REASON_SHIFT 28 -#define ENA_REGS_DEV_CTL_RESET_REASON_MASK 0xf0000000 +#define ENA_REGS_DEV_CTL_DEV_RESET_MASK 0x1 +#define ENA_REGS_DEV_CTL_AQ_RESTART_SHIFT 1 +#define ENA_REGS_DEV_CTL_AQ_RESTART_MASK 0x2 +#define ENA_REGS_DEV_CTL_QUIESCENT_SHIFT 2 +#define ENA_REGS_DEV_CTL_QUIESCENT_MASK 0x4 +#define ENA_REGS_DEV_CTL_IO_RESUME_SHIFT 3 +#define ENA_REGS_DEV_CTL_IO_RESUME_MASK 0x8 +#define ENA_REGS_DEV_CTL_RESET_REASON_SHIFT 28 +#define ENA_REGS_DEV_CTL_RESET_REASON_MASK 0xf0000000 /* dev_sts register */ -#define ENA_REGS_DEV_STS_READY_MASK 0x1 -#define ENA_REGS_DEV_STS_AQ_RESTART_IN_PROGRESS_SHIFT 1 -#define ENA_REGS_DEV_STS_AQ_RESTART_IN_PROGRESS_MASK 0x2 -#define ENA_REGS_DEV_STS_AQ_RESTART_FINISHED_SHIFT 2 -#define ENA_REGS_DEV_STS_AQ_RESTART_FINISHED_MASK 0x4 -#define ENA_REGS_DEV_STS_RESET_IN_PROGRESS_SHIFT 3 -#define ENA_REGS_DEV_STS_RESET_IN_PROGRESS_MASK 0x8 -#define ENA_REGS_DEV_STS_RESET_FINISHED_SHIFT 4 -#define ENA_REGS_DEV_STS_RESET_FINISHED_MASK 0x10 -#define ENA_REGS_DEV_STS_FATAL_ERROR_SHIFT 5 -#define ENA_REGS_DEV_STS_FATAL_ERROR_MASK 0x20 -#define ENA_REGS_DEV_STS_QUIESCENT_STATE_IN_PROGRESS_SHIFT 6 -#define ENA_REGS_DEV_STS_QUIESCENT_STATE_IN_PROGRESS_MASK 0x40 -#define ENA_REGS_DEV_STS_QUIESCENT_STATE_ACHIEVED_SHIFT 7 -#define ENA_REGS_DEV_STS_QUIESCENT_STATE_ACHIEVED_MASK 0x80 +#define ENA_REGS_DEV_STS_READY_MASK 0x1 +#define ENA_REGS_DEV_STS_AQ_RESTART_IN_PROGRESS_SHIFT 1 +#define ENA_REGS_DEV_STS_AQ_RESTART_IN_PROGRESS_MASK 0x2 +#define ENA_REGS_DEV_STS_AQ_RESTART_FINISHED_SHIFT 2 +#define ENA_REGS_DEV_STS_AQ_RESTART_FINISHED_MASK 0x4 +#define ENA_REGS_DEV_STS_RESET_IN_PROGRESS_SHIFT 3 +#define ENA_REGS_DEV_STS_RESET_IN_PROGRESS_MASK 0x8 +#define ENA_REGS_DEV_STS_RESET_FINISHED_SHIFT 4 +#define ENA_REGS_DEV_STS_RESET_FINISHED_MASK 0x10 +#define ENA_REGS_DEV_STS_FATAL_ERROR_SHIFT 5 +#define ENA_REGS_DEV_STS_FATAL_ERROR_MASK 0x20 +#define ENA_REGS_DEV_STS_QUIESCENT_STATE_IN_PROGRESS_SHIFT 6 +#define ENA_REGS_DEV_STS_QUIESCENT_STATE_IN_PROGRESS_MASK 0x40 +#define ENA_REGS_DEV_STS_QUIESCENT_STATE_ACHIEVED_SHIFT 7 +#define ENA_REGS_DEV_STS_QUIESCENT_STATE_ACHIEVED_MASK 0x80 /* mmio_reg_read register */ -#define ENA_REGS_MMIO_REG_READ_REQ_ID_MASK 0xffff -#define ENA_REGS_MMIO_REG_READ_REG_OFF_SHIFT 16 -#define ENA_REGS_MMIO_REG_READ_REG_OFF_MASK 0xffff0000 +#define ENA_REGS_MMIO_REG_READ_REQ_ID_MASK 0xffff +#define ENA_REGS_MMIO_REG_READ_REG_OFF_SHIFT 16 +#define ENA_REGS_MMIO_REG_READ_REG_OFF_MASK 0xffff0000 /* rss_ind_entry_update register */ -#define ENA_REGS_RSS_IND_ENTRY_UPDATE_INDEX_MASK 0xffff -#define ENA_REGS_RSS_IND_ENTRY_UPDATE_CQ_IDX_SHIFT 16 -#define ENA_REGS_RSS_IND_ENTRY_UPDATE_CQ_IDX_MASK 0xffff0000 +#define ENA_REGS_RSS_IND_ENTRY_UPDATE_INDEX_MASK 0xffff +#define ENA_REGS_RSS_IND_ENTRY_UPDATE_CQ_IDX_SHIFT 16 +#define ENA_REGS_RSS_IND_ENTRY_UPDATE_CQ_IDX_MASK 0xffff0000 #endif /*_ENA_REGS_H_ */ diff --git a/drivers/net/ena/base/ena_eth_com.c b/drivers/net/ena/base/ena_eth_com.c index 4c4989a3f..9d250db7e 100644 --- a/drivers/net/ena/base/ena_eth_com.c +++ b/drivers/net/ena/base/ena_eth_com.c @@ -46,25 +46,21 @@ static inline struct ena_eth_io_rx_cdesc_base *ena_com_get_next_rx_cdesc( cdesc = (struct ena_eth_io_rx_cdesc_base *)(io_cq->cdesc_addr.virt_addr + (head_masked * io_cq->cdesc_entry_size_in_bytes)); - desc_phase = (READ_ONCE(cdesc->status) & ENA_ETH_IO_RX_CDESC_BASE_PHASE_MASK) >> + desc_phase = (READ_ONCE32(cdesc->status) & ENA_ETH_IO_RX_CDESC_BASE_PHASE_MASK) >> ENA_ETH_IO_RX_CDESC_BASE_PHASE_SHIFT; if (desc_phase != expected_phase) return NULL; - return cdesc; -} - -static inline void ena_com_cq_inc_head(struct ena_com_io_cq *io_cq) -{ - io_cq->head++; + /* Make sure we read the rest of the descriptor after the phase bit + * has been read + */ + dma_rmb(); - /* Switch phase bit in case of wrap around */ - if (unlikely((io_cq->head & (io_cq->q_depth - 1)) == 0)) - io_cq->phase ^= 1; + return cdesc; } -static inline void *get_sq_desc(struct ena_com_io_sq *io_sq) +static inline void *get_sq_desc_regular_queue(struct ena_com_io_sq *io_sq) { u16 tail_masked; u32 offset; @@ -76,49 +72,174 @@ static inline void *get_sq_desc(struct ena_com_io_sq *io_sq) return (void *)((uintptr_t)io_sq->desc_addr.virt_addr + offset); } -static inline void ena_com_copy_curr_sq_desc_to_dev(struct ena_com_io_sq *io_sq) +static inline int ena_com_write_bounce_buffer_to_dev(struct ena_com_io_sq *io_sq, + u8 *bounce_buffer) { - u16 tail_masked = io_sq->tail & (io_sq->q_depth - 1); - u32 offset = tail_masked * io_sq->desc_entry_size; + struct ena_com_llq_info *llq_info = &io_sq->llq_info; - /* In case this queue isn't a LLQ */ - if (io_sq->mem_queue_type == ENA_ADMIN_PLACEMENT_POLICY_HOST) - return; + u16 dst_tail_mask; + u32 dst_offset; - memcpy_toio(io_sq->desc_addr.pbuf_dev_addr + offset, - io_sq->desc_addr.virt_addr + offset, - io_sq->desc_entry_size); -} + dst_tail_mask = io_sq->tail & (io_sq->q_depth - 1); + dst_offset = dst_tail_mask * llq_info->desc_list_entry_size; + + if (is_llq_max_tx_burst_exists(io_sq)) { + if (!io_sq->entries_in_tx_burst_left) { + ena_trc_err("Error: trying to write an llq entry to a full llq entries cache\n"); + return ENA_COM_NO_SPACE; + } + + io_sq->entries_in_tx_burst_left--; + ena_trc_dbg("decreasing entries_in_tx_burst_left of queue %d to %d\n", + io_sq->qid, io_sq->entries_in_tx_burst_left); + } + + /* Make sure everything was written into the bounce buffer before + * writing the bounce buffer to the device + */ + wmb(); + + /* The line is completed. Copy it to dev */ + ENA_MEMCPY_TO_DEVICE_64(io_sq->desc_addr.pbuf_dev_addr + dst_offset, + bounce_buffer, + llq_info->desc_list_entry_size); -static inline void ena_com_sq_update_tail(struct ena_com_io_sq *io_sq) -{ io_sq->tail++; /* Switch phase bit in case of wrap around */ if (unlikely((io_sq->tail & (io_sq->q_depth - 1)) == 0)) io_sq->phase ^= 1; + + return ENA_COM_OK; } -static inline int ena_com_write_header(struct ena_com_io_sq *io_sq, - u8 *head_src, u16 header_len) +static inline int ena_com_write_header_to_bounce(struct ena_com_io_sq *io_sq, + u8 *header_src, + u16 header_len) { - u16 tail_masked = io_sq->tail & (io_sq->q_depth - 1); - u8 __iomem *dev_head_addr = - io_sq->header_addr + (tail_masked * io_sq->tx_max_header_size); + struct ena_com_llq_pkt_ctrl *pkt_ctrl = &io_sq->llq_buf_ctrl; + struct ena_com_llq_info *llq_info = &io_sq->llq_info; + u8 *bounce_buffer = pkt_ctrl->curr_bounce_buf; + u16 header_offset; if (io_sq->mem_queue_type == ENA_ADMIN_PLACEMENT_POLICY_HOST) return 0; - if (unlikely(!io_sq->header_addr)) { - ena_trc_err("Push buffer header ptr is NULL\n"); - return ENA_COM_INVAL; + header_offset = + llq_info->descs_num_before_header * io_sq->desc_entry_size; + + if (unlikely((header_offset + header_len) > llq_info->desc_list_entry_size)) { + ena_trc_err("trying to write header larger than llq entry can accommodate\n"); + return ENA_COM_FAULT; } - memcpy_toio(dev_head_addr, head_src, header_len); + if (unlikely(!bounce_buffer)) { + ena_trc_err("bounce buffer is NULL\n"); + return ENA_COM_FAULT; + } + + memcpy(bounce_buffer + header_offset, header_src, header_len); return 0; } +static inline void *get_sq_desc_llq(struct ena_com_io_sq *io_sq) +{ + struct ena_com_llq_pkt_ctrl *pkt_ctrl = &io_sq->llq_buf_ctrl; + u8 *bounce_buffer; + void *sq_desc; + + bounce_buffer = pkt_ctrl->curr_bounce_buf; + + if (unlikely(!bounce_buffer)) { + ena_trc_err("bounce buffer is NULL\n"); + return NULL; + } + + sq_desc = bounce_buffer + pkt_ctrl->idx * io_sq->desc_entry_size; + pkt_ctrl->idx++; + pkt_ctrl->descs_left_in_line--; + + return sq_desc; +} + +static inline int ena_com_close_bounce_buffer(struct ena_com_io_sq *io_sq) +{ + struct ena_com_llq_pkt_ctrl *pkt_ctrl = &io_sq->llq_buf_ctrl; + struct ena_com_llq_info *llq_info = &io_sq->llq_info; + int rc; + + if (io_sq->mem_queue_type == ENA_ADMIN_PLACEMENT_POLICY_HOST) + return ENA_COM_OK; + + /* bounce buffer was used, so write it and get a new one */ + if (pkt_ctrl->idx) { + rc = ena_com_write_bounce_buffer_to_dev(io_sq, + pkt_ctrl->curr_bounce_buf); + if (unlikely(rc)) + return rc; + + pkt_ctrl->curr_bounce_buf = + ena_com_get_next_bounce_buffer(&io_sq->bounce_buf_ctrl); + memset(io_sq->llq_buf_ctrl.curr_bounce_buf, + 0x0, llq_info->desc_list_entry_size); + } + + pkt_ctrl->idx = 0; + pkt_ctrl->descs_left_in_line = llq_info->descs_num_before_header; + return ENA_COM_OK; +} + +static inline void *get_sq_desc(struct ena_com_io_sq *io_sq) +{ + if (io_sq->mem_queue_type == ENA_ADMIN_PLACEMENT_POLICY_DEV) + return get_sq_desc_llq(io_sq); + + return get_sq_desc_regular_queue(io_sq); +} + +static inline int ena_com_sq_update_llq_tail(struct ena_com_io_sq *io_sq) +{ + struct ena_com_llq_pkt_ctrl *pkt_ctrl = &io_sq->llq_buf_ctrl; + struct ena_com_llq_info *llq_info = &io_sq->llq_info; + int rc; + + if (!pkt_ctrl->descs_left_in_line) { + rc = ena_com_write_bounce_buffer_to_dev(io_sq, + pkt_ctrl->curr_bounce_buf); + if (unlikely(rc)) + return rc; + + pkt_ctrl->curr_bounce_buf = + ena_com_get_next_bounce_buffer(&io_sq->bounce_buf_ctrl); + memset(io_sq->llq_buf_ctrl.curr_bounce_buf, + 0x0, llq_info->desc_list_entry_size); + + pkt_ctrl->idx = 0; + if (llq_info->desc_stride_ctrl == ENA_ADMIN_SINGLE_DESC_PER_ENTRY) + pkt_ctrl->descs_left_in_line = 1; + else + pkt_ctrl->descs_left_in_line = + llq_info->desc_list_entry_size / io_sq->desc_entry_size; + } + + return ENA_COM_OK; +} + +static inline int ena_com_sq_update_tail(struct ena_com_io_sq *io_sq) +{ + if (io_sq->mem_queue_type == ENA_ADMIN_PLACEMENT_POLICY_DEV) + return ena_com_sq_update_llq_tail(io_sq); + + io_sq->tail++; + + /* Switch phase bit in case of wrap around */ + if (unlikely((io_sq->tail & (io_sq->q_depth - 1)) == 0)) + io_sq->phase ^= 1; + + return ENA_COM_OK; +} + static inline struct ena_eth_io_rx_cdesc_base * ena_com_rx_cdesc_idx_to_ptr(struct ena_com_io_cq *io_cq, u16 idx) { @@ -142,7 +263,7 @@ static inline u16 ena_com_cdesc_rx_pkt_get(struct ena_com_io_cq *io_cq, ena_com_cq_inc_head(io_cq); count++; - last = (READ_ONCE(cdesc->status) & ENA_ETH_IO_RX_CDESC_BASE_LAST_MASK) >> + last = (READ_ONCE32(cdesc->status) & ENA_ETH_IO_RX_CDESC_BASE_LAST_MASK) >> ENA_ETH_IO_RX_CDESC_BASE_LAST_SHIFT; } while (!last); @@ -182,8 +303,8 @@ static inline bool ena_com_meta_desc_changed(struct ena_com_io_sq *io_sq, return false; } -static inline void ena_com_create_and_store_tx_meta_desc(struct ena_com_io_sq *io_sq, - struct ena_com_tx_ctx *ena_tx_ctx) +static inline int ena_com_create_and_store_tx_meta_desc(struct ena_com_io_sq *io_sq, + struct ena_com_tx_ctx *ena_tx_ctx) { struct ena_eth_io_tx_meta_desc *meta_desc = NULL; struct ena_com_tx_meta *ena_meta = &ena_tx_ctx->ena_meta; @@ -228,8 +349,7 @@ static inline void ena_com_create_and_store_tx_meta_desc(struct ena_com_io_sq *i memcpy(&io_sq->cached_tx_meta, ena_meta, sizeof(struct ena_com_tx_meta)); - ena_com_copy_curr_sq_desc_to_dev(io_sq); - ena_com_sq_update_tail(io_sq); + return ena_com_sq_update_tail(io_sq); } static inline void ena_com_rx_set_flags(struct ena_com_rx_ctx *ena_rx_ctx, @@ -241,11 +361,14 @@ static inline void ena_com_rx_set_flags(struct ena_com_rx_ctx *ena_rx_ctx, (cdesc->status & ENA_ETH_IO_RX_CDESC_BASE_L4_PROTO_IDX_MASK) >> ENA_ETH_IO_RX_CDESC_BASE_L4_PROTO_IDX_SHIFT; ena_rx_ctx->l3_csum_err = - (cdesc->status & ENA_ETH_IO_RX_CDESC_BASE_L3_CSUM_ERR_MASK) >> - ENA_ETH_IO_RX_CDESC_BASE_L3_CSUM_ERR_SHIFT; + !!((cdesc->status & ENA_ETH_IO_RX_CDESC_BASE_L3_CSUM_ERR_MASK) >> + ENA_ETH_IO_RX_CDESC_BASE_L3_CSUM_ERR_SHIFT); ena_rx_ctx->l4_csum_err = - (cdesc->status & ENA_ETH_IO_RX_CDESC_BASE_L4_CSUM_ERR_MASK) >> - ENA_ETH_IO_RX_CDESC_BASE_L4_CSUM_ERR_SHIFT; + !!((cdesc->status & ENA_ETH_IO_RX_CDESC_BASE_L4_CSUM_ERR_MASK) >> + ENA_ETH_IO_RX_CDESC_BASE_L4_CSUM_ERR_SHIFT); + ena_rx_ctx->l4_csum_checked = + !!((cdesc->status & ENA_ETH_IO_RX_CDESC_BASE_L4_CSUM_CHECKED_MASK) >> + ENA_ETH_IO_RX_CDESC_BASE_L4_CSUM_CHECKED_SHIFT); ena_rx_ctx->hash = cdesc->hash; ena_rx_ctx->frag = (cdesc->status & ENA_ETH_IO_RX_CDESC_BASE_IPV4_FRAG_MASK) >> @@ -265,16 +388,50 @@ static inline void ena_com_rx_set_flags(struct ena_com_rx_ctx *ena_rx_ctx, /***************************** API **********************************/ /*****************************************************************************/ +bool ena_com_is_doorbell_needed(struct ena_com_io_sq *io_sq, + struct ena_com_tx_ctx *ena_tx_ctx) +{ + u16 num_descs; + int num_entries_needed; + int descs_after_first_entry; + bool have_meta; + struct ena_com_llq_info *llq_info; + + if (!is_llq_max_tx_burst_exists(io_sq)) + return false; + + num_entries_needed = 1; + llq_info = &io_sq->llq_info; + num_descs = ena_tx_ctx->num_bufs; + have_meta = ena_tx_ctx->meta_valid && + ena_com_meta_desc_changed(io_sq, ena_tx_ctx); + + if (have_meta) + ++num_descs; + + if (num_descs > llq_info->descs_num_before_header) { + descs_after_first_entry = num_descs - llq_info->descs_num_before_header; + num_entries_needed += DIV_ROUND_UP(descs_after_first_entry, + llq_info->descs_per_entry); + } + + ena_trc_dbg("queue: %d num_descs: %d num_entries_needed: %d\n", + io_sq->qid, num_descs, num_entries_needed); + + return num_entries_needed > io_sq->entries_in_tx_burst_left; +} + int ena_com_prepare_tx(struct ena_com_io_sq *io_sq, struct ena_com_tx_ctx *ena_tx_ctx, int *nb_hw_desc) { struct ena_eth_io_tx_desc *desc = NULL; struct ena_com_buf *ena_bufs = ena_tx_ctx->ena_bufs; - void *push_header = ena_tx_ctx->push_header; + void *buffer_to_push = ena_tx_ctx->push_header; u16 header_len = ena_tx_ctx->header_len; u16 num_bufs = ena_tx_ctx->num_bufs; - int total_desc, i, rc; + u16 start_tail = io_sq->tail; + int i, rc; bool have_meta; u64 addr_hi; @@ -282,8 +439,8 @@ int ena_com_prepare_tx(struct ena_com_io_sq *io_sq, "wrong Q type"); /* num_bufs +1 for potential meta desc */ - if (ena_com_sq_empty_space(io_sq) < (num_bufs + 1)) { - ena_trc_err("Not enough space in the tx queue\n"); + if (!ena_com_sq_have_enough_space(io_sq, num_bufs + 1)) { + ena_trc_dbg("Not enough space in the tx queue\n"); return ENA_COM_NO_MEM; } @@ -293,23 +450,32 @@ int ena_com_prepare_tx(struct ena_com_io_sq *io_sq, return ENA_COM_INVAL; } - /* start with pushing the header (if needed) */ - rc = ena_com_write_header(io_sq, push_header, header_len); + if (unlikely((io_sq->mem_queue_type == ENA_ADMIN_PLACEMENT_POLICY_DEV) + && !buffer_to_push)) + return ENA_COM_INVAL; + + rc = ena_com_write_header_to_bounce(io_sq, buffer_to_push, header_len); if (unlikely(rc)) return rc; have_meta = ena_tx_ctx->meta_valid && ena_com_meta_desc_changed(io_sq, ena_tx_ctx); - if (have_meta) - ena_com_create_and_store_tx_meta_desc(io_sq, ena_tx_ctx); + if (have_meta) { + rc = ena_com_create_and_store_tx_meta_desc(io_sq, ena_tx_ctx); + if (unlikely(rc)) + return rc; + } - /* If the caller doesn't want send packets */ + /* If the caller doesn't want to send packets */ if (unlikely(!num_bufs && !header_len)) { - *nb_hw_desc = have_meta ? 0 : 1; - return 0; + rc = ena_com_close_bounce_buffer(io_sq); + *nb_hw_desc = io_sq->tail - start_tail; + return rc; } desc = get_sq_desc(io_sq); + if (unlikely(!desc)) + return ENA_COM_FAULT; memset(desc, 0x0, sizeof(struct ena_eth_io_tx_desc)); /* Set first desc when we don't have meta descriptor */ @@ -361,10 +527,14 @@ int ena_com_prepare_tx(struct ena_com_io_sq *io_sq, for (i = 0; i < num_bufs; i++) { /* The first desc share the same desc as the header */ if (likely(i != 0)) { - ena_com_copy_curr_sq_desc_to_dev(io_sq); - ena_com_sq_update_tail(io_sq); + rc = ena_com_sq_update_tail(io_sq); + if (unlikely(rc)) + return rc; desc = get_sq_desc(io_sq); + if (unlikely(!desc)) + return ENA_COM_FAULT; + memset(desc, 0x0, sizeof(struct ena_eth_io_tx_desc)); desc->len_ctrl |= (io_sq->phase << @@ -387,15 +557,14 @@ int ena_com_prepare_tx(struct ena_com_io_sq *io_sq, /* set the last desc indicator */ desc->len_ctrl |= ENA_ETH_IO_TX_DESC_LAST_MASK; - ena_com_copy_curr_sq_desc_to_dev(io_sq); - - ena_com_sq_update_tail(io_sq); + rc = ena_com_sq_update_tail(io_sq); + if (unlikely(rc)) + return rc; - total_desc = ENA_MAX16(num_bufs, 1); - total_desc += have_meta ? 1 : 0; + rc = ena_com_close_bounce_buffer(io_sq); - *nb_hw_desc = total_desc; - return 0; + *nb_hw_desc = io_sq->tail - start_tail; + return rc; } int ena_com_rx_pkt(struct ena_com_io_cq *io_cq, @@ -456,10 +625,13 @@ int ena_com_add_single_rx_desc(struct ena_com_io_sq *io_sq, ENA_WARN(io_sq->direction != ENA_COM_IO_QUEUE_DIRECTION_RX, "wrong Q type"); - if (unlikely(ena_com_sq_empty_space(io_sq) == 0)) + if (unlikely(!ena_com_sq_have_enough_space(io_sq, 1))) return ENA_COM_NO_SPACE; desc = get_sq_desc(io_sq); + if (unlikely(!desc)) + return ENA_COM_FAULT; + memset(desc, 0x0, sizeof(struct ena_eth_io_rx_desc)); desc->length = ena_buf->len; @@ -475,42 +647,7 @@ int ena_com_add_single_rx_desc(struct ena_com_io_sq *io_sq, desc->buff_addr_hi = ((ena_buf->paddr & GENMASK_ULL(io_sq->dma_addr_bits - 1, 32)) >> 32); - ena_com_sq_update_tail(io_sq); - - return 0; -} - -int ena_com_tx_comp_req_id_get(struct ena_com_io_cq *io_cq, u16 *req_id) -{ - u8 expected_phase, cdesc_phase; - struct ena_eth_io_tx_cdesc *cdesc; - u16 masked_head; - - masked_head = io_cq->head & (io_cq->q_depth - 1); - expected_phase = io_cq->phase; - - cdesc = (struct ena_eth_io_tx_cdesc *) - ((uintptr_t)io_cq->cdesc_addr.virt_addr + - (masked_head * io_cq->cdesc_entry_size_in_bytes)); - - /* When the current completion descriptor phase isn't the same as the - * expected, it mean that the device still didn't update - * this completion. - */ - cdesc_phase = READ_ONCE(cdesc->flags) & ENA_ETH_IO_TX_CDESC_PHASE_MASK; - if (cdesc_phase != expected_phase) - return ENA_COM_TRY_AGAIN; - - if (unlikely(cdesc->req_id >= io_cq->q_depth)) { - ena_trc_err("Invalid req id %d\n", cdesc->req_id); - return ENA_COM_INVAL; - } - - ena_com_cq_inc_head(io_cq); - - *req_id = READ_ONCE(cdesc->req_id); - - return 0; + return ena_com_sq_update_tail(io_sq); } bool ena_com_cq_empty(struct ena_com_io_cq *io_cq) @@ -523,4 +660,3 @@ bool ena_com_cq_empty(struct ena_com_io_cq *io_cq) else return true; } - diff --git a/drivers/net/ena/base/ena_eth_com.h b/drivers/net/ena/base/ena_eth_com.h index 56ea4ae64..79acbf31d 100644 --- a/drivers/net/ena/base/ena_eth_com.h +++ b/drivers/net/ena/base/ena_eth_com.h @@ -71,6 +71,7 @@ struct ena_com_rx_ctx { enum ena_eth_io_l4_proto_index l4_proto; bool l3_csum_err; bool l4_csum_err; + bool l4_csum_checked; /* fragmented packet */ bool frag; u32 hash; @@ -78,6 +79,9 @@ struct ena_com_rx_ctx { int max_bufs; }; +bool ena_com_is_doorbell_needed(struct ena_com_io_sq *io_sq, + struct ena_com_tx_ctx *ena_tx_ctx); + int ena_com_prepare_tx(struct ena_com_io_sq *io_sq, struct ena_com_tx_ctx *ena_tx_ctx, int *nb_hw_desc); @@ -90,8 +94,6 @@ int ena_com_add_single_rx_desc(struct ena_com_io_sq *io_sq, struct ena_com_buf *ena_buf, u16 req_id); -int ena_com_tx_comp_req_id_get(struct ena_com_io_cq *io_cq, u16 *req_id); - bool ena_com_cq_empty(struct ena_com_io_cq *io_cq); static inline void ena_com_unmask_intr(struct ena_com_io_cq *io_cq, @@ -100,7 +102,7 @@ static inline void ena_com_unmask_intr(struct ena_com_io_cq *io_cq, ENA_REG_WRITE32(io_cq->bus, intr_reg->intr_control, io_cq->unmask_reg); } -static inline int ena_com_sq_empty_space(struct ena_com_io_sq *io_sq) +static inline int ena_com_free_desc(struct ena_com_io_sq *io_sq) { u16 tail, next_to_comp, cnt; @@ -111,17 +113,47 @@ static inline int ena_com_sq_empty_space(struct ena_com_io_sq *io_sq) return io_sq->q_depth - 1 - cnt; } -static inline int ena_com_write_sq_doorbell(struct ena_com_io_sq *io_sq) +/* Check if the submission queue has enough space to hold required_buffers */ +static inline bool ena_com_sq_have_enough_space(struct ena_com_io_sq *io_sq, + u16 required_buffers) { - u16 tail; + int temp; - tail = io_sq->tail; + if (io_sq->mem_queue_type == ENA_ADMIN_PLACEMENT_POLICY_HOST) + return ena_com_free_desc(io_sq) >= required_buffers; + + /* This calculation doesn't need to be 100% accurate. So to reduce + * the calculation overhead just Subtract 2 lines from the free descs + * (one for the header line and one to compensate the devision + * down calculation. + */ + temp = required_buffers / io_sq->llq_info.descs_per_entry + 2; + + return ena_com_free_desc(io_sq) > temp; +} + +static inline bool is_llq_max_tx_burst_exists(struct ena_com_io_sq *io_sq) +{ + return (io_sq->mem_queue_type == ENA_ADMIN_PLACEMENT_POLICY_DEV) && + io_sq->llq_info.max_entries_in_tx_burst > 0; +} + +static inline int ena_com_write_sq_doorbell(struct ena_com_io_sq *io_sq) +{ + u16 tail = io_sq->tail; + u16 max_entries_in_tx_burst = io_sq->llq_info.max_entries_in_tx_burst; ena_trc_dbg("write submission queue doorbell for queue: %d tail: %d\n", io_sq->qid, tail); ENA_REG_WRITE32(io_sq->bus, tail, io_sq->db_addr); + if (is_llq_max_tx_burst_exists(io_sq)) { + ena_trc_dbg("reset available entries in tx burst for queue %d to %d\n", + io_sq->qid, max_entries_in_tx_burst); + io_sq->entries_in_tx_burst_left = max_entries_in_tx_burst; + } + return 0; } @@ -163,6 +195,49 @@ static inline void ena_com_comp_ack(struct ena_com_io_sq *io_sq, u16 elem) io_sq->next_to_comp += elem; } +static inline void ena_com_cq_inc_head(struct ena_com_io_cq *io_cq) +{ + io_cq->head++; + + /* Switch phase bit in case of wrap around */ + if (unlikely((io_cq->head & (io_cq->q_depth - 1)) == 0)) + io_cq->phase ^= 1; +} + +static inline int ena_com_tx_comp_req_id_get(struct ena_com_io_cq *io_cq, u16 *req_id) +{ + u8 expected_phase, cdesc_phase; + struct ena_eth_io_tx_cdesc *cdesc; + u16 masked_head; + + masked_head = io_cq->head & (io_cq->q_depth - 1); + expected_phase = io_cq->phase; + + cdesc = (struct ena_eth_io_tx_cdesc *) + ((uintptr_t)io_cq->cdesc_addr.virt_addr + + (masked_head * io_cq->cdesc_entry_size_in_bytes)); + + /* When the current completion descriptor phase isn't the same as the + * expected, it mean that the device still didn't update + * this completion. + */ + cdesc_phase = READ_ONCE16(cdesc->flags) & ENA_ETH_IO_TX_CDESC_PHASE_MASK; + if (cdesc_phase != expected_phase) + return ENA_COM_TRY_AGAIN; + + dma_rmb(); + + *req_id = READ_ONCE16(cdesc->req_id); + if (unlikely(*req_id >= io_cq->q_depth)) { + ena_trc_err("Invalid req id %d\n", cdesc->req_id); + return ENA_COM_INVAL; + } + + ena_com_cq_inc_head(io_cq); + + return 0; +} + #if defined(__cplusplus) } #endif diff --git a/drivers/net/ena/base/ena_plat.h b/drivers/net/ena/base/ena_plat.h index f829936b2..d57de057c 100644 --- a/drivers/net/ena/base/ena_plat.h +++ b/drivers/net/ena/base/ena_plat.h @@ -35,21 +35,21 @@ #define ENA_PLAT_H_ #if defined(ENA_IPXE) -#include "ena_plat_ipxe.h" +#include #elif defined(__linux__) #if defined(__KERNEL__) -#include "ena_plat_linux.h" +#include #else -#include "ena_plat_dpdk.h" +#include #endif #elif defined(__FreeBSD__) #if defined(_KERNEL) -#include "ena_plat_fbsd.h" +#include #else -#include "ena_plat_dpdk.h" +#include #endif #elif defined(_WIN32) -#include "ena_plat_windows.h" +#include #else #error "Invalid platform" #endif diff --git a/drivers/net/ena/base/ena_plat_dpdk.h b/drivers/net/ena/base/ena_plat_dpdk.h index 900ba1a6b..902d91efb 100644 --- a/drivers/net/ena/base/ena_plat_dpdk.h +++ b/drivers/net/ena/base/ena_plat_dpdk.h @@ -48,6 +48,7 @@ #include #include #include +#include #include #include @@ -88,8 +89,9 @@ typedef uint64_t dma_addr_t; #define ENA_TOUCH(x) ((void)(x)) #define memcpy_toio memcpy #define wmb rte_wmb -#define rmb rte_wmb +#define rmb rte_rmb #define mb rte_mb +#define mmiowb rte_io_wmb #define __iomem #define US_PER_S 1000000 @@ -156,6 +158,7 @@ do { \ ({(void)flags; rte_spinlock_lock(&spinlock); }) #define ENA_SPINLOCK_UNLOCK(spinlock, flags) \ ({(void)flags; rte_spinlock_unlock(&(spinlock)); }) +#define ENA_SPINLOCK_DESTROY(spinlock) ((void)spinlock) #define q_waitqueue_t \ struct { \ @@ -257,7 +260,11 @@ extern uint32_t ena_alloc_cnt; #define ENA_MEM_ALLOC(dmadev, size) rte_zmalloc(NULL, size, 1) #define ENA_MEM_FREE(dmadev, ptr) ({ENA_TOUCH(dmadev); rte_free(ptr); }) +#define ENA_DB_SYNC(mem_handle) ((void)mem_handle) + #define ENA_REG_WRITE32(bus, value, reg) \ + ({ (void)(bus); rte_write32((value), (reg)); }) +#define ENA_REG_WRITE32_RELAXED(bus, value, reg) \ ({ (void)(bus); rte_write32_relaxed((value), (reg)); }) #define ENA_REG_READ32(bus, reg) \ ({ (void)(bus); rte_read32_relaxed((reg)); }) @@ -270,17 +277,43 @@ extern uint32_t ena_alloc_cnt; #define msleep(x) rte_delay_us(x * 1000) #define udelay(x) rte_delay_us(x) +#define dma_rmb() rmb() + #define MAX_ERRNO 4095 #define IS_ERR(x) (((unsigned long)x) >= (unsigned long)-MAX_ERRNO) #define ERR_PTR(error) ((void *)(long)error) #define PTR_ERR(error) ((long)(void *)error) #define might_sleep() +#define prefetch(x) rte_prefetch0(x) + #define lower_32_bits(x) ((uint32_t)(x)) #define upper_32_bits(x) ((uint32_t)(((x) >> 16) >> 16)) +#define ENA_TIME_EXPIRE(timeout) (timeout < rte_get_timer_cycles()) +#define ENA_GET_SYSTEM_TIMEOUT(timeout_us) \ + (timeout_us * rte_get_timer_hz() / 1000000 + rte_get_timer_cycles()) +#define ENA_WAIT_EVENT_DESTROY(waitqueue) ((void)(waitqueue)) + #ifndef READ_ONCE #define READ_ONCE(var) (*((volatile typeof(var) *)(&(var)))) #endif +#define READ_ONCE8(var) READ_ONCE(var) +#define READ_ONCE16(var) READ_ONCE(var) +#define READ_ONCE32(var) READ_ONCE(var) + +/* The size must be 8 byte align */ +#define ENA_MEMCPY_TO_DEVICE_64(dst, src, size) \ + do { \ + int count, i; \ + uint64_t *to = (uint64_t *)(dst); \ + const uint64_t *from = (const uint64_t *)(src); \ + count = (size) / 8; \ + for (i = 0; i < count; i++, from++, to++) \ + rte_write64_relaxed(*from, to); \ + } while(0) + +#define DIV_ROUND_UP(n, d) (((n) + (d) - 1) / (d)) + #endif /* DPDK_ENA_COM_ENA_PLAT_DPDK_H_ */ diff --git a/drivers/net/ena/ena_ethdev.c b/drivers/net/ena/ena_ethdev.c index 709ad2edb..667820806 100644 --- a/drivers/net/ena/ena_ethdev.c +++ b/drivers/net/ena/ena_ethdev.c @@ -947,7 +947,7 @@ ena_calc_queue_size(struct ena_com_dev *ena_dev, if (ena_dev->tx_mem_queue_type == ENA_ADMIN_PLACEMENT_POLICY_DEV) queue_size = RTE_MIN(queue_size, - get_feat_ctx->max_queues.max_llq_depth); + get_feat_ctx->max_queues.max_legacy_llq_depth); /* Round down to power of 2 */ if (!rte_is_power_of_2(queue_size)) @@ -1472,7 +1472,7 @@ static int ena_device_init(struct ena_com_dev *ena_dev, ena_dev->dma_addr_bits = ena_com_get_dma_width(ena_dev); /* ENA device administration layer init */ - rc = ena_com_admin_init(ena_dev, &aenq_handlers, true); + rc = ena_com_admin_init(ena_dev, &aenq_handlers); if (rc) { RTE_LOG(ERR, PMD, "cannot initialize ena admin queue with device\n"); From patchwork Fri Dec 14 13:18:28 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michal Krawczyk X-Patchwork-Id: 48859 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 53C0C1BB12; Fri, 14 Dec 2018 14:18:59 +0100 (CET) Received: from mail-lf1-f65.google.com (mail-lf1-f65.google.com [209.85.167.65]) by dpdk.org (Postfix) with ESMTP id 47BB81BB12 for ; Fri, 14 Dec 2018 14:18:58 +0100 (CET) Received: by mail-lf1-f65.google.com with SMTP id f23so4201680lfc.13 for ; Fri, 14 Dec 2018 05:18:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=semihalf-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=Glh/Z6iS10V58dXO5CqGQKx2d+tX3Ft4L192baiyHwc=; b=TUPqeTt/iiZVUK/vrjP1PW7XGq90fys76jP8G3Stuyq96DJSW0pZ5TtdWK8ntcImvD gysMXb0idJbpMq9UOFiPVFNw6BJWs9ZW8geUTRmpbTa6PlkhAX8vhZPFBqx+996+LbIl q/AOfErjqtQvfLkMseU7ETcN8qqxfdtATGximtODQmtcTB5VmPmIAyH9oqS8SzG+vlJw WEqmPI3XLti2g/RGTgR42MxaqOZzasBEo6kzMAiErUVJhBQrWgPmeFEEyBwHNDquwOn3 K4ZUrTkN5R1Feo142wjnyXgvzWZ/ijPZeITXYfhAb2/KpEzA9aoS+w06qFu0Ck8TL/9K nv0w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=Glh/Z6iS10V58dXO5CqGQKx2d+tX3Ft4L192baiyHwc=; b=gomFsjUkAk1RrBkJ+91qJMpj+ob0GHYhetHzb8Pl90OG/FyOfCAXhKqlotD7K6cAog Gc1WE7NFwOxcYezsq9Otewst39CcYcbt7UOn9XQiL+zOaDT4pDMszz8uRETYz24B8PIk b9nvpO5Dj9DFFUs/XRBLNe8jqRNAKVAklNnTDctZmpAOvnFThRr+v/ZeP3JsI42sXqqs bTE/IDXP/lUZ3mU9IUEz4NWXASX4nEdSPoL1wA+On0MU88NF4IP6gii5KnBk6Mzss7+I Me6HQoKcFDreq77ZpgIqv0gbymXYkDWunfpQ57f2rYTCqczHmh7detmIvvPB4b7xkRNU 7VbQ== X-Gm-Message-State: AA+aEWbf6ozHRzG7HAnLKXx8thKbFXuv//oyn/bwNehosQQSyaMu8gp/ 9tub/O2K05lti19F/LKilUexfSOH0kE= X-Google-Smtp-Source: AFSGD/Wj0pDUIBJF7tnUloMdeki0oEHjz3fiSO/bcDTWzYWmtS3k2CK2ioHymoFXV6W/sdBDO065Kw== X-Received: by 2002:a19:1bd2:: with SMTP id b201mr1739276lfb.136.1544793537320; Fri, 14 Dec 2018 05:18:57 -0800 (PST) Received: from mkPC.semihalf.local (31-172-191-173.noc.fibertech.net.pl. [31.172.191.173]) by smtp.gmail.com with ESMTPSA id o25sm873884lfd.29.2018.12.14.05.18.55 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 14 Dec 2018 05:18:56 -0800 (PST) From: Michal Krawczyk To: dev@dpdk.org Cc: gtzalik@dpdk.org, mw@dpdk.org, matua@amazon.com, rk@semihalf.com Date: Fri, 14 Dec 2018 14:18:28 +0100 Message-Id: <20181214131846.22439-3-mk@semihalf.com> X-Mailer: git-send-email 2.14.1 In-Reply-To: <20181214131846.22439-1-mk@semihalf.com> References: <20181214131846.22439-1-mk@semihalf.com> Subject: [dpdk-dev] [PATCH 02/20] net/ena: pass number of CPUs to the host info structure X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Rafal Kozik The new ena_com allows the number of CPUs to be passed to the device in the host info structure. Signed-off-by: Rafal Kozik Acked-by: Michal Krawczyk --- drivers/net/ena/ena_ethdev.c | 1 + 1 file changed, 1 insertion(+) diff --git a/drivers/net/ena/ena_ethdev.c b/drivers/net/ena/ena_ethdev.c index 667820806..5e9666f74 100644 --- a/drivers/net/ena/ena_ethdev.c +++ b/drivers/net/ena/ena_ethdev.c @@ -437,6 +437,7 @@ static void ena_config_host_info(struct ena_com_dev *ena_dev) (DRV_MODULE_VER_MINOR << ENA_ADMIN_HOST_INFO_MINOR_SHIFT) | (DRV_MODULE_VER_SUBMINOR << ENA_ADMIN_HOST_INFO_SUB_MINOR_SHIFT); + host_info->num_cpus = rte_lcore_count(); rc = ena_com_set_host_attributes(ena_dev); if (rc) { From patchwork Fri Dec 14 13:18:29 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michal Krawczyk X-Patchwork-Id: 48861 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 7094B1BB4B; Fri, 14 Dec 2018 14:19:03 +0100 (CET) Received: from mail-lf1-f65.google.com (mail-lf1-f65.google.com [209.85.167.65]) by dpdk.org (Postfix) with ESMTP id 59DFA1BB15 for ; Fri, 14 Dec 2018 14:18:59 +0100 (CET) Received: by mail-lf1-f65.google.com with SMTP id i26so4249072lfc.0 for ; Fri, 14 Dec 2018 05:18:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=semihalf-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=aRaePE+NbjKUvY+DmEjCv0zMewmUA97zvOyHamScE5A=; b=YivkwBDSeksWGKs2eMXqtqwVxS/Oj4mqFjCa/0YbQp+YoIFIAEh3cdVeqPQccOOX0w X/EhWCSN/DoOGYQeIDURavo0KdbLw5/ctjOKt72oRillmyVqQd/E6y7VHK85hnhIH3p7 skQpjMRkPBu9L7OIawNMOIBkW6UDR6rBk5E+0MXQaeoKpMCawQmXRm/d8S9lN3mZKfkb nT6xgYesl9ZBHjDMv82tCZMhU36QrDkTHKbeGRtSwTE3zwlW4bN/LlWcS1ZTcwDxzGfk /b97ZLxWF0prHFS0FJ1Z5WxLJA0VEUMjMrk8spRWtiq+KVtoml9OsGVqz/qrASCoxrGI GHBg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=aRaePE+NbjKUvY+DmEjCv0zMewmUA97zvOyHamScE5A=; b=bRytVXTgLyyfpifQsWo+0Pabqu2my1slXYzcafKF+DLSb0DYmKVT7z2WUXn6TslC2Y g8EIg3uHWqkgiBxDc2wN5sSYmJEQSv96i/vYaWFnoYH2JGrWSae6cZBGjPAyyaOK5mhd Hihasefev8Jbx/GD9BWqjCS3vEzQmMcyoqEXjPuKZF9V9IJB5c5IOMw68v7OURQsBoJS 0ZOC5uuLbB5qU2dth6Ls6wFtmsP+CcyaZho66M2Gbsg/A8tmPU6SutNiOwWtO6Ok9pAN mxDst3PjTPVZM3K4OtrnSqXe2/BxvA/jZGbpA8VWHLM9cO90iWbPoAw756O6BUQi7x3E Dwhw== X-Gm-Message-State: AA+aEWauvPfLFGmFUXaRpoLNmE+cfQ/1ftpdVmX/IG2LYJaIT/5TQVSq 5wY5apzPiIPZx46QHo39E3MgjmtQ2lU= X-Google-Smtp-Source: AFSGD/UUklISkYyxxvdb7yyfGWtQbbsDFBlz1ZyfLooOPKRAtUwYUrj8mz+uohPcibqpFTgkTd09Mw== X-Received: by 2002:a19:7919:: with SMTP id u25mr1747954lfc.18.1544793538556; Fri, 14 Dec 2018 05:18:58 -0800 (PST) Received: from mkPC.semihalf.local (31-172-191-173.noc.fibertech.net.pl. [31.172.191.173]) by smtp.gmail.com with ESMTPSA id o25sm873884lfd.29.2018.12.14.05.18.57 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 14 Dec 2018 05:18:57 -0800 (PST) From: Michal Krawczyk To: dev@dpdk.org Cc: gtzalik@dpdk.org, mw@dpdk.org, matua@amazon.com, rk@semihalf.com, stable@dpdk.org Date: Fri, 14 Dec 2018 14:18:29 +0100 Message-Id: <20181214131846.22439-4-mk@semihalf.com> X-Mailer: git-send-email 2.14.1 In-Reply-To: <20181214131846.22439-1-mk@semihalf.com> References: <20181214131846.22439-1-mk@semihalf.com> Subject: [dpdk-dev] [PATCH 03/20] net/ena: add reset reason in rx error X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Rafal Kozik Whenever the driver will receive too many descriptors from the device, it should trigger the device reset with reset reason set to ENA_REGS_RESET_TOO_MANY_RX_DESCS. Fixes: 241da076b1f7 ("net/ena: adjust error checking and cleaning") Cc: stable@dpdk.org Signed-off-by: Rafal Kozik Acked-by: Michal Krawczyk --- drivers/net/ena/ena_ethdev.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/drivers/net/ena/ena_ethdev.c b/drivers/net/ena/ena_ethdev.c index 5e9666f74..a47f5f7fa 100644 --- a/drivers/net/ena/ena_ethdev.c +++ b/drivers/net/ena/ena_ethdev.c @@ -1910,6 +1910,8 @@ static uint16_t eth_ena_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, &ena_rx_ctx); if (unlikely(rc)) { RTE_LOG(ERR, PMD, "ena_com_rx_pkt error %d\n", rc); + rx_ring->adapter->reset_reason = + ENA_REGS_RESET_TOO_MANY_RX_DESCS; rx_ring->adapter->trigger_reset = true; return 0; } From patchwork Fri Dec 14 13:18:30 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michal Krawczyk X-Patchwork-Id: 48862 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id A847A1BB52; Fri, 14 Dec 2018 14:19:05 +0100 (CET) Received: from mail-lj1-f194.google.com (mail-lj1-f194.google.com [209.85.208.194]) by dpdk.org (Postfix) with ESMTP id 8C6F21BB2C for ; Fri, 14 Dec 2018 14:19:01 +0100 (CET) Received: by mail-lj1-f194.google.com with SMTP id 83-v6so4853018ljf.10 for ; Fri, 14 Dec 2018 05:19:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=semihalf-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=TE4anncdD5N/HP4qXHsu4Yksylt9sR59MlDZ8en19JI=; b=K3fWYmJ4oS6o+dTxMGt0E9YKUICl/URzEWI7ITVjQ5Xpj0eTR10pyExZVgwEr5QRAm 5CNdCzHk3+oqcILvb1Vzg2hVymftarixyRXE3iXK5ozFxV9AHZzTKvys2fPypKHyT6Vw 5qa5Axni/PjHIKs4qRxV4x+1+2nMoGEZWGkbCRQa5wwlfyc9aIRIjv+ID26+J9XeiSEL 3WvNu9lyDbTQGmynZhiVIoz2WsgagbZxYTKUNLk04tT6PZoh6AzMrsAcx8AYxS50+tX2 FMt9kQ5ZMABJbgqECrCxbl7/Q9VCIrV4wSQNArszro5K8klVSC9OeJXjlqsMWk+W8jSd n1tQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=TE4anncdD5N/HP4qXHsu4Yksylt9sR59MlDZ8en19JI=; b=BNdJrUbQtZvbItWS15rDJ9nwacWATjYHlIQCg4eiRGSUe5A+nKq4bfIAjrBkuVbQZ6 fLf/5AH4RvF1aXncmdA3iQ4p6RD/lbiLMUYqvHL4mBe+Ld6a5OPWDGkKbMoDd6lTDNIk SF/qB3noh1Q/bpGzl1E3NLrhSszQIi4dUJaumATuaxHmPkw48pEtdpb2npxgrh3sEL71 KrLe08HtKUH5HHtJ+Fn6pjImpwf7yoO9TeJ3tOWXTD/y27DQ8BmyeYH0W+qs+JBonQYc 9KhnoOUDOV37o0qw1l0ATQ7aiL6gTHBr86p41ueUtBTJDEu29oRHHsxGOQwBTUI3CsJy l8ng== X-Gm-Message-State: AA+aEWZfLdwgcbrCOOp9hm+1sSIg+5X9q0Y7Miil7LOZgXxoVnVqAUFe a/q3CCGXt1IMP3FluoB6rCLoWbRIP/A= X-Google-Smtp-Source: AFSGD/X8beYnchrNAbnJQtLvUIrGmKzcdcxIiB+N9Q478ZhtciEG7T3LTXjWj5ALg5o7ipXu8ATwGA== X-Received: by 2002:a2e:8156:: with SMTP id t22-v6mr1825736ljg.32.1544793540601; Fri, 14 Dec 2018 05:19:00 -0800 (PST) Received: from mkPC.semihalf.local (31-172-191-173.noc.fibertech.net.pl. [31.172.191.173]) by smtp.gmail.com with ESMTPSA id o25sm873884lfd.29.2018.12.14.05.18.58 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 14 Dec 2018 05:18:58 -0800 (PST) From: Michal Krawczyk To: dev@dpdk.org Cc: gtzalik@dpdk.org, mw@dpdk.org, matua@amazon.com, rk@semihalf.com Date: Fri, 14 Dec 2018 14:18:30 +0100 Message-Id: <20181214131846.22439-5-mk@semihalf.com> X-Mailer: git-send-email 2.14.1 In-Reply-To: <20181214131846.22439-1-mk@semihalf.com> References: <20181214131846.22439-1-mk@semihalf.com> Subject: [dpdk-dev] [PATCH 04/20] net/ena: add hw queues depth setup X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Rafal Kozik The device now allows driver to reconfigure Tx and Rx queues depth independently. Moreover, maximum size for Tx and Rx can be different. Those maximum values are received from the device. After reset, previous ring configuration is restored. If number of descriptor is set to RTE_ETH_DEV_FALLBACK_RX_RINGSIZE or RTE_ETH_DEV_FALLBACK_TX_RINGSIZE, the maximum value is restored. Remove checks, if provided number is not too big, as this is done in generic functions (rte_eth_rx_queue_setup and rte_eth_tx_queue_setup). Maximum number of segments is being set for Rx packets and provided to ena_com_rx_pkt() for validation. Unused definitions were removed. Signed-off-by: Rafal Kozik Acked-by: Michal Krawczyk --- drivers/net/ena/ena_ethdev.c | 125 ++++++++++++++++++++++++++++--------------- drivers/net/ena/ena_ethdev.h | 11 +++- 2 files changed, 93 insertions(+), 43 deletions(-) -- 2.14.1 diff --git a/drivers/net/ena/ena_ethdev.c b/drivers/net/ena/ena_ethdev.c index a47f5f7fa..617f85f02 100644 --- a/drivers/net/ena/ena_ethdev.c +++ b/drivers/net/ena/ena_ethdev.c @@ -85,7 +85,6 @@ #define ARRAY_SIZE(x) (sizeof(x) / sizeof((x)[0])) -#define ENA_MAX_RING_DESC ENA_DEFAULT_RING_SIZE #define ENA_MIN_RING_DESC 128 enum ethtool_stringset { @@ -591,11 +590,12 @@ ena_dev_reset(struct rte_eth_dev *dev) ena_com_admin_aenq_enable(ena_dev); for (i = 0; i < nb_queues; ++i) - ena_rx_queue_setup(eth_dev, i, adapter->rx_ring_size, 0, NULL, - mb_pool_rx[i]); + ena_rx_queue_setup(eth_dev, i, adapter->rx_ring[i].ring_size, 0, + NULL, mb_pool_rx[i]); for (i = 0; i < nb_queues; ++i) - ena_tx_queue_setup(eth_dev, i, adapter->tx_ring_size, 0, NULL); + ena_tx_queue_setup(eth_dev, i, adapter->tx_ring[i].ring_size, 0, + NULL); adapter->trigger_reset = false; @@ -935,34 +935,46 @@ static int ena_check_valid_conf(struct ena_adapter *adapter) } static int -ena_calc_queue_size(struct ena_com_dev *ena_dev, - u16 *max_tx_sgl_size, - struct ena_com_dev_get_features_ctx *get_feat_ctx) +ena_calc_queue_size(struct ena_calc_queue_size_ctx *ctx) { - uint32_t queue_size = ENA_DEFAULT_RING_SIZE; - - queue_size = RTE_MIN(queue_size, - get_feat_ctx->max_queues.max_cq_depth); - queue_size = RTE_MIN(queue_size, - get_feat_ctx->max_queues.max_sq_depth); - - if (ena_dev->tx_mem_queue_type == ENA_ADMIN_PLACEMENT_POLICY_DEV) - queue_size = RTE_MIN(queue_size, - get_feat_ctx->max_queues.max_legacy_llq_depth); - - /* Round down to power of 2 */ - if (!rte_is_power_of_2(queue_size)) - queue_size = rte_align32pow2(queue_size >> 1); - - if (unlikely(queue_size == 0)) { + uint32_t tx_queue_size, rx_queue_size; + + if (ctx->ena_dev->supported_features & BIT(ENA_ADMIN_MAX_QUEUES_EXT)) { + struct ena_admin_queue_ext_feature_fields *max_queue_ext = + &ctx->get_feat_ctx->max_queue_ext.max_queue_ext; + rx_queue_size = RTE_MIN(max_queue_ext->max_rx_cq_depth, + max_queue_ext->max_rx_sq_depth); + tx_queue_size = RTE_MIN(max_queue_ext->max_tx_cq_depth, + max_queue_ext->max_tx_sq_depth); + ctx->max_rx_sgl_size = RTE_MIN(ENA_PKT_MAX_BUFS, + max_queue_ext->max_per_packet_rx_descs); + ctx->max_tx_sgl_size = RTE_MIN(ENA_PKT_MAX_BUFS, + max_queue_ext->max_per_packet_tx_descs); + } else { + struct ena_admin_queue_feature_desc *max_queues = + &ctx->get_feat_ctx->max_queues; + rx_queue_size = RTE_MIN(max_queues->max_cq_depth, + max_queues->max_sq_depth); + tx_queue_size = rx_queue_size; + ctx->max_rx_sgl_size = RTE_MIN(ENA_PKT_MAX_BUFS, + max_queues->max_packet_tx_descs); + ctx->max_tx_sgl_size = RTE_MIN(ENA_PKT_MAX_BUFS, + max_queues->max_packet_rx_descs); + } + + /* Round down to the nearest power of 2 */ + rx_queue_size = rte_align32prevpow2(rx_queue_size); + tx_queue_size = rte_align32prevpow2(tx_queue_size); + + if (unlikely(rx_queue_size == 0 || tx_queue_size == 0)) { PMD_INIT_LOG(ERR, "Invalid queue size"); return -EFAULT; } - *max_tx_sgl_size = RTE_MIN(ENA_PKT_MAX_BUFS, - get_feat_ctx->max_queues.max_packet_tx_descs); + ctx->rx_queue_size = rx_queue_size; + ctx->tx_queue_size = tx_queue_size; - return queue_size; + return 0; } static void ena_stats_restart(struct rte_eth_dev *dev) @@ -1239,6 +1251,9 @@ static int ena_tx_queue_setup(struct rte_eth_dev *dev, return -EINVAL; } + if (nb_desc == RTE_ETH_DEV_FALLBACK_TX_RINGSIZE) + nb_desc = adapter->tx_ring_size; + txq->port_id = dev->data->port_id; txq->next_to_clean = 0; txq->next_to_use = 0; @@ -1297,6 +1312,9 @@ static int ena_rx_queue_setup(struct rte_eth_dev *dev, return ENA_COM_FAULT; } + if (nb_desc == RTE_ETH_DEV_FALLBACK_RX_RINGSIZE) + nb_desc = adapter->rx_ring_size; + if (!rte_is_power_of_2(nb_desc)) { RTE_LOG(ERR, PMD, "Unsupported size of RX queue: %d is not a power of 2.", @@ -1574,13 +1592,26 @@ static void ena_timer_wd_callback(__rte_unused struct rte_timer *timer, } } -static int ena_calc_io_queue_num(__rte_unused struct ena_com_dev *ena_dev, +static int ena_calc_io_queue_num(struct ena_com_dev *ena_dev, struct ena_com_dev_get_features_ctx *get_feat_ctx) { - int io_sq_num, io_cq_num, io_queue_num; + uint32_t io_sq_num, io_cq_num, io_queue_num; + + /* Regular queues capabilities */ + if (ena_dev->supported_features & BIT(ENA_ADMIN_MAX_QUEUES_EXT)) { + struct ena_admin_queue_ext_feature_fields *max_queue_ext = + &get_feat_ctx->max_queue_ext.max_queue_ext; + io_sq_num = max_queue_ext->max_rx_sq_num; + io_sq_num = RTE_MIN(io_sq_num, max_queue_ext->max_tx_sq_num); - io_sq_num = get_feat_ctx->max_queues.max_sq_num; - io_cq_num = get_feat_ctx->max_queues.max_cq_num; + io_cq_num = max_queue_ext->max_rx_cq_num; + io_cq_num = RTE_MIN(io_cq_num, max_queue_ext->max_tx_cq_num); + } else { + struct ena_admin_queue_feature_desc *max_queues = + &get_feat_ctx->max_queues; + io_sq_num = max_queues->max_sq_num; + io_cq_num = max_queues->max_cq_num; + } io_queue_num = RTE_MIN(io_sq_num, io_cq_num); @@ -1594,14 +1625,14 @@ static int ena_calc_io_queue_num(__rte_unused struct ena_com_dev *ena_dev, static int eth_ena_dev_init(struct rte_eth_dev *eth_dev) { + struct ena_calc_queue_size_ctx calc_queue_ctx = { 0 }; struct rte_pci_device *pci_dev; struct rte_intr_handle *intr_handle; struct ena_adapter *adapter = (struct ena_adapter *)(eth_dev->data->dev_private); struct ena_com_dev *ena_dev = &adapter->ena_dev; struct ena_com_dev_get_features_ctx get_feat_ctx; - int queue_size, rc; - u16 tx_sgl_size = 0; + int rc; static int adapters_found; bool wd_state; @@ -1656,19 +1687,24 @@ static int eth_ena_dev_init(struct rte_eth_dev *eth_dev) adapter->wd_state = wd_state; ena_dev->tx_mem_queue_type = ENA_ADMIN_PLACEMENT_POLICY_HOST; + + calc_queue_ctx.ena_dev = ena_dev; + calc_queue_ctx.get_feat_ctx = &get_feat_ctx; + adapter->num_queues = ena_calc_io_queue_num(ena_dev, &get_feat_ctx); - queue_size = ena_calc_queue_size(ena_dev, &tx_sgl_size, &get_feat_ctx); - if (queue_size <= 0 || adapter->num_queues <= 0) { + rc = ena_calc_queue_size(&calc_queue_ctx); + if (unlikely((rc != 0) || (adapter->num_queues <= 0))) { rc = -EFAULT; goto err_device_destroy; } - adapter->tx_ring_size = queue_size; - adapter->rx_ring_size = queue_size; + adapter->tx_ring_size = calc_queue_ctx.tx_queue_size; + adapter->rx_ring_size = calc_queue_ctx.rx_queue_size; - adapter->max_tx_sgl_size = tx_sgl_size; + adapter->max_tx_sgl_size = calc_queue_ctx.max_tx_sgl_size; + adapter->max_rx_sgl_size = calc_queue_ctx.max_rx_sgl_size; /* prepare ring structures */ ena_init_rings(adapter); @@ -1785,6 +1821,7 @@ static void ena_init_rings(struct ena_adapter *adapter) ring->type = ENA_RING_TYPE_RX; ring->adapter = adapter; ring->id = i; + ring->sgl_size = adapter->max_rx_sgl_size; } } @@ -1857,15 +1894,19 @@ static void ena_infos_get(struct rte_eth_dev *dev, adapter->tx_supported_offloads = tx_feat; adapter->rx_supported_offloads = rx_feat; - dev_info->rx_desc_lim.nb_max = ENA_MAX_RING_DESC; + dev_info->rx_desc_lim.nb_max = adapter->rx_ring_size; dev_info->rx_desc_lim.nb_min = ENA_MIN_RING_DESC; + dev_info->rx_desc_lim.nb_seg_max = RTE_MIN(ENA_PKT_MAX_BUFS, + adapter->max_rx_sgl_size); + dev_info->rx_desc_lim.nb_mtu_seg_max = RTE_MIN(ENA_PKT_MAX_BUFS, + adapter->max_rx_sgl_size); - dev_info->tx_desc_lim.nb_max = ENA_MAX_RING_DESC; + dev_info->tx_desc_lim.nb_max = adapter->tx_ring_size; dev_info->tx_desc_lim.nb_min = ENA_MIN_RING_DESC; dev_info->tx_desc_lim.nb_seg_max = RTE_MIN(ENA_PKT_MAX_BUFS, - feat.max_queues.max_packet_tx_descs); + adapter->max_tx_sgl_size); dev_info->tx_desc_lim.nb_mtu_seg_max = RTE_MIN(ENA_PKT_MAX_BUFS, - feat.max_queues.max_packet_tx_descs); + adapter->max_tx_sgl_size); } static uint16_t eth_ena_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, @@ -1901,7 +1942,7 @@ static uint16_t eth_ena_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, for (completed = 0; completed < nb_pkts; completed++) { int segments = 0; - ena_rx_ctx.max_bufs = rx_ring->ring_size; + ena_rx_ctx.max_bufs = rx_ring->sgl_size; ena_rx_ctx.ena_bufs = rx_ring->ena_bufs; ena_rx_ctx.descs = 0; /* receive packet context */ diff --git a/drivers/net/ena/ena_ethdev.h b/drivers/net/ena/ena_ethdev.h index 322e90ace..e6f7bd012 100644 --- a/drivers/net/ena/ena_ethdev.h +++ b/drivers/net/ena/ena_ethdev.h @@ -45,7 +45,6 @@ #define ENA_MEM_BAR 2 #define ENA_MAX_NUM_QUEUES 128 -#define ENA_DEFAULT_RING_SIZE (1024) #define ENA_MIN_FRAME_LEN 64 #define ENA_NAME_MAX_LEN 20 #define ENA_PKT_MAX_BUFS 17 @@ -71,6 +70,15 @@ struct ena_tx_buffer { struct ena_com_buf bufs[ENA_PKT_MAX_BUFS]; }; +struct ena_calc_queue_size_ctx { + struct ena_com_dev_get_features_ctx *get_feat_ctx; + struct ena_com_dev *ena_dev; + u16 rx_queue_size; + u16 tx_queue_size; + u16 max_tx_sgl_size; + u16 max_rx_sgl_size; +}; + struct ena_ring { u16 next_to_use; u16 next_to_clean; @@ -176,6 +184,7 @@ struct ena_adapter { /* RX */ struct ena_ring rx_ring[ENA_MAX_NUM_QUEUES] __rte_cache_aligned; int rx_ring_size; + u16 max_rx_sgl_size; u16 num_queues; u16 max_mtu; From patchwork Fri Dec 14 13:18:31 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michal Krawczyk X-Patchwork-Id: 48863 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 4C6D71BB71; Fri, 14 Dec 2018 14:19:07 +0100 (CET) Received: from mail-lj1-f194.google.com (mail-lj1-f194.google.com [209.85.208.194]) by dpdk.org (Postfix) with ESMTP id 5DEC81BB43 for ; Fri, 14 Dec 2018 14:19:03 +0100 (CET) Received: by mail-lj1-f194.google.com with SMTP id s5-v6so4846080ljd.12 for ; Fri, 14 Dec 2018 05:19:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=semihalf-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=EuyI/XdksgNbjTKERvJ3HMF0JYYHdZo/dr8/e87qIJE=; b=pWqjUWvDwb5yzVHftfijsdMcN9GTMvFbKV4hT96+esYChZ1H6B27Pb6Ay0uuMjV4Rw oFSvkWgiS0BxLXMs6HkhNNCwgO81F17hjqE8xXSzMVFwVfHy/r0ljqEqH3EkP0JzY7Lo mJ5Q/pVl1w71iSPGM0hQysPWW2zLRTq1ohgJb/Qg3sjtlHgWrC5vlpfeZiil2YZg5kc+ D4CFdP3nRpYOoyJN7McXjG7nND8PLZCaMYKEtEPLN6ysz9h2KYkWshjqxECLFZ7QljY2 nS7JcGGbM4tI/SufpIpErVswvi0OkIW9JPGgG2McbypvtVUVB9zXLBiAp6J99p75oWFi lJmA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=EuyI/XdksgNbjTKERvJ3HMF0JYYHdZo/dr8/e87qIJE=; b=b/N6GpzYS9HzL5tMzo7L+aiRZe7+zVlUK3ECakRBKSZSKqZGsMHOaAh43V5ePqggLb kN/aZcVv9MQ/RVHWu1VwZm6D1pPVlRmzH6uPCFGWRWh+KAQUcxbJoJ+/jgrG+70OGd8w wP47wQ8L6A7uVD0Fv1hJvgpWtQkeRTDZpRnRvdByhURw0Mm4GWZLoBuKpqgB5JcH3QEr IC24q+LSPZ9pBEWWHOtMDmUO0JxJgSrDxHfccJ6it6C7A98OSoW9TOawHULWODZGgOQm yKEMfqeetvL7T8aJdwAwO8UyFylAP/EglpWj6sEtCTLuqv82cZ5VFbKVaIhxt4TD+raO fE1g== X-Gm-Message-State: AA+aEWZy7xw+AlpTmLY5aNHSAjejFqCjdA/f6QbQwwM6JV7gLZYQe12T ixGP3gqW+we35o/KnzMfd1D1gfzaDec= X-Google-Smtp-Source: AFSGD/XM7AvtxjjHnVIhitbQ3qUIj0jshxwlQYEdQ+07nwbmIJoAPspyHMEKCjIZuRn3xWCiVQetnA== X-Received: by 2002:a2e:c41:: with SMTP id o1-v6mr1871151ljd.152.1544793542606; Fri, 14 Dec 2018 05:19:02 -0800 (PST) Received: from mkPC.semihalf.local (31-172-191-173.noc.fibertech.net.pl. [31.172.191.173]) by smtp.gmail.com with ESMTPSA id o25sm873884lfd.29.2018.12.14.05.19.00 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 14 Dec 2018 05:19:01 -0800 (PST) From: Michal Krawczyk To: dev@dpdk.org Cc: gtzalik@dpdk.org, mw@dpdk.org, matua@amazon.com, rk@semihalf.com, stable@dpdk.org Date: Fri, 14 Dec 2018 14:18:31 +0100 Message-Id: <20181214131846.22439-6-mk@semihalf.com> X-Mailer: git-send-email 2.14.1 In-Reply-To: <20181214131846.22439-1-mk@semihalf.com> References: <20181214131846.22439-1-mk@semihalf.com> Subject: [dpdk-dev] [PATCH 05/20] net/ena: skip packet with wrong req_id X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Rafal Kozik When invalid req_id is received, the reset should be handled by the application, as it is indicating invalid rings state, so further Rx is not making any sense. Fixes: c2034976673d ("net/ena: add Rx out of order completion") Cc: stable@dpdk.org Signed-off-by: Rafal Kozik Acked-by: Michal Krawczyk --- drivers/net/ena/ena_ethdev.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/drivers/net/ena/ena_ethdev.c b/drivers/net/ena/ena_ethdev.c index 617f85f02..505d9bf30 100644 --- a/drivers/net/ena/ena_ethdev.c +++ b/drivers/net/ena/ena_ethdev.c @@ -1988,6 +1988,8 @@ static uint16_t eth_ena_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, segments++; next_to_clean++; } + if (unlikely(rc)) + break; /* fill mbuf attributes if any */ ena_rx_mbuf_prepare(mbuf_head, &ena_rx_ctx); From patchwork Fri Dec 14 13:18:32 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michal Krawczyk X-Patchwork-Id: 48864 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id F02FD1BB7D; Fri, 14 Dec 2018 14:19:08 +0100 (CET) Received: from mail-lj1-f196.google.com (mail-lj1-f196.google.com [209.85.208.196]) by dpdk.org (Postfix) with ESMTP id AB0C61BB3E for ; Fri, 14 Dec 2018 14:19:04 +0100 (CET) Received: by mail-lj1-f196.google.com with SMTP id v15-v6so4846490ljh.13 for ; Fri, 14 Dec 2018 05:19:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=semihalf-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=YISctMDiggCy11WwPwF3pNKU9vcXmo0uCF3bFp9Ur6A=; b=X6Vp73KHvz5F8Dj+HNOxqhfj5nhv+5rUhXg0riyfekHurS0ncGbsZwW1dnv9R94XCH SjIkn3wi9z1DiUbCbApi4tYlYjXUp4Iz/Sx0vSJbTc1hTgmM/PGL6D1y5ER0gjmBdcNj KELIt3ldMCbmbV3tPHfreRm4umOxIvYxUmWlR5PRzxxy41gqgSku9w5ysGP1485wX1mc 6X/3TXuztndUpAV6g+16GwsXHrkYPueqBkoyV0hJjgdFE/NYeqv+CcNDOZdpGiXOoujI ESSZU5CoSUr8GJaWzIF1lLf+e1+4DfYwLt3rZM48Ep+wW8nq8w2Q3CxSCPn4uLzdlt9K PLIA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=YISctMDiggCy11WwPwF3pNKU9vcXmo0uCF3bFp9Ur6A=; b=AOi0JEuLdBE5S19qStxm7G/fobyC6xKJ1D/syLBj5J0enRGu/ytfPNl9YGHhEZm8eU K9164eaE3/M3E1JjLGPsbU2IIuViJajg+f0DEVzvApM/BUDqMvc3rz0ssX5DvQsE9oPy bVafuGM9dONWbZND82Hn8DPd5rjsEHikI7Ziv9hK787tr/zQZTpLAzKkMGA69Cz7gE1o WzHutHrZTEVySU5qpzeX4BhnaxnK3H9pXGPgvfvFNMktaDu78ZhJB9wXhf20zQND/i8V nm5cKRjQVUDAtim+xRS412fSJZTjw6BZfZv4AmDoPDScH4pASJaPdVyzJdzlpVZ1hYEn /0Ww== X-Gm-Message-State: AA+aEWa16TX27EZIcVkQPXxuRlnslC/buFSqPPvy54DG4Gw5GKf9ZkBd oBMmMCJYbBhPbtyxuDC0jGzyw+hyyqo= X-Google-Smtp-Source: AFSGD/XglXtMz1sHBlMZjY2cI+ZE1+uuQVJlcRSeWTWRK0IZMNGFLFAASfOcm3iPWdpErxQO7TZ3yg== X-Received: by 2002:a2e:880a:: with SMTP id x10-v6mr2053624ljh.174.1544793543753; Fri, 14 Dec 2018 05:19:03 -0800 (PST) Received: from mkPC.semihalf.local (31-172-191-173.noc.fibertech.net.pl. [31.172.191.173]) by smtp.gmail.com with ESMTPSA id o25sm873884lfd.29.2018.12.14.05.19.02 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 14 Dec 2018 05:19:03 -0800 (PST) From: Michal Krawczyk To: dev@dpdk.org Cc: gtzalik@dpdk.org, mw@dpdk.org, matua@amazon.com, rk@semihalf.com, Michal Krawczyk Date: Fri, 14 Dec 2018 14:18:32 +0100 Message-Id: <20181214131846.22439-7-mk@semihalf.com> X-Mailer: git-send-email 2.14.1 In-Reply-To: <20181214131846.22439-1-mk@semihalf.com> References: <20181214131846.22439-1-mk@semihalf.com> Subject: [dpdk-dev] [PATCH 06/20] net/ena: add LLQv2 support X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" LLQ (Low Latency Queue) is the feature that allows pushing header directly to the device through PCI before even DMA is triggered. It reduces latency, bacause device can start preparing packet before payload is sent through DMA. Signed-off-by: Michal Krawczyk --- drivers/net/ena/ena_ethdev.c | 220 ++++++++++++++++++++++++++++++++++++------- drivers/net/ena/ena_ethdev.h | 3 + 2 files changed, 190 insertions(+), 33 deletions(-) diff --git a/drivers/net/ena/ena_ethdev.c b/drivers/net/ena/ena_ethdev.c index 505d9bf30..73070ad96 100644 --- a/drivers/net/ena/ena_ethdev.c +++ b/drivers/net/ena/ena_ethdev.c @@ -116,6 +116,9 @@ struct ena_stats { #define ENA_STAT_GLOBAL_ENTRY(stat) \ ENA_STAT_ENTRY(stat, dev) +#define ENA_MAX_RING_SIZE_RX 1024 +#define ENA_MAX_RING_SIZE_TX 1024 + /* * Each rte_memzone should have unique name. * To satisfy it, count number of allocation and add it to name. @@ -806,6 +809,9 @@ static void ena_tx_queue_release(void *queue) ena_tx_queue_release_bufs(ring); /* Free ring resources */ + if (ring->push_buf_intermediate_buf) + rte_free(ring->push_buf_intermediate_buf); + if (ring->tx_buffer_info) rte_free(ring->tx_buffer_info); @@ -814,6 +820,7 @@ static void ena_tx_queue_release(void *queue) ring->empty_tx_reqs = NULL; ring->tx_buffer_info = NULL; + ring->push_buf_intermediate_buf = NULL; ring->configured = 0; @@ -937,15 +944,30 @@ static int ena_check_valid_conf(struct ena_adapter *adapter) static int ena_calc_queue_size(struct ena_calc_queue_size_ctx *ctx) { - uint32_t tx_queue_size, rx_queue_size; + struct ena_admin_feature_llq_desc *llq = &ctx->get_feat_ctx->llq; + struct ena_com_dev *ena_dev = ctx->ena_dev; + uint32_t tx_queue_size = ENA_MAX_RING_SIZE_TX; + uint32_t rx_queue_size = ENA_MAX_RING_SIZE_RX; - if (ctx->ena_dev->supported_features & BIT(ENA_ADMIN_MAX_QUEUES_EXT)) { + if (ena_dev->supported_features & BIT(ENA_ADMIN_MAX_QUEUES_EXT)) { struct ena_admin_queue_ext_feature_fields *max_queue_ext = &ctx->get_feat_ctx->max_queue_ext.max_queue_ext; - rx_queue_size = RTE_MIN(max_queue_ext->max_rx_cq_depth, + rx_queue_size = RTE_MIN(rx_queue_size, + max_queue_ext->max_rx_cq_depth); + rx_queue_size = RTE_MIN(rx_queue_size, max_queue_ext->max_rx_sq_depth); - tx_queue_size = RTE_MIN(max_queue_ext->max_tx_cq_depth, - max_queue_ext->max_tx_sq_depth); + tx_queue_size = RTE_MIN(tx_queue_size, + max_queue_ext->max_tx_cq_depth); + + if (ena_dev->tx_mem_queue_type == + ENA_ADMIN_PLACEMENT_POLICY_DEV) { + tx_queue_size = RTE_MIN(tx_queue_size, + llq->max_llq_depth); + } else { + tx_queue_size = RTE_MIN(tx_queue_size, + max_queue_ext->max_tx_sq_depth); + } + ctx->max_rx_sgl_size = RTE_MIN(ENA_PKT_MAX_BUFS, max_queue_ext->max_per_packet_rx_descs); ctx->max_tx_sgl_size = RTE_MIN(ENA_PKT_MAX_BUFS, @@ -953,9 +975,22 @@ ena_calc_queue_size(struct ena_calc_queue_size_ctx *ctx) } else { struct ena_admin_queue_feature_desc *max_queues = &ctx->get_feat_ctx->max_queues; - rx_queue_size = RTE_MIN(max_queues->max_cq_depth, + rx_queue_size = RTE_MIN(rx_queue_size, + max_queues->max_cq_depth); + rx_queue_size = RTE_MIN(rx_queue_size, max_queues->max_sq_depth); - tx_queue_size = rx_queue_size; + tx_queue_size = RTE_MIN(tx_queue_size, + max_queues->max_cq_depth); + + if (ena_dev->tx_mem_queue_type == + ENA_ADMIN_PLACEMENT_POLICY_DEV) { + tx_queue_size = RTE_MIN(tx_queue_size, + llq->max_llq_depth); + } else { + tx_queue_size = RTE_MIN(tx_queue_size, + max_queues->max_sq_depth); + } + ctx->max_rx_sgl_size = RTE_MIN(ENA_PKT_MAX_BUFS, max_queues->max_packet_tx_descs); ctx->max_tx_sgl_size = RTE_MIN(ENA_PKT_MAX_BUFS, @@ -1277,6 +1312,17 @@ static int ena_tx_queue_setup(struct rte_eth_dev *dev, return -ENOMEM; } + txq->push_buf_intermediate_buf = + rte_zmalloc("txq->push_buf_intermediate_buf", + txq->tx_max_header_size, + RTE_CACHE_LINE_SIZE); + if (!txq->push_buf_intermediate_buf) { + RTE_LOG(ERR, PMD, "failed to alloc push buff for LLQ\n"); + rte_free(txq->tx_buffer_info); + rte_free(txq->empty_tx_reqs); + return -ENOMEM; + } + for (i = 0; i < txq->ring_size; i++) txq->empty_tx_reqs[i] = i; @@ -1592,28 +1638,87 @@ static void ena_timer_wd_callback(__rte_unused struct rte_timer *timer, } } +static inline void +set_default_llq_configurations(struct ena_llq_configurations *llq_config) +{ + llq_config->llq_header_location = ENA_ADMIN_INLINE_HEADER; + llq_config->llq_ring_entry_size = ENA_ADMIN_LIST_ENTRY_SIZE_128B; + llq_config->llq_stride_ctrl = ENA_ADMIN_MULTIPLE_DESCS_PER_ENTRY; + llq_config->llq_num_decs_before_header = + ENA_ADMIN_LLQ_NUM_DESCS_BEFORE_HEADER_2; + llq_config->llq_ring_entry_size_value = 128; +} + +static int +ena_set_queues_placement_policy(struct ena_adapter *adapter, + struct ena_com_dev *ena_dev, + struct ena_admin_feature_llq_desc *llq, + struct ena_llq_configurations *llq_default_configurations) +{ + int rc; + u32 llq_feature_mask; + + llq_feature_mask = 1 << ENA_ADMIN_LLQ; + if (!(ena_dev->supported_features & llq_feature_mask)) { + RTE_LOG(INFO, PMD, + "LLQ is not supported. Fallback to host mode policy.\n"); + ena_dev->tx_mem_queue_type = ENA_ADMIN_PLACEMENT_POLICY_HOST; + return 0; + } + + rc = ena_com_config_dev_mode(ena_dev, llq, llq_default_configurations); + if (unlikely(rc)) { + PMD_INIT_LOG(WARNING, "Failed to config dev mode. " + "Fallback to host mode policy.\n"); + ena_dev->tx_mem_queue_type = ENA_ADMIN_PLACEMENT_POLICY_HOST; + return 0; + } + + /* Nothing to config, exit */ + if (ena_dev->tx_mem_queue_type == ENA_ADMIN_PLACEMENT_POLICY_HOST) + return 0; + + if (!adapter->dev_mem_base) { + RTE_LOG(ERR, PMD, "Unable to access LLQ bar resource. " + "Fallback to host mode policy.\n."); + ena_dev->tx_mem_queue_type = ENA_ADMIN_PLACEMENT_POLICY_HOST; + return 0; + } + + ena_dev->mem_bar = adapter->dev_mem_base; + + return 0; +} + static int ena_calc_io_queue_num(struct ena_com_dev *ena_dev, struct ena_com_dev_get_features_ctx *get_feat_ctx) { - uint32_t io_sq_num, io_cq_num, io_queue_num; + uint32_t io_tx_sq_num, io_tx_cq_num, io_rx_num, io_queue_num; /* Regular queues capabilities */ if (ena_dev->supported_features & BIT(ENA_ADMIN_MAX_QUEUES_EXT)) { struct ena_admin_queue_ext_feature_fields *max_queue_ext = &get_feat_ctx->max_queue_ext.max_queue_ext; - io_sq_num = max_queue_ext->max_rx_sq_num; - io_sq_num = RTE_MIN(io_sq_num, max_queue_ext->max_tx_sq_num); - - io_cq_num = max_queue_ext->max_rx_cq_num; - io_cq_num = RTE_MIN(io_cq_num, max_queue_ext->max_tx_cq_num); + io_rx_num = RTE_MIN(max_queue_ext->max_rx_sq_num, + max_queue_ext->max_rx_cq_num); + io_tx_sq_num = max_queue_ext->max_tx_sq_num; + io_tx_cq_num = max_queue_ext->max_tx_cq_num; } else { struct ena_admin_queue_feature_desc *max_queues = &get_feat_ctx->max_queues; - io_sq_num = max_queues->max_sq_num; - io_cq_num = max_queues->max_cq_num; + io_tx_sq_num = max_queues->max_sq_num; + io_tx_cq_num = max_queues->max_cq_num; + io_rx_num = RTE_MIN(io_tx_sq_num, io_tx_cq_num); } - io_queue_num = RTE_MIN(io_sq_num, io_cq_num); + /* In case of LLQ use the llq number in the get feature cmd */ + if (ena_dev->tx_mem_queue_type == ENA_ADMIN_PLACEMENT_POLICY_DEV) + io_tx_sq_num = get_feat_ctx->llq.max_llq_num; + + io_queue_num = RTE_MIN(rte_lcore_count(), ENA_MAX_NUM_IO_QUEUES); + io_queue_num = RTE_MIN(io_queue_num, io_rx_num); + io_queue_num = RTE_MIN(io_queue_num, io_tx_sq_num); + io_queue_num = RTE_MIN(io_queue_num, io_tx_cq_num); if (unlikely(io_queue_num == 0)) { RTE_LOG(ERR, PMD, "Number of IO queues should not be 0\n"); @@ -1632,6 +1737,8 @@ static int eth_ena_dev_init(struct rte_eth_dev *eth_dev) (struct ena_adapter *)(eth_dev->data->dev_private); struct ena_com_dev *ena_dev = &adapter->ena_dev; struct ena_com_dev_get_features_ctx get_feat_ctx; + struct ena_llq_configurations llq_config; + const char *queue_type_str; int rc; static int adapters_found; @@ -1686,11 +1793,22 @@ static int eth_ena_dev_init(struct rte_eth_dev *eth_dev) } adapter->wd_state = wd_state; - ena_dev->tx_mem_queue_type = ENA_ADMIN_PLACEMENT_POLICY_HOST; + set_default_llq_configurations(&llq_config); + rc = ena_set_queues_placement_policy(adapter, ena_dev, + &get_feat_ctx.llq, &llq_config); + if (unlikely(rc)) { + PMD_INIT_LOG(CRIT, "Failed to set placement policy"); + return rc; + } + + if (ena_dev->tx_mem_queue_type == ENA_ADMIN_PLACEMENT_POLICY_HOST) + queue_type_str = "Regular"; + else + queue_type_str = "Low latency"; + RTE_LOG(INFO, PMD, "Placement policy: %s\n", queue_type_str); calc_queue_ctx.ena_dev = ena_dev; calc_queue_ctx.get_feat_ctx = &get_feat_ctx; - adapter->num_queues = ena_calc_io_queue_num(ena_dev, &get_feat_ctx); @@ -2106,13 +2224,21 @@ static void ena_update_hints(struct ena_adapter *adapter, static int ena_check_and_linearize_mbuf(struct ena_ring *tx_ring, struct rte_mbuf *mbuf) { - int num_segments, rc; + struct ena_com_dev *ena_dev; + int num_segments, header_len, rc; + ena_dev = &tx_ring->adapter->ena_dev; num_segments = mbuf->nb_segs; + header_len = mbuf->data_len; if (likely(num_segments < tx_ring->sgl_size)) return 0; + if (ena_dev->tx_mem_queue_type == ENA_ADMIN_PLACEMENT_POLICY_DEV && + (num_segments == tx_ring->sgl_size) && + (header_len < tx_ring->tx_max_header_size)) + return 0; + rc = rte_pktmbuf_linearize(mbuf); if (unlikely(rc)) RTE_LOG(WARNING, PMD, "Mbuf linearize failed\n"); @@ -2127,6 +2253,7 @@ static uint16_t eth_ena_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t next_to_use = tx_ring->next_to_use; uint16_t next_to_clean = tx_ring->next_to_clean; struct rte_mbuf *mbuf; + uint16_t seg_len; unsigned int ring_size = tx_ring->ring_size; unsigned int ring_mask = ring_size - 1; struct ena_com_tx_ctx ena_tx_ctx; @@ -2134,6 +2261,8 @@ static uint16_t eth_ena_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, struct ena_com_buf *ebuf; uint16_t rc, req_id, total_tx_descs = 0; uint16_t sent_idx = 0, empty_tx_reqs; + uint16_t push_len = 0; + uint16_t delta = 0; int nb_hw_desc; /* Check adapter state */ @@ -2166,17 +2295,32 @@ static uint16_t eth_ena_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, sizeof(struct ena_com_tx_meta)); ena_tx_ctx.ena_bufs = ebuf; ena_tx_ctx.req_id = req_id; + + delta = 0; + seg_len = mbuf->data_len; + if (tx_ring->tx_mem_queue_type == ENA_ADMIN_PLACEMENT_POLICY_DEV) { - /* prepare the push buffer with - * virtual address of the data - */ - ena_tx_ctx.header_len = - RTE_MIN(mbuf->data_len, - tx_ring->tx_max_header_size); - ena_tx_ctx.push_header = - (void *)((char *)mbuf->buf_addr + - mbuf->data_off); + push_len = RTE_MIN(mbuf->pkt_len, + tx_ring->tx_max_header_size); + ena_tx_ctx.header_len = push_len; + + if (likely(push_len <= seg_len)) { + /* If the push header is in the single segment, + * then just point it to the 1st mbuf data. + */ + ena_tx_ctx.push_header = + rte_pktmbuf_mtod(mbuf, uint8_t *); + } else { + /* If the push header lays in the several + * segments, copy it to the intermediate buffer. + */ + rte_pktmbuf_read(mbuf, 0, push_len, + tx_ring->push_buf_intermediate_buf); + ena_tx_ctx.push_header = + tx_ring->push_buf_intermediate_buf; + delta = push_len - seg_len; + } } /* there's no else as we take advantage of memset zeroing */ /* Set TX offloads flags, if applicable */ @@ -2191,20 +2335,30 @@ static uint16_t eth_ena_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, /* Process first segment taking into * consideration pushed header */ - if (mbuf->data_len > ena_tx_ctx.header_len) { + if (seg_len > push_len) { ebuf->paddr = mbuf->buf_iova + mbuf->data_off + - ena_tx_ctx.header_len; - ebuf->len = mbuf->data_len - ena_tx_ctx.header_len; + push_len; + ebuf->len = seg_len - push_len; ebuf++; tx_info->num_of_bufs++; } while ((mbuf = mbuf->next) != NULL) { - ebuf->paddr = mbuf->buf_iova + mbuf->data_off; - ebuf->len = mbuf->data_len; + seg_len = mbuf->data_len; + + /* Skip mbufs if whole data is pushed as a header */ + if (unlikely(delta > seg_len)) { + delta -= seg_len; + continue; + } + + ebuf->paddr = mbuf->buf_iova + mbuf->data_off + delta; + ebuf->len = seg_len - delta; ebuf++; tx_info->num_of_bufs++; + + delta = 0; } ena_tx_ctx.num_bufs = tx_info->num_of_bufs; diff --git a/drivers/net/ena/ena_ethdev.h b/drivers/net/ena/ena_ethdev.h index e6f7bd012..713cdea97 100644 --- a/drivers/net/ena/ena_ethdev.h +++ b/drivers/net/ena/ena_ethdev.h @@ -110,6 +110,9 @@ struct ena_ring { /* Max length PMD can push to device for LLQ */ uint8_t tx_max_header_size; int configured; + + uint8_t *push_buf_intermediate_buf; + struct ena_adapter *adapter; uint64_t offloads; u16 sgl_size; From patchwork Fri Dec 14 13:18:33 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michal Krawczyk X-Patchwork-Id: 48865 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 215311BB9D; Fri, 14 Dec 2018 14:19:12 +0100 (CET) Received: from mail-lj1-f196.google.com (mail-lj1-f196.google.com [209.85.208.196]) by dpdk.org (Postfix) with ESMTP id 114D91BB32 for ; Fri, 14 Dec 2018 14:19:06 +0100 (CET) Received: by mail-lj1-f196.google.com with SMTP id n18-v6so4868328lji.7 for ; Fri, 14 Dec 2018 05:19:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=semihalf-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=VUTlNm1kiF437GEok2rFW+4o6gDmJYDyt99AglV+TFI=; b=bJ9DgsFa6QWC9wUMhDoeJCMJpWUmPnreY5Zk5uW4sVkp7btYeejgGaPFGMbkk6Q72E 2QpojjWWNLuWRBktJqEAS0yfAUpOQyYnUsEU72HndjMlM/rTbJ5V5WidIyHi5PZrCnUa 7hVjNHlXdrF495whzcE51iBajYuXxyGUkjv6DmYcItipdulLgAq4BH2PyAP5up1uKPOU psjUeXUfMTZ46RrBubmeiN8WedcmWcTszFipkHCDCXDZK1gbD15gQAyeBX48SPGvLUc6 7oLDrundXUUhXfP9xpS20xEHhQvGhMyV27QqBl56pxLp6Cdb3MZAltwYzZVC+tiJERlZ qiyA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=VUTlNm1kiF437GEok2rFW+4o6gDmJYDyt99AglV+TFI=; b=Pt37ze2HEAPiDiZmScfbKPwLca43Vi7ZDNRJwUJptG0wBwmcthMiWezyPpnon58Ily VVArMzV38rYxg7AUvdudk6jcWM9YnoVc52cpQ+X8juoIg7OymXFAYO5h2qViTTGrj1ZE 9sh1Vjt0PK2JMX3KgXOmvjJsc6rIKJ1opnfKUqs7+JeoAJ/GnpCPrNhK1s1f0U4FQOuX Rw3m2LKAmHskIGX6wLx/adZDHc/odLKzRwqok9XNh4EFxW+frzZOvfaypJDznt676KgJ xgoE+6GPR7xY+Yxgrd2hDzcPfmx9pXNHsiGzZLM/0qNHxZJlaeByj684uL2o8qhYP1T/ D4IA== X-Gm-Message-State: AA+aEWbohj790jqnSAzIMVGVu+1H9ySNLYvfyOsQuq0FoJ3KxffFTZYV 2wRsRm5eri1aPjyqez6V2JdixmILA00= X-Google-Smtp-Source: AFSGD/XkiwrlLk8G59z0Tsx5JQADS8G1n9cRV/56zc6NWpPJfp5HMOag3fDgBwU0wXqDW9YBP5WT6A== X-Received: by 2002:a2e:5654:: with SMTP id k81-v6mr1807955ljb.48.1544793545182; Fri, 14 Dec 2018 05:19:05 -0800 (PST) Received: from mkPC.semihalf.local (31-172-191-173.noc.fibertech.net.pl. [31.172.191.173]) by smtp.gmail.com with ESMTPSA id o25sm873884lfd.29.2018.12.14.05.19.03 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 14 Dec 2018 05:19:04 -0800 (PST) From: Michal Krawczyk To: dev@dpdk.org Cc: gtzalik@dpdk.org, mw@dpdk.org, matua@amazon.com, rk@semihalf.com Date: Fri, 14 Dec 2018 14:18:33 +0100 Message-Id: <20181214131846.22439-8-mk@semihalf.com> X-Mailer: git-send-email 2.14.1 In-Reply-To: <20181214131846.22439-1-mk@semihalf.com> References: <20181214131846.22439-1-mk@semihalf.com> Subject: [dpdk-dev] [PATCH 07/20] net/ena: increase maximum Rx ring size X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Rafal Kozik Some of ENA devices supports 8k Rx rings. Maximum supported size is received upon device initialization. As ENA_DEFAULT_RING_SIZE_RX macro is upper limit, it needs to be adjusted. Signed-off-by: Rafal Kozik Acked-by: Michal Krawczyk --- drivers/net/ena/ena_ethdev.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/net/ena/ena_ethdev.c b/drivers/net/ena/ena_ethdev.c index 73070ad96..f83552377 100644 --- a/drivers/net/ena/ena_ethdev.c +++ b/drivers/net/ena/ena_ethdev.c @@ -116,7 +116,7 @@ struct ena_stats { #define ENA_STAT_GLOBAL_ENTRY(stat) \ ENA_STAT_ENTRY(stat, dev) -#define ENA_MAX_RING_SIZE_RX 1024 +#define ENA_MAX_RING_SIZE_RX 8192 #define ENA_MAX_RING_SIZE_TX 1024 /* From patchwork Fri Dec 14 13:18:34 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michal Krawczyk X-Patchwork-Id: 48866 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 00E571BBD4; Fri, 14 Dec 2018 14:19:16 +0100 (CET) Received: from mail-lf1-f66.google.com (mail-lf1-f66.google.com [209.85.167.66]) by dpdk.org (Postfix) with ESMTP id C6AC01BB76 for ; Fri, 14 Dec 2018 14:19:07 +0100 (CET) Received: by mail-lf1-f66.google.com with SMTP id e26so4240751lfc.2 for ; Fri, 14 Dec 2018 05:19:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=semihalf-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=py47PY4bUtyRmvnQIB4Zl7xUgCU41vwWA7oOHJsWKA0=; b=0kso+1e0cvubi0LSlBeuhjcxYSj7YF0/JA/UJ3eVHZF10DJV3uDlJl5WC3MwOmVnlg hKPBs9JIZLbthLoTZRWPy4kWklfJntw+3+RsQow19pGvSYSonZHjx0BT0GX6wkTC/eIc bMxa0Z9cQZL9DZQq82JZEQ+JGNLVVAGJJ3DqqZTZFxwjPBdTLKQWeJizKkR9wHkpOoM6 FJdHh9jQhVE3Q3LzwNQmcVMGtdvD4UY/HOIJqEdoL4KyQwdcyXsPNmedP25CkgT73DR3 V8VUzP7rGCTU2Nzumxoociv+kE6MGHUs2cIRYIqMG2PhhMoJe8DQ59obLyUBRgWmGi36 JmZw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=py47PY4bUtyRmvnQIB4Zl7xUgCU41vwWA7oOHJsWKA0=; b=YgzdlORn4WhbFemUAhGRA/DMT/LBrnRUQeSotJzGcUY3HwNAZKthk34RFUrwBN7cfp Y7sLVINDAbTnEaw6ZijPCfkFI9xZ8f38ib2hAicvaQkr1bYPQ67PMfQkZj9LKZ5RKg+m 2dHc555jZSMEHY5n3wuLgMYYGGAI/mQQok32xdYmfQR1SV/2vH8t+8rD3FGuANBQ9L12 zKKfopxdLtJsD046SmheEFlQhJi/ukX4m+J+3tNghj4RnBrm4VKEnkl+rJy7v5lt5s3Z Iv4fWXJ5aT8fJCT3DGkFWNNSgWYpayXy54fpw5hbC4D5Qvz03NHf5C3xK037+4fzp6CK 0YLA== X-Gm-Message-State: AA+aEWaxGhQpLPxIqbmcnDQ1nlZnppKugaQpAZJMIlGbLi2VsZqFMmon sBFuwRRcCgM32SXqSiDus3fywLsFX6s= X-Google-Smtp-Source: AFSGD/XBz6cf1o+QW3EuY96RHbnxjOQ9+Yj/qTXBSzFxeJEUButdQOQnLUE0n6k+a/MNFDh76+z3gg== X-Received: by 2002:a19:920a:: with SMTP id u10mr1760980lfd.122.1544793547037; Fri, 14 Dec 2018 05:19:07 -0800 (PST) Received: from mkPC.semihalf.local (31-172-191-173.noc.fibertech.net.pl. [31.172.191.173]) by smtp.gmail.com with ESMTPSA id o25sm873884lfd.29.2018.12.14.05.19.05 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 14 Dec 2018 05:19:06 -0800 (PST) From: Michal Krawczyk To: dev@dpdk.org Cc: gtzalik@dpdk.org, mw@dpdk.org, matua@amazon.com, rk@semihalf.com Date: Fri, 14 Dec 2018 14:18:34 +0100 Message-Id: <20181214131846.22439-9-mk@semihalf.com> X-Mailer: git-send-email 2.14.1 In-Reply-To: <20181214131846.22439-1-mk@semihalf.com> References: <20181214131846.22439-1-mk@semihalf.com> Subject: [dpdk-dev] [PATCH 08/20] net/ena: call additional doorbells if needed X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Rafal Kozik Before sending next packet, check if calling doorbell is needed. Signed-off-by: Rafal Kozik Acked-by: Michal Krawczyk --- drivers/net/ena/ena_ethdev.c | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/drivers/net/ena/ena_ethdev.c b/drivers/net/ena/ena_ethdev.c index f83552377..5f7dec086 100644 --- a/drivers/net/ena/ena_ethdev.c +++ b/drivers/net/ena/ena_ethdev.c @@ -2363,7 +2363,16 @@ static uint16_t eth_ena_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, ena_tx_ctx.num_bufs = tx_info->num_of_bufs; - /* Write data to device */ + if (ena_com_is_doorbell_needed(tx_ring->ena_com_io_sq, + &ena_tx_ctx)) { + RTE_LOG(DEBUG, PMD, "llq tx max burst size of queue %d" + " achieved, writing doorbell to send burst\n", + tx_ring->id); + rte_wmb(); + ena_com_write_sq_doorbell(tx_ring->ena_com_io_sq); + } + + /* prepare the packet's descriptors to dma engine */ rc = ena_com_prepare_tx(tx_ring->ena_com_io_sq, &ena_tx_ctx, &nb_hw_desc); if (unlikely(rc)) From patchwork Fri Dec 14 13:18:35 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michal Krawczyk X-Patchwork-Id: 48867 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 77D661BBDE; Fri, 14 Dec 2018 14:19:19 +0100 (CET) Received: from mail-lf1-f66.google.com (mail-lf1-f66.google.com [209.85.167.66]) by dpdk.org (Postfix) with ESMTP id 8A03C1BB86 for ; Fri, 14 Dec 2018 14:19:09 +0100 (CET) Received: by mail-lf1-f66.google.com with SMTP id v5so4229183lfe.7 for ; Fri, 14 Dec 2018 05:19:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=semihalf-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=IKuFkPtSSH7BOtevw+4YqX+a4SaDUIPBBm9lraPmsJg=; b=Mrmj7/sWRdw7NZXVSO5PrzuV6UdYiO/aAovF+WsE+m6F95yepW3qXRSoet98e+VxL6 hqf4ACaeIyhJqB0DL8ikwEycQX7Mr1ZVP7iaH2Sdqv+EGXkVeFr8uVtJRIWD84p8KSeq Qw84GGfeB57h5u5wzwxubbq+7YKidZkGNwtQfsz/g4y2jAXSBbWa5Le+7QxwS24GZJyc fsiSAflRo0mNXXhvWILhYH1yUrToNxNYoK4WgSR43mHH8ggx0HwfH5yeoxl2hIVHde9j Hh6FtF2XLoEpFbDgoCdjEdkZ27KRn5gWEOEVFy3a9mOH3y+omc+j8IBPoxxqrwNsMcBI 41yg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=IKuFkPtSSH7BOtevw+4YqX+a4SaDUIPBBm9lraPmsJg=; b=caE3IjkS6PYCX0ELAFC7qsIgND2UAGoiZeI4fO2c/fhGK2TBRU43nnsYdrbzt5DfY3 xTO3+9cN4j1Ql24e4gwxjM6PeRXVpBg1mLp1P4bXYwMn5SU2pOiZxdDnebxGHsXZynNN TnoIoKcpK9Hc/Nwk2JUP2OckWNePrOGISU0oD0+MObav7gYNvSv8JQvASUTrraHg0dJR Fpv0pgu8diwSLXgSvE/CzjByQiuZ58iU6gH73aaQxlpKYfvmbaD3skINQ3A3+wPlgirL WHVt+Ry/2diWgW1QWrJvNy6cK8dIB5SDMCZ+9dyCne2paArP+yEDuE2azhiPnrYAmvHG jxmQ== X-Gm-Message-State: AA+aEWayYGxXVQFVfO4/fdhY1ArAf5xC3/zTJbOkRFMk66ibikCJ/v2d aNsfKr71e5E8vTt8Azm/dtUPLAuQ1k0= X-Google-Smtp-Source: AFSGD/XTmwBqSdavtTTyWoeOMKZ4lg3LnS7Op5jNagKM/UFb5MUgRtkQN/sGBpYnL/5ZAa1mXndRuA== X-Received: by 2002:a19:1901:: with SMTP id 1mr1595062lfz.99.1544793548399; Fri, 14 Dec 2018 05:19:08 -0800 (PST) Received: from mkPC.semihalf.local (31-172-191-173.noc.fibertech.net.pl. [31.172.191.173]) by smtp.gmail.com with ESMTPSA id o25sm873884lfd.29.2018.12.14.05.19.07 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 14 Dec 2018 05:19:07 -0800 (PST) From: Michal Krawczyk To: dev@dpdk.org Cc: gtzalik@dpdk.org, mw@dpdk.org, matua@amazon.com, rk@semihalf.com, stable@dpdk.org Date: Fri, 14 Dec 2018 14:18:35 +0100 Message-Id: <20181214131846.22439-10-mk@semihalf.com> X-Mailer: git-send-email 2.14.1 In-Reply-To: <20181214131846.22439-1-mk@semihalf.com> References: <20181214131846.22439-1-mk@semihalf.com> Subject: [dpdk-dev] [PATCH 09/20] net/ena: destroy queues if start failed X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Rafal Kozik If start function fails, previously created queues have to be removed. ena_queue_restart_all() and ena_queue_restart() are renamed to ena_queue_start_all() and ena_queue_start(). ena_free_io_queues_all() is renamed to ena_queue_stop_all(). Fixes: df238f84c0a2 ("net/ena: recreate HW IO rings on start and stop") Cc: stable@dpdk.org Signed-off-by: Rafal Kozik Acked-by: Michal Krawczyk --- drivers/net/ena/ena_ethdev.c | 95 ++++++++++++++++++++++++++++---------------- 1 file changed, 60 insertions(+), 35 deletions(-) diff --git a/drivers/net/ena/ena_ethdev.c b/drivers/net/ena/ena_ethdev.c index 5f7dec086..deb97151d 100644 --- a/drivers/net/ena/ena_ethdev.c +++ b/drivers/net/ena/ena_ethdev.c @@ -244,10 +244,12 @@ static void ena_tx_queue_release_bufs(struct ena_ring *ring); static int ena_link_update(struct rte_eth_dev *dev, int wait_to_complete); static int ena_create_io_queue(struct ena_ring *ring); -static void ena_free_io_queues_all(struct ena_adapter *adapter); -static int ena_queue_restart(struct ena_ring *ring); -static int ena_queue_restart_all(struct rte_eth_dev *dev, - enum ena_ring_type ring_type); +static void ena_queue_stop(struct ena_ring *ring); +static void ena_queue_stop_all(struct rte_eth_dev *dev, + enum ena_ring_type ring_type); +static int ena_queue_start(struct ena_ring *ring); +static int ena_queue_start_all(struct rte_eth_dev *dev, + enum ena_ring_type ring_type); static void ena_stats_restart(struct rte_eth_dev *dev); static void ena_infos_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info); @@ -805,9 +807,6 @@ static void ena_tx_queue_release(void *queue) ena_assert_msg(ring->adapter->state != ENA_ADAPTER_STATE_RUNNING, "API violation"); - /* Free all bufs */ - ena_tx_queue_release_bufs(ring); - /* Free ring resources */ if (ring->push_buf_intermediate_buf) rte_free(ring->push_buf_intermediate_buf); @@ -872,8 +871,8 @@ static int ena_link_update(struct rte_eth_dev *dev, return 0; } -static int ena_queue_restart_all(struct rte_eth_dev *dev, - enum ena_ring_type ring_type) +static int ena_queue_start_all(struct rte_eth_dev *dev, + enum ena_ring_type ring_type) { struct ena_adapter *adapter = (struct ena_adapter *)(dev->data->dev_private); @@ -901,18 +900,25 @@ static int ena_queue_restart_all(struct rte_eth_dev *dev, "Inconsistent state of tx queues\n"); } - rc = ena_queue_restart(&queues[i]); + rc = ena_queue_start(&queues[i]); if (rc) { PMD_INIT_LOG(ERR, - "failed to restart queue %d type(%d)", + "failed to start queue %d type(%d)", i, ring_type); - return rc; + goto err; } } } return 0; + +err: + while (i--) + if (queues[i].configured) + ena_queue_stop(&queues[i]); + + return rc; } static uint32_t ena_get_mtu_conf(struct ena_adapter *adapter) @@ -1101,19 +1107,19 @@ static int ena_start(struct rte_eth_dev *dev) if (rc) return rc; - rc = ena_queue_restart_all(dev, ENA_RING_TYPE_RX); + rc = ena_queue_start_all(dev, ENA_RING_TYPE_RX); if (rc) return rc; - rc = ena_queue_restart_all(dev, ENA_RING_TYPE_TX); + rc = ena_queue_start_all(dev, ENA_RING_TYPE_TX); if (rc) - return rc; + goto err_start_tx; if (adapter->rte_dev->data->dev_conf.rxmode.mq_mode & ETH_MQ_RX_RSS_FLAG && adapter->rte_dev->data->nb_rx_queues > 0) { rc = ena_rss_init_default(adapter); if (rc) - return rc; + goto err_rss_init; } ena_stats_restart(dev); @@ -1128,6 +1134,12 @@ static int ena_start(struct rte_eth_dev *dev) adapter->state = ENA_ADAPTER_STATE_RUNNING; return 0; + +err_rss_init: + ena_queue_stop_all(dev, ENA_RING_TYPE_TX); +err_start_tx: + ena_queue_stop_all(dev, ENA_RING_TYPE_RX); + return rc; } static void ena_stop(struct rte_eth_dev *dev) @@ -1136,7 +1148,8 @@ static void ena_stop(struct rte_eth_dev *dev) (struct ena_adapter *)(dev->data->dev_private); rte_timer_stop_sync(&adapter->timer_wd); - ena_free_io_queues_all(adapter); + ena_queue_stop_all(dev, ENA_RING_TYPE_TX); + ena_queue_stop_all(dev, ENA_RING_TYPE_RX); adapter->state = ENA_ADAPTER_STATE_STOPPED; } @@ -1199,36 +1212,46 @@ static int ena_create_io_queue(struct ena_ring *ring) return 0; } -static void ena_free_io_queues_all(struct ena_adapter *adapter) +static void ena_queue_stop(struct ena_ring *ring) { - struct rte_eth_dev *eth_dev = adapter->rte_dev; - struct ena_com_dev *ena_dev = &adapter->ena_dev; - int i; - uint16_t ena_qid; - uint16_t nb_rxq = eth_dev->data->nb_rx_queues; - uint16_t nb_txq = eth_dev->data->nb_tx_queues; - - for (i = 0; i < nb_txq; ++i) { - ena_qid = ENA_IO_TXQ_IDX(i); - ena_com_destroy_io_queue(ena_dev, ena_qid); + struct ena_com_dev *ena_dev = &ring->adapter->ena_dev; - ena_tx_queue_release_bufs(&adapter->tx_ring[i]); + if (ring->type == ENA_RING_TYPE_RX) { + ena_com_destroy_io_queue(ena_dev, ENA_IO_RXQ_IDX(ring->id)); + ena_rx_queue_release_bufs(ring); + } else { + ena_com_destroy_io_queue(ena_dev, ENA_IO_TXQ_IDX(ring->id)); + ena_tx_queue_release_bufs(ring); } +} - for (i = 0; i < nb_rxq; ++i) { - ena_qid = ENA_IO_RXQ_IDX(i); - ena_com_destroy_io_queue(ena_dev, ena_qid); +static void ena_queue_stop_all(struct rte_eth_dev *dev, + enum ena_ring_type ring_type) +{ + struct ena_adapter *adapter = + (struct ena_adapter *)(dev->data->dev_private); + struct ena_ring *queues = NULL; + uint16_t nb_queues, i; - ena_rx_queue_release_bufs(&adapter->rx_ring[i]); + if (ring_type == ENA_RING_TYPE_RX) { + queues = adapter->rx_ring; + nb_queues = dev->data->nb_rx_queues; + } else { + queues = adapter->tx_ring; + nb_queues = dev->data->nb_tx_queues; } + + for (i = 0; i < nb_queues; ++i) + if (queues[i].configured) + ena_queue_stop(&queues[i]); } -static int ena_queue_restart(struct ena_ring *ring) +static int ena_queue_start(struct ena_ring *ring) { int rc, bufs_num; ena_assert_msg(ring->configured == 1, - "Trying to restart unconfigured queue\n"); + "Trying to start unconfigured queue\n"); rc = ena_create_io_queue(ring); if (rc) { @@ -1245,6 +1268,8 @@ static int ena_queue_restart(struct ena_ring *ring) bufs_num = ring->ring_size - 1; rc = ena_populate_rx_queue(ring, bufs_num); if (rc != bufs_num) { + ena_com_destroy_io_queue(&ring->adapter->ena_dev, + ENA_IO_RXQ_IDX(ring->id)); PMD_INIT_LOG(ERR, "Failed to populate rx ring !"); return ENA_COM_FAULT; } From patchwork Fri Dec 14 13:18:36 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michal Krawczyk X-Patchwork-Id: 48868 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 1C4B21BBEA; Fri, 14 Dec 2018 14:19:22 +0100 (CET) Received: from mail-lj1-f196.google.com (mail-lj1-f196.google.com [209.85.208.196]) by dpdk.org (Postfix) with ESMTP id 8EB511BB91 for ; Fri, 14 Dec 2018 14:19:10 +0100 (CET) Received: by mail-lj1-f196.google.com with SMTP id 83-v6so4853422ljf.10 for ; Fri, 14 Dec 2018 05:19:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=semihalf-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=AnpAtvdvPTKaDBtDTJ2qmFSz2k3ARGPx/Y/PGrheWck=; b=jfIZLtikHIEXo0Lgs3jFOxP8gCKkojfCoAkrFZQg9kGzApE0yGmyf7clRHwjCvwgNn amBhrR5S5tf4XIrhzc6+4EKiqw2+tF8T4NCTsWQILQPN5ih0aHZjbbR2/jEtmz6i/4wA iQRxrzTweOMB4Y5HL5i+jsX/9huAlJdje6gODoLO9pa1nbe7MX7Rv4ydNqIeQaXu6DFc eXRvunYXLnkmc5G2dwZMonlJBTAQ1xDDXwT2OYOVlmGqyeHKsnVEHmBEhk2+YVn5OjSo Cj1Yxl2bIIMKdsiZUasL7ZSh2HCVeozR/4qADKQuSaZ7Ne1h9qQdaV0LX9mniTdQGzjq 8tDA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=AnpAtvdvPTKaDBtDTJ2qmFSz2k3ARGPx/Y/PGrheWck=; b=pKCdG/lTJ2gFpMRV1z/spGdV2zdinCPaBxIcmOZsPzUvtl0VIPE0iqu9H8NpeNT8b8 PeODO5xb3tf0xqGi3LjJwQtfF/wq1erngD7L0AVVe5muhZCb6IaE5B412BQASL/4Mqn1 RvbNBwQPvtd2JDmCuBEbFBR/7x3ldC0mFZMWsyU/3obgy5WLN6rti5yxnubW4+XYjDGD lQN+v16hdrXee8mGJlQnYdWhOeAUNrBuCVsb1khNO0XJ/WqCWEn6LSV3Xr/oMMQojEyr nntYJncEKrfMuB1TtSPhnRwGhaGuKzRVMiTGv16Fq0bJNMPCGCtijtTOkUkYFuEVmmk7 tCTA== X-Gm-Message-State: AA+aEWajvKPVtlKidVCAzIaaRbwp0HtaQHPIgHl7QrBWHFUYMlo0zmw3 n4PJ3xWR0aiKzVIuFgg+uZZvghq7AiM= X-Google-Smtp-Source: AFSGD/UimBG+PNS4a760COBbhv1Ii3Pv0dCIe7cp/kEdQyFKR5kXjZL7tB6JN2kpSLwgxV8gbyobLA== X-Received: by 2002:a2e:98c9:: with SMTP id s9-v6mr1569493ljj.166.1544793549805; Fri, 14 Dec 2018 05:19:09 -0800 (PST) Received: from mkPC.semihalf.local (31-172-191-173.noc.fibertech.net.pl. [31.172.191.173]) by smtp.gmail.com with ESMTPSA id o25sm873884lfd.29.2018.12.14.05.19.08 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 14 Dec 2018 05:19:09 -0800 (PST) From: Michal Krawczyk To: dev@dpdk.org Cc: gtzalik@dpdk.org, mw@dpdk.org, matua@amazon.com, rk@semihalf.com, stable@dpdk.org Date: Fri, 14 Dec 2018 14:18:36 +0100 Message-Id: <20181214131846.22439-11-mk@semihalf.com> X-Mailer: git-send-email 2.14.1 In-Reply-To: <20181214131846.22439-1-mk@semihalf.com> References: <20181214131846.22439-1-mk@semihalf.com> Subject: [dpdk-dev] [PATCH 10/20] net/ena: reset should not reconfigure queues X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Rafal Kozik Reset function should return the port to initial state, in which no Tx and Rx queues are setup. Then application should reconfigure the queues. According to DPDK documentation the rte_eth_dev_reset() itself is a generic function which only does some hardware reset operations through calling dev_unint() and dev_init(). ena_com_dev_reset which perform NIC registers reset should be called during stop. Fixes: 2081d5e2e92d ("net/ena: add reset routine") Cc: stable@dpdk.org Signed-off-by: Rafal Kozik Acked-by: Michal Krawczyk --- drivers/net/ena/ena_ethdev.c | 108 +++++++++++++++---------------------------- 1 file changed, 38 insertions(+), 70 deletions(-) diff --git a/drivers/net/ena/ena_ethdev.c b/drivers/net/ena/ena_ethdev.c index deb97151d..3715619a7 100644 --- a/drivers/net/ena/ena_ethdev.c +++ b/drivers/net/ena/ena_ethdev.c @@ -262,6 +262,8 @@ static int ena_rss_reta_query(struct rte_eth_dev *dev, static int ena_get_sset_count(struct rte_eth_dev *dev, int sset); static void ena_interrupt_handler_rte(void *cb_arg); static void ena_timer_wd_callback(struct rte_timer *timer, void *arg); +static void ena_destroy_device(struct rte_eth_dev *eth_dev); +static int eth_ena_dev_init(struct rte_eth_dev *eth_dev); static const struct eth_dev_ops ena_dev_ops = { .dev_configure = ena_dev_configure, @@ -546,65 +548,14 @@ static void ena_close(struct rte_eth_dev *dev) static int ena_dev_reset(struct rte_eth_dev *dev) { - struct rte_mempool *mb_pool_rx[ENA_MAX_NUM_QUEUES]; - struct rte_eth_dev *eth_dev; - struct rte_pci_device *pci_dev; - struct rte_intr_handle *intr_handle; - struct ena_com_dev *ena_dev; - struct ena_com_dev_get_features_ctx get_feat_ctx; - struct ena_adapter *adapter; - int nb_queues; - int rc, i; - bool wd_state; - - adapter = (struct ena_adapter *)(dev->data->dev_private); - ena_dev = &adapter->ena_dev; - eth_dev = adapter->rte_dev; - pci_dev = RTE_ETH_DEV_TO_PCI(eth_dev); - intr_handle = &pci_dev->intr_handle; - nb_queues = eth_dev->data->nb_rx_queues; - - ena_com_set_admin_running_state(ena_dev, false); + int rc = 0; - rc = ena_com_dev_reset(ena_dev, adapter->reset_reason); + ena_destroy_device(dev); + rc = eth_ena_dev_init(dev); if (rc) - RTE_LOG(ERR, PMD, "Device reset failed\n"); - - for (i = 0; i < nb_queues; i++) - mb_pool_rx[i] = adapter->rx_ring[i].mb_pool; - - ena_rx_queue_release_all(eth_dev); - ena_tx_queue_release_all(eth_dev); - - rte_intr_disable(intr_handle); - - ena_com_abort_admin_commands(ena_dev); - ena_com_wait_for_abort_completion(ena_dev); - ena_com_admin_destroy(ena_dev); - ena_com_mmio_reg_read_request_destroy(ena_dev); - - rc = ena_device_init(ena_dev, &get_feat_ctx, &wd_state); - if (rc) { PMD_INIT_LOG(CRIT, "Cannot initialize device\n"); - return rc; - } - adapter->wd_state = wd_state; - - rte_intr_enable(intr_handle); - ena_com_set_admin_polling_mode(ena_dev, false); - ena_com_admin_aenq_enable(ena_dev); - - for (i = 0; i < nb_queues; ++i) - ena_rx_queue_setup(eth_dev, i, adapter->rx_ring[i].ring_size, 0, - NULL, mb_pool_rx[i]); - for (i = 0; i < nb_queues; ++i) - ena_tx_queue_setup(eth_dev, i, adapter->tx_ring[i].ring_size, 0, - NULL); - - adapter->trigger_reset = false; - - return 0; + return rc; } static int ena_rss_reta_update(struct rte_eth_dev *dev, @@ -774,11 +725,6 @@ static void ena_rx_queue_release(void *queue) { struct ena_ring *ring = (struct ena_ring *)queue; - ena_assert_msg(ring->configured, - "API violation - releasing not configured queue"); - ena_assert_msg(ring->adapter->state != ENA_ADAPTER_STATE_RUNNING, - "API violation"); - /* Free ring resources */ if (ring->rx_buffer_info) rte_free(ring->rx_buffer_info); @@ -802,11 +748,6 @@ static void ena_tx_queue_release(void *queue) { struct ena_ring *ring = (struct ena_ring *)queue; - ena_assert_msg(ring->configured, - "API violation. Releasing not configured queue"); - ena_assert_msg(ring->adapter->state != ENA_ADAPTER_STATE_RUNNING, - "API violation"); - /* Free ring resources */ if (ring->push_buf_intermediate_buf) rte_free(ring->push_buf_intermediate_buf); @@ -1146,11 +1087,19 @@ static void ena_stop(struct rte_eth_dev *dev) { struct ena_adapter *adapter = (struct ena_adapter *)(dev->data->dev_private); + struct ena_com_dev *ena_dev = &adapter->ena_dev; + int rc; rte_timer_stop_sync(&adapter->timer_wd); ena_queue_stop_all(dev, ENA_RING_TYPE_TX); ena_queue_stop_all(dev, ENA_RING_TYPE_RX); + if (adapter->trigger_reset) { + rc = ena_com_dev_reset(ena_dev, adapter->reset_reason); + if (rc) + RTE_LOG(ERR, PMD, "Device reset failed rc=%d\n", rc); + } + adapter->state = ENA_ADAPTER_STATE_STOPPED; } @@ -1908,24 +1857,43 @@ static int eth_ena_dev_init(struct rte_eth_dev *eth_dev) return rc; } -static int eth_ena_dev_uninit(struct rte_eth_dev *eth_dev) +static void ena_destroy_device(struct rte_eth_dev *eth_dev) { struct ena_adapter *adapter = (struct ena_adapter *)(eth_dev->data->dev_private); + struct ena_com_dev *ena_dev = &adapter->ena_dev; - if (rte_eal_process_type() != RTE_PROC_PRIMARY) - return 0; + if (adapter->state == ENA_ADAPTER_STATE_FREE) + return; + + ena_com_set_admin_running_state(ena_dev, false); if (adapter->state != ENA_ADAPTER_STATE_CLOSED) ena_close(eth_dev); + ena_com_delete_debug_area(ena_dev); + ena_com_delete_host_info(ena_dev); + + ena_com_abort_admin_commands(ena_dev); + ena_com_wait_for_abort_completion(ena_dev); + ena_com_admin_destroy(ena_dev); + ena_com_mmio_reg_read_request_destroy(ena_dev); + + adapter->state = ENA_ADAPTER_STATE_FREE; +} + +static int eth_ena_dev_uninit(struct rte_eth_dev *eth_dev) +{ + if (rte_eal_process_type() != RTE_PROC_PRIMARY) + return 0; + + ena_destroy_device(eth_dev); + eth_dev->dev_ops = NULL; eth_dev->rx_pkt_burst = NULL; eth_dev->tx_pkt_burst = NULL; eth_dev->tx_pkt_prepare = NULL; - adapter->state = ENA_ADAPTER_STATE_FREE; - return 0; } From patchwork Fri Dec 14 13:18:37 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michal Krawczyk X-Patchwork-Id: 48869 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 8CDD01BBED; Fri, 14 Dec 2018 14:19:25 +0100 (CET) Received: from mail-lj1-f195.google.com (mail-lj1-f195.google.com [209.85.208.195]) by dpdk.org (Postfix) with ESMTP id BF56C1BB3B for ; Fri, 14 Dec 2018 14:19:11 +0100 (CET) Received: by mail-lj1-f195.google.com with SMTP id v1-v6so4901916ljd.0 for ; Fri, 14 Dec 2018 05:19:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=semihalf-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=yKFLIxxNSAOzYQiuNB9YNQkgCvJCTjgDgBeqiem55UE=; b=vHbUo7D0fIXQdDzb0jWQ5dXGbqkuOuy3Koy4Yxes3UAaSYI28DQJapT/XKEzqPAMA/ oSyAsNmhMTmLisHSd7MsxsIGav3itUOHxJZllyqF9rzZgq2CQOzWgyB3AThcPX1zCdW0 Tw9BvGvw2QJgz17NrjFaYI+ext51FejfqIcqAyFF5qJHsMt+ascM9A1Qf55dmIiIupwu NI70LySuC59h206T8d7tLphpecROrDOi6/OcsKduLTVRRPY8V3lIdaESt3N+lixeevQG jN1DcmDul3ZDCnuaNN5Gqucwa67ZWP6GN42Ee6dbWds1RbwsE/SUTMlgpQbxh+XS5eQk YItw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=yKFLIxxNSAOzYQiuNB9YNQkgCvJCTjgDgBeqiem55UE=; b=dpMnTdmzQOD0+oCmf1sGxBnPGDm9WiZJIEiix5R21Q77HWhhdvfiOGVDDN4KjHG6Li tp0VgD+1ddYiwK4E/A6it9Btt4oncveSGXkEaFXiFhmPOigkbM3XYqX0ddUIBRt0cV0r 0OdKdVLpLjjOLPWK79ZqEmrH0r2EnB3EZUTCbfPDLH67OBpTKnF3gNPC6weOCWgank3z qdC1MuXAbznA+/9MnXAjRvw01WRbS/fTr5VBAugIs60TDbffiWxl+CMKz2ZCDOxVZ3PA pQRFh5Z3GW67RpCUbtn3fjV6lbr39oZIweXRWtfYazxlSzWz1qCSIq3+xtHqaSNORfhW faPQ== X-Gm-Message-State: AA+aEWZlp8E/oSUJqjU8nAf2I1iyTGvO8VDkv5eFuQ+KotWAxuQKLI6Q 4myJMDuZ0gVpZ9zqP3er5geG3W2feo0= X-Google-Smtp-Source: AFSGD/XW3O9wqIQ0uMnIVfNCEq72Y0boca81k5vNEPnrQ2VNs5zaCT+XSq3luUcu0Rf6tXZ/OCmWjw== X-Received: by 2002:a2e:9819:: with SMTP id a25-v6mr2042837ljj.6.1544793550851; Fri, 14 Dec 2018 05:19:10 -0800 (PST) Received: from mkPC.semihalf.local (31-172-191-173.noc.fibertech.net.pl. [31.172.191.173]) by smtp.gmail.com with ESMTPSA id o25sm873884lfd.29.2018.12.14.05.19.09 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 14 Dec 2018 05:19:10 -0800 (PST) From: Michal Krawczyk To: dev@dpdk.org Cc: gtzalik@dpdk.org, mw@dpdk.org, matua@amazon.com, rk@semihalf.com Date: Fri, 14 Dec 2018 14:18:37 +0100 Message-Id: <20181214131846.22439-12-mk@semihalf.com> X-Mailer: git-send-email 2.14.1 In-Reply-To: <20181214131846.22439-1-mk@semihalf.com> References: <20181214131846.22439-1-mk@semihalf.com> Subject: [dpdk-dev] [PATCH 11/20] net/ena: adjust new line in log messages X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Rafal Kozik Only PMD_*_LOG is adding new line character to the log message. All printouts were adjusted for consistency. Signed-off-by: Rafal Kozik Acked-by: Michal Krawczyk --- drivers/net/ena/ena_ethdev.c | 28 ++++++++++++++-------------- 1 file changed, 14 insertions(+), 14 deletions(-) diff --git a/drivers/net/ena/ena_ethdev.c b/drivers/net/ena/ena_ethdev.c index 3715619a7..5adc87355 100644 --- a/drivers/net/ena/ena_ethdev.c +++ b/drivers/net/ena/ena_ethdev.c @@ -553,7 +553,7 @@ ena_dev_reset(struct rte_eth_dev *dev) ena_destroy_device(dev); rc = eth_ena_dev_init(dev); if (rc) - PMD_INIT_LOG(CRIT, "Cannot initialize device\n"); + PMD_INIT_LOG(CRIT, "Cannot initialize device"); return rc; } @@ -880,7 +880,7 @@ static int ena_check_valid_conf(struct ena_adapter *adapter) if (max_frame_len > adapter->max_mtu || max_frame_len < ENA_MIN_MTU) { PMD_INIT_LOG(ERR, "Unsupported MTU of %d. " - "max mtu: %d, min mtu: %d\n", + "max mtu: %d, min mtu: %d", max_frame_len, adapter->max_mtu, ENA_MIN_MTU); return ENA_COM_UNSUPPORTED; } @@ -984,7 +984,7 @@ static int ena_stats_get(struct rte_eth_dev *dev, memset(&ena_stats, 0, sizeof(ena_stats)); rc = ena_com_get_dev_basic_stats(ena_dev, &ena_stats); if (unlikely(rc)) { - RTE_LOG(ERR, PMD, "Could not retrieve statistics from ENA"); + RTE_LOG(ERR, PMD, "Could not retrieve statistics from ENA\n"); return rc; } @@ -1013,12 +1013,12 @@ static int ena_mtu_set(struct rte_eth_dev *dev, uint16_t mtu) struct ena_com_dev *ena_dev; int rc = 0; - ena_assert_msg(dev->data != NULL, "Uninitialized device"); - ena_assert_msg(dev->data->dev_private != NULL, "Uninitialized device"); + ena_assert_msg(dev->data != NULL, "Uninitialized device\n"); + ena_assert_msg(dev->data->dev_private != NULL, "Uninitialized device\n"); adapter = (struct ena_adapter *)(dev->data->dev_private); ena_dev = &adapter->ena_dev; - ena_assert_msg(ena_dev != NULL, "Uninitialized device"); + ena_assert_msg(ena_dev != NULL, "Uninitialized device\n"); if (mtu > ena_get_mtu_conf(adapter) || mtu < ENA_MIN_MTU) { RTE_LOG(ERR, PMD, @@ -1204,7 +1204,7 @@ static int ena_queue_start(struct ena_ring *ring) rc = ena_create_io_queue(ring); if (rc) { - PMD_INIT_LOG(ERR, "Failed to create IO queue!\n"); + PMD_INIT_LOG(ERR, "Failed to create IO queue!"); return rc; } @@ -1248,7 +1248,7 @@ static int ena_tx_queue_setup(struct rte_eth_dev *dev, if (!rte_is_power_of_2(nb_desc)) { RTE_LOG(ERR, PMD, - "Unsupported size of TX queue: %d is not a power of 2.", + "Unsupported size of TX queue: %d is not a power of 2.\n", nb_desc); return -EINVAL; } @@ -1337,7 +1337,7 @@ static int ena_rx_queue_setup(struct rte_eth_dev *dev, if (!rte_is_power_of_2(nb_desc)) { RTE_LOG(ERR, PMD, - "Unsupported size of RX queue: %d is not a power of 2.", + "Unsupported size of RX queue: %d is not a power of 2.\n", nb_desc); return -EINVAL; } @@ -1410,7 +1410,7 @@ static int ena_populate_rx_queue(struct ena_ring *rxq, unsigned int count) return 0; in_use = rxq->next_to_use - rxq->next_to_clean; - ena_assert_msg(((in_use + count) < ring_size), "bad ring state"); + ena_assert_msg(((in_use + count) < ring_size), "bad ring state\n"); /* get resources for incoming packets */ rc = rte_mempool_get_bulk(rxq->mb_pool, (void **)mbufs, count); @@ -1643,7 +1643,7 @@ ena_set_queues_placement_policy(struct ena_adapter *adapter, rc = ena_com_config_dev_mode(ena_dev, llq, llq_default_configurations); if (unlikely(rc)) { PMD_INIT_LOG(WARNING, "Failed to config dev mode. " - "Fallback to host mode policy.\n"); + "Fallback to host mode policy."); ena_dev->tx_mem_queue_type = ENA_ADMIN_PLACEMENT_POLICY_HOST; return 0; } @@ -1945,12 +1945,12 @@ static void ena_infos_get(struct rte_eth_dev *dev, uint64_t rx_feat = 0, tx_feat = 0; int rc = 0; - ena_assert_msg(dev->data != NULL, "Uninitialized device"); - ena_assert_msg(dev->data->dev_private != NULL, "Uninitialized device"); + ena_assert_msg(dev->data != NULL, "Uninitialized device\n"); + ena_assert_msg(dev->data->dev_private != NULL, "Uninitialized device\n"); adapter = (struct ena_adapter *)(dev->data->dev_private); ena_dev = &adapter->ena_dev; - ena_assert_msg(ena_dev != NULL, "Uninitialized device"); + ena_assert_msg(ena_dev != NULL, "Uninitialized device\n"); dev_info->speed_capa = ETH_LINK_SPEED_1G | From patchwork Fri Dec 14 13:18:38 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michal Krawczyk X-Patchwork-Id: 48870 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 0906B1BBF3; Fri, 14 Dec 2018 14:19:29 +0100 (CET) Received: from mail-lj1-f194.google.com (mail-lj1-f194.google.com [209.85.208.194]) by dpdk.org (Postfix) with ESMTP id E89411BBD0 for ; Fri, 14 Dec 2018 14:19:12 +0100 (CET) Received: by mail-lj1-f194.google.com with SMTP id c19-v6so4881717lja.5 for ; Fri, 14 Dec 2018 05:19:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=semihalf-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=1rA1K7tUuyTgLke0GFRguKuFn3LWXL03S06K5tnaMcg=; b=UuLeDRE1QTAa0B6QdQnIEyWZcHGTFgsJqB2Vg6BdqJlpWqtQOZaoHkoJ94WZMSLk71 JOKPcbJeTComKTi3E7qVOyMyzZkI9XsXmOuR6zgqUZzPHDAN+LRRrpfPIybYcD/gdXAD nL8x/YL0VQgpX9s1RorXY3sbhnAlkWU7dZk0n1y2dw8KM3m6u1KJDvkyz5eOUAghtpQK P/Dm/t4VWRyjYo42pkEBI8OC2CSpd/Nss4b2t+ACXu6+NOZJHJyVn6nTSlVkb9A757Xi NSx0ClZvJLlxbF7AgXQ6HUTL/g6QOINMkSgWdfAJGNBURFBz6HKZl4vHvTJK0K1n3acU kBFQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=1rA1K7tUuyTgLke0GFRguKuFn3LWXL03S06K5tnaMcg=; b=QqwR7oG6N94jSoPWz3JKsCe6X5usmqSHHylZOdIP7r25Hg1SiB6jDHH7sUMN+cuYRU cnJl6HmR6XXBZGMHh6yoWiHKjGbrOGRptG4iXJ1uRJHdQqOnIZ8CI3nLTA3ezD1uyYF+ vhWZyAn/BzzpfOAvPgLczgy45O0AWV2MlGbNsX+dkLmc07/pcVm49m/Q/1ttKA7GXs+k JUU/0b46iX9YIrjm21xiWQULAZFbGfWMab/DtLSFz+EbsgP6HJI1Ppru0Hy/zsRAMvzv 9N1WAA1uuC5+YwWxfP+ZpBa/c2K3d6c+yVJTcRQ84CVCIirx7svOjRPSUrkn7yDKciyg wHkg== X-Gm-Message-State: AA+aEWZ4Tm+cRP5UM2sQDieKuCDsfJbwNYWW+CiRkaXI8c80T+GiFh44 mHBqzRfMERNHxDLbz/HDfIbxaVJrx3U= X-Google-Smtp-Source: AFSGD/W5p4CTSM8OiBbDO/ryetRewv9Nb1pV9ewsDqbK+x/ufNdp1Gt0yBcej07Uour8S54y+lAulA== X-Received: by 2002:a2e:6a13:: with SMTP id f19-v6mr2007524ljc.41.1544793552196; Fri, 14 Dec 2018 05:19:12 -0800 (PST) Received: from mkPC.semihalf.local (31-172-191-173.noc.fibertech.net.pl. [31.172.191.173]) by smtp.gmail.com with ESMTPSA id o25sm873884lfd.29.2018.12.14.05.19.10 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 14 Dec 2018 05:19:11 -0800 (PST) From: Michal Krawczyk To: dev@dpdk.org Cc: gtzalik@dpdk.org, mw@dpdk.org, matua@amazon.com, rk@semihalf.com, stable@dpdk.org Date: Fri, 14 Dec 2018 14:18:38 +0100 Message-Id: <20181214131846.22439-13-mk@semihalf.com> X-Mailer: git-send-email 2.14.1 In-Reply-To: <20181214131846.22439-1-mk@semihalf.com> References: <20181214131846.22439-1-mk@semihalf.com> Subject: [dpdk-dev] [PATCH 12/20] net/ena: add supported RSS offloads types X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Rafal Kozik The PMD was not passing RSS offloads values although it was supporting the RSS. To allow application to probe the PMD for RSS support, the missing information was added. Fixes: 1173fca25af9 ("ena: add polling-mode driver") Cc: stable@dpdk.org Signed-off-by: Rafal Kozik Acked-by: Michal Krawczyk --- drivers/net/ena/ena_ethdev.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/drivers/net/ena/ena_ethdev.c b/drivers/net/ena/ena_ethdev.c index 5adc87355..3aa32a3bf 100644 --- a/drivers/net/ena/ena_ethdev.c +++ b/drivers/net/ena/ena_ethdev.c @@ -1994,6 +1994,9 @@ static void ena_infos_get(struct rte_eth_dev *dev, dev_info->tx_offload_capa = tx_feat; dev_info->tx_queue_offload_capa = tx_feat; + dev_info->flow_type_rss_offloads = ETH_RSS_IP | ETH_RSS_TCP | + ETH_RSS_UDP; + dev_info->min_rx_bufsize = ENA_MIN_FRAME_LEN; dev_info->max_rx_pktlen = adapter->max_mtu; dev_info->max_mac_addrs = 1; From patchwork Fri Dec 14 13:18:39 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michal Krawczyk X-Patchwork-Id: 48871 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 72CEA1BC00; Fri, 14 Dec 2018 14:19:31 +0100 (CET) Received: from mail-lf1-f67.google.com (mail-lf1-f67.google.com [209.85.167.67]) by dpdk.org (Postfix) with ESMTP id E9A281BBB4 for ; Fri, 14 Dec 2018 14:19:13 +0100 (CET) Received: by mail-lf1-f67.google.com with SMTP id b20so4217750lfa.12 for ; Fri, 14 Dec 2018 05:19:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=semihalf-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=sxV7G4kgEPZjQvV+9mXVYuMaz6/pvrsPOMxvGIa99uA=; b=TXDRoQIV6hV6qGSD5/+UAo5ClPCEAOfUuNrBzPW8RhQvaXcasxHnaTgP2nvZf75FdL 0L64nW1h05lk2PmQ90Nn6i5SeruABIIkxdM2bQo1opGiGTM2XsrFt7Caw8j87ErPE/f+ 8EcmmzB4ffC9NzI7WUfeWNfXz/yztLzU1qv0Mvgotm/h71OlEU5viMeabdkqVA1CLc06 qUu7cM7u2yVPwnMhmyjcrMT6DpmqvnHTOh8EBbV1XIhRrNj2b8ytxTsza/kN/gX2byrD YBnxVxoqYsvqZrRl51qEHlE6bomlZG9O+gCe0BW8oceTrV5VvW1vbbUQdHDrQ3US+TMe agqQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=sxV7G4kgEPZjQvV+9mXVYuMaz6/pvrsPOMxvGIa99uA=; b=ZoDJ9GcYcaTLPm5JzLspdDJeDcHeRCnTdZH3NeELBG8+38VJcVnLw3c72nqJpKGLhq VijKBadQf3Q8sYSrYdmV5UDnIHeqTYOYTiwqg+EPoXBdkAGmvd7Ax3cxA3SWS6lUrhcf EYoyQGuchkXUNuTIjDsn/EdAL/mf6/Etik02eOvsl7PWnQ5kgY0qldtWu4IHQvzkQios FCwXOwRaI43zWPyHoKso3Ag0DzOeyps/wr+ft3yum+PNtJ6EoccIxWiJb/IWI7uOxBS6 QCRhanJ6f/s/1wrvok/Zbb0s2lVzZ6dhOqvPR9a7z6e/JtiTJ/HY1lL+pTjwK9Bbz+Lg 5+FQ== X-Gm-Message-State: AA+aEWaYpqXDXoD7eWdgidOtLqhXSu2W7Xk18MRs03ujLeTCIpIRqZQR ggF1lBTf2YPRkpN/p8N3aZ1b8aoc1KY= X-Google-Smtp-Source: AFSGD/WlxCO7otvHUx9vDFu4WzIgNS7VQATE/NR0/BHN6pS8hAhGuaMCV9IDynMrKeMw07vM87Eg8g== X-Received: by 2002:a19:c4cc:: with SMTP id u195mr1763600lff.141.1544793553231; Fri, 14 Dec 2018 05:19:13 -0800 (PST) Received: from mkPC.semihalf.local (31-172-191-173.noc.fibertech.net.pl. [31.172.191.173]) by smtp.gmail.com with ESMTPSA id o25sm873884lfd.29.2018.12.14.05.19.12 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 14 Dec 2018 05:19:12 -0800 (PST) From: Michal Krawczyk To: dev@dpdk.org Cc: gtzalik@dpdk.org, mw@dpdk.org, matua@amazon.com, rk@semihalf.com, stable@dpdk.org Date: Fri, 14 Dec 2018 14:18:39 +0100 Message-Id: <20181214131846.22439-14-mk@semihalf.com> X-Mailer: git-send-email 2.14.1 In-Reply-To: <20181214131846.22439-1-mk@semihalf.com> References: <20181214131846.22439-1-mk@semihalf.com> Subject: [dpdk-dev] [PATCH 13/20] net/ena: fix invalid reference to variable in union X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Rafal Kozik Use empty_rx_reqs instead of empty_tx_reqs. As those two variables are part of union this not cause any failure, but for consistency should be changed. Fixes: c2034976673d ("net/ena: add Rx out of order completion") Cc: stable@dpdk.org Signed-off-by: Rafal Kozik Acked-by: Michal Krawczyk --- drivers/net/ena/ena_ethdev.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/net/ena/ena_ethdev.c b/drivers/net/ena/ena_ethdev.c index 3aa32a3bf..14165561e 100644 --- a/drivers/net/ena/ena_ethdev.c +++ b/drivers/net/ena/ena_ethdev.c @@ -1387,7 +1387,7 @@ static int ena_rx_queue_setup(struct rte_eth_dev *dev, } for (i = 0; i < nb_desc; i++) - rxq->empty_tx_reqs[i] = i; + rxq->empty_rx_reqs[i] = i; /* Store pointer to this queue in upper layer */ rxq->configured = 1; From patchwork Fri Dec 14 13:18:40 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michal Krawczyk X-Patchwork-Id: 48872 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 2AA9C1BC06; Fri, 14 Dec 2018 14:19:34 +0100 (CET) Received: from mail-lj1-f195.google.com (mail-lj1-f195.google.com [209.85.208.195]) by dpdk.org (Postfix) with ESMTP id 7EA0A1BBD4 for ; Fri, 14 Dec 2018 14:19:15 +0100 (CET) Received: by mail-lj1-f195.google.com with SMTP id s5-v6so4846611ljd.12 for ; Fri, 14 Dec 2018 05:19:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=semihalf-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=66B0odI/gR5V4nlyBIux/xYcrmUbkqL+7rj/8042agU=; b=A+jU8KbhPXHPuXRlhIrNQ5J//rDO43EkmahdXl6SQ+LLwNV9fVQLeuo0l25qMy4B1C DYSlZESWiCZsfohzB08TKIQF4W1VQc+8RharwFXjFa9M2TRlbh4n0AyU8mxpFs0iOBnI esxqzFHg8IdDQ2Ki0ojqspHAldqlMWQV1HtWPXIbuOKZk0RlgTj0f8TUcWw7c1gkUcj4 s/FDNAEjhZYc/Phb0qvoonpjDCse/tcNBx5JHXE7kij2iylI8rdHugqq3tjVPJi9UN94 M1ESnW6+1x/BVSyriAyAMBQKAmgzO+8/bzZcm9umGvZ+nn9XJnIaCRjzN6Ud1Oqrwl7Q WTWQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=66B0odI/gR5V4nlyBIux/xYcrmUbkqL+7rj/8042agU=; b=NU0DCkIJw1WruxIKx8OeEMwlTRw1tYCKnzWN62SMuHFzld8MfkySBSJ2cKSgZfmyUT kOmzWRIViuEKF1N7uDFC9tkbobvQTVrjRfJkZO3kq5w1PL+cPpH/lTFkpGXDpjSC0EOY J4uyLog0PPatb8VmdoYwErCZOeuFZdQxSi4WTF5eCoeUran4Kgd4bB1+acSFgZ3IWSff jfnOOtmjBEU3nTNoef55iAaIIeHZABm+mJjE0klcTu7LBnLO67hVNk6dkNiUmjfO+Irq JGCMuC14Rfx+xR2k1Bq0p4Sv/ESTHXYaLybpnj8+0EOltRVvdPmRCp6eMjDJh0f5oxdG WivA== X-Gm-Message-State: AA+aEWYRjfRuLO1nnumRVBbtNdZ3dgUzPg9GZbL/XFvpR+jN8SoLjHlp FNtCm/75zkqADikQd0lTKD8UlHEMyJc= X-Google-Smtp-Source: AFSGD/UxODFraYroAFvHyIkVvl3lnqWu7hOY/7aA+4/6+nBFEMRDoQz2UMqCE0qSjwzKk3BHpAqPGw== X-Received: by 2002:a2e:908b:: with SMTP id l11-v6mr1830739ljg.150.1544793554793; Fri, 14 Dec 2018 05:19:14 -0800 (PST) Received: from mkPC.semihalf.local (31-172-191-173.noc.fibertech.net.pl. [31.172.191.173]) by smtp.gmail.com with ESMTPSA id o25sm873884lfd.29.2018.12.14.05.19.13 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 14 Dec 2018 05:19:14 -0800 (PST) From: Michal Krawczyk To: dev@dpdk.org Cc: gtzalik@dpdk.org, mw@dpdk.org, matua@amazon.com, rk@semihalf.com, stable@dpdk.org Date: Fri, 14 Dec 2018 14:18:40 +0100 Message-Id: <20181214131846.22439-15-mk@semihalf.com> X-Mailer: git-send-email 2.14.1 In-Reply-To: <20181214131846.22439-1-mk@semihalf.com> References: <20181214131846.22439-1-mk@semihalf.com> Subject: [dpdk-dev] [PATCH 14/20] net/ena: fix cleanup for out of order packets X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Rafal Kozik When wrong req_id is detected some previous mbufs could be used for receiving different segments of received packets. In such cases chained mbufs will be twice returned to pool. To prevent it chained mbuf is now freed just after error detection. To simplify cleaning, pointers taken for Rx ring are set to NULL. As after ena_rx_queue_release_bufs and ena_tx_queue_release_bufs queues are not used updating of next_to_clean pointer is not necessary. Fixes: c2034976673d ("net/ena: add Rx out of order completion") Cc: stable@dpdk.org Signed-off-by: Rafal Kozik Acked-by: Michal Krawczyk --- drivers/net/ena/ena_ethdev.c | 24 ++++++++++-------------- 1 file changed, 10 insertions(+), 14 deletions(-) diff --git a/drivers/net/ena/ena_ethdev.c b/drivers/net/ena/ena_ethdev.c index 14165561e..ce0ca40c4 100644 --- a/drivers/net/ena/ena_ethdev.c +++ b/drivers/net/ena/ena_ethdev.c @@ -770,17 +770,11 @@ static void ena_tx_queue_release(void *queue) static void ena_rx_queue_release_bufs(struct ena_ring *ring) { - unsigned int ring_mask = ring->ring_size - 1; - - while (ring->next_to_clean != ring->next_to_use) { - struct rte_mbuf *m = - ring->rx_buffer_info[ring->next_to_clean & ring_mask]; - - if (m) - rte_mbuf_raw_free(m); - - ring->next_to_clean++; - } + for (unsigned int i = 0; i < ring->ring_size; ++i) + if (ring->rx_buffer_info[i]) { + rte_mbuf_raw_free(ring->rx_buffer_info[i]); + ring->rx_buffer_info[i] = NULL; + } } static void ena_tx_queue_release_bufs(struct ena_ring *ring) @@ -792,8 +786,6 @@ static void ena_tx_queue_release_bufs(struct ena_ring *ring) if (tx_buf->mbuf) rte_pktmbuf_free(tx_buf->mbuf); - - ring->next_to_clean++; } } @@ -2077,10 +2069,14 @@ static uint16_t eth_ena_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, while (segments < ena_rx_ctx.descs) { req_id = ena_rx_ctx.ena_bufs[segments].req_id; rc = validate_rx_req_id(rx_ring, req_id); - if (unlikely(rc)) + if (unlikely(rc)) { + if (segments != 0) + rte_mbuf_raw_free(mbuf_head); break; + } mbuf = rx_buff_info[req_id]; + rx_buff_info[req_id] = NULL; mbuf->data_len = ena_rx_ctx.ena_bufs[segments].len; mbuf->data_off = RTE_PKTMBUF_HEADROOM; mbuf->refcnt = 1; From patchwork Fri Dec 14 13:18:41 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michal Krawczyk X-Patchwork-Id: 48873 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 6CF5D1BC0C; Fri, 14 Dec 2018 14:19:36 +0100 (CET) Received: from mail-lj1-f194.google.com (mail-lj1-f194.google.com [209.85.208.194]) by dpdk.org (Postfix) with ESMTP id C76501BBD8 for ; Fri, 14 Dec 2018 14:19:16 +0100 (CET) Received: by mail-lj1-f194.google.com with SMTP id l15-v6so4857208lja.9 for ; Fri, 14 Dec 2018 05:19:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=semihalf-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=5xFEFGHAsNLgJhKcjue8GxC66YHYFu816mwu83F5sQE=; b=U1Uh1LnYvoMCU4e2796TeQvM7IT4u8nzQYwkRUz/EQBXotjrTWCPHvmAZSwz6tvY/4 dQmsFR8lxEHn0brLtJ0KCeirxRuXeZmVcmyiP36abYHmc81WMMfpy8Vm2Li/vN+dY5oa AgSarQ+CxBnA4VliQ5e3ZqseH68KVbVUCobgUWwMiyn1KGX/8cGfRxqzhB3MmBWvnSmk 08EWqG9ub1IT+x+B0sQsYRqdVSyn4D3ruCY7tsrYyJSPBOCSlBiDPul+7xX9UsWJQ1m0 tksULHO5pO8RzEmcHq142NQTx+V5CeEbwHu4QblJMCr9qffKDwXA9/578p1SjFYiLy+n ff9g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=5xFEFGHAsNLgJhKcjue8GxC66YHYFu816mwu83F5sQE=; b=bzcirkNl03OlRnJzuu/LqZI17oQnXs6s4RYHH7s3uhZic1hrMtTxIzkPpTP2gJ6lab ZmGRarBbKRdG1rIpnIziytOW6DzzLbQMYJaoFNkn1ofoyEp6MvYxnwtInx6iw2N9F+2r 1iRTBycixKYVy2fhb69W89joVs0LIVea5sGK0vG0UEL9Dk/M8x/uY5w/ZtU/iGU5qMjG KzORhq5jqwGznpcYWiETXPUKFKFLI6BSqNuyIWCBX73Eh8g7M9OcUx8b9FHyQhjDJfab Q30UzbJPQfQNe+y1nPaHO119QGSqQdDQs2UnRvGwPp96xOJMjEEaIh1/fZkeZ3w2nGe1 R39w== X-Gm-Message-State: AA+aEWZIrON2PBEk3J0t8iZL08imrFtxKkXmXpAF2iyjOZgPhvKUTInh 8PlW4Xx8E8EXr+20sjCqNzDBkmDx188= X-Google-Smtp-Source: AFSGD/VzqVYm92EkuRY4zVMnGhaV6ZAAce+9iPuTNJd6XSgOrKhW/cZe+SfUzGvxwBeQFPvMlJgAaQ== X-Received: by 2002:a2e:6594:: with SMTP id e20-v6mr1998598ljf.123.1544793556064; Fri, 14 Dec 2018 05:19:16 -0800 (PST) Received: from mkPC.semihalf.local (31-172-191-173.noc.fibertech.net.pl. [31.172.191.173]) by smtp.gmail.com with ESMTPSA id o25sm873884lfd.29.2018.12.14.05.19.14 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 14 Dec 2018 05:19:15 -0800 (PST) From: Michal Krawczyk To: dev@dpdk.org Cc: gtzalik@dpdk.org, mw@dpdk.org, matua@amazon.com, rk@semihalf.com, Michal Krawczyk , Solganik Alexander Date: Fri, 14 Dec 2018 14:18:41 +0100 Message-Id: <20181214131846.22439-16-mk@semihalf.com> X-Mailer: git-send-email 2.14.1 In-Reply-To: <20181214131846.22439-1-mk@semihalf.com> References: <20181214131846.22439-1-mk@semihalf.com> Subject: [dpdk-dev] [PATCH 15/20] net/ena: add per-queue software counters stats X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Those counters provide information regards sent/received bytes and packets per queue. Signed-off-by: Solganik Alexander Signed-off-by: Michal Krawczyk --- drivers/net/ena/ena_ethdev.c | 36 +++++++++++++++++++++++++- drivers/net/ena/ena_ethdev.h | 60 ++++++++++++++++++++++++-------------------- 2 files changed, 68 insertions(+), 28 deletions(-) diff --git a/drivers/net/ena/ena_ethdev.c b/drivers/net/ena/ena_ethdev.c index ce0ca40c4..799dcefcd 100644 --- a/drivers/net/ena/ena_ethdev.c +++ b/drivers/net/ena/ena_ethdev.c @@ -160,6 +160,7 @@ static const struct ena_stats ena_stats_rx_strings[] = { ENA_STAT_RX_ENTRY(dma_mapping_err), ENA_STAT_RX_ENTRY(bad_desc_num), ENA_STAT_RX_ENTRY(small_copy_len_pkt), + ENA_STAT_TX_ENTRY(bad_req_id), }; static const struct ena_stats ena_stats_ena_com_strings[] = { @@ -390,6 +391,7 @@ static inline int validate_rx_req_id(struct ena_ring *rx_ring, uint16_t req_id) rx_ring->adapter->reset_reason = ENA_REGS_RESET_INV_RX_REQ_ID; rx_ring->adapter->trigger_reset = true; + ++rx_ring->rx_stats.bad_req_id; return -EFAULT; } @@ -969,6 +971,8 @@ static int ena_stats_get(struct rte_eth_dev *dev, (struct ena_adapter *)(dev->data->dev_private); struct ena_com_dev *ena_dev = &adapter->ena_dev; int rc; + int i; + int max_rings_stats; if (rte_eal_process_type() != RTE_PROC_PRIMARY) return -ENOTSUP; @@ -996,6 +1000,27 @@ static int ena_stats_get(struct rte_eth_dev *dev, stats->ierrors = rte_atomic64_read(&adapter->drv_stats->ierrors); stats->oerrors = rte_atomic64_read(&adapter->drv_stats->oerrors); stats->rx_nombuf = rte_atomic64_read(&adapter->drv_stats->rx_nombuf); + + max_rings_stats = RTE_MIN(dev->data->nb_rx_queues, + RTE_ETHDEV_QUEUE_STAT_CNTRS); + for (i = 0; i < max_rings_stats; ++i) { + struct ena_stats_rx *rx_stats = &adapter->rx_ring[i].rx_stats; + + stats->q_ibytes[i] = rx_stats->bytes; + stats->q_ipackets[i] = rx_stats->cnt; + stats->q_errors[i] = rx_stats->bad_desc_num + + rx_stats->bad_req_id; + } + + max_rings_stats = RTE_MIN(dev->data->nb_tx_queues, + RTE_ETHDEV_QUEUE_STAT_CNTRS); + for (i = 0; i < max_rings_stats; ++i) { + struct ena_stats_tx *tx_stats = &adapter->tx_ring[i].tx_stats; + + stats->q_obytes[i] = tx_stats->bytes; + stats->q_opackets[i] = tx_stats->cnt; + } + return 0; } @@ -1408,6 +1433,7 @@ static int ena_populate_rx_queue(struct ena_ring *rxq, unsigned int count) rc = rte_mempool_get_bulk(rxq->mb_pool, (void **)mbufs, count); if (unlikely(rc < 0)) { rte_atomic64_inc(&rxq->adapter->drv_stats->rx_nombuf); + ++rxq->rx_stats.page_alloc_fail; PMD_RX_LOG(DEBUG, "there are no enough free buffers"); return 0; } @@ -2108,8 +2134,10 @@ static uint16_t eth_ena_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, /* pass to DPDK application head mbuf */ rx_pkts[recv_idx] = mbuf_head; recv_idx++; + rx_ring->rx_stats.bytes += mbuf_head->pkt_len; } + rx_ring->rx_stats.cnt += recv_idx; rx_ring->next_to_clean = next_to_clean; desc_in_use = desc_in_use - completed + 1; @@ -2256,6 +2284,7 @@ static uint16_t eth_ena_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t push_len = 0; uint16_t delta = 0; int nb_hw_desc; + uint32_t total_length; /* Check adapter state */ if (unlikely(tx_ring->adapter->state != ENA_ADAPTER_STATE_RUNNING)) { @@ -2270,6 +2299,7 @@ static uint16_t eth_ena_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, for (sent_idx = 0; sent_idx < nb_pkts; sent_idx++) { mbuf = tx_pkts[sent_idx]; + total_length = 0; rc = ena_check_and_linearize_mbuf(tx_ring, mbuf); if (unlikely(rc)) @@ -2335,6 +2365,7 @@ static uint16_t eth_ena_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, ebuf++; tx_info->num_of_bufs++; } + total_length += mbuf->data_len; while ((mbuf = mbuf->next) != NULL) { seg_len = mbuf->data_len; @@ -2347,6 +2378,7 @@ static uint16_t eth_ena_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, ebuf->paddr = mbuf->buf_iova + mbuf->data_off + delta; ebuf->len = seg_len - delta; + total_length += ebuf->len; ebuf++; tx_info->num_of_bufs++; @@ -2373,6 +2405,8 @@ static uint16_t eth_ena_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, tx_info->tx_descs = nb_hw_desc; next_to_use++; + tx_ring->tx_stats.cnt += tx_info->num_of_bufs; + tx_ring->tx_stats.bytes += total_length; } /* If there are ready packets to be xmitted... */ @@ -2380,7 +2414,7 @@ static uint16_t eth_ena_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, /* ...let HW do its best :-) */ rte_wmb(); ena_com_write_sq_doorbell(tx_ring->ena_com_io_sq); - + tx_ring->tx_stats.doorbells++; tx_ring->next_to_use = next_to_use; } diff --git a/drivers/net/ena/ena_ethdev.h b/drivers/net/ena/ena_ethdev.h index 713cdea97..255e1c0ec 100644 --- a/drivers/net/ena/ena_ethdev.h +++ b/drivers/net/ena/ena_ethdev.h @@ -79,6 +79,34 @@ struct ena_calc_queue_size_ctx { u16 max_rx_sgl_size; }; +struct ena_stats_tx { + u64 cnt; + u64 bytes; + u64 queue_stop; + u64 prepare_ctx_err; + u64 queue_wakeup; + u64 dma_mapping_err; + u64 linearize; + u64 linearize_failed; + u64 tx_poll; + u64 doorbells; + u64 missing_tx_comp; + u64 bad_req_id; +}; + +struct ena_stats_rx { + u64 cnt; + u64 bytes; + u64 refil_partial; + u64 bad_csum; + u64 page_alloc_fail; + u64 skb_alloc_fail; + u64 dma_mapping_err; + u64 bad_desc_num; + u64 small_copy_len_pkt; + u64 bad_req_id; +}; + struct ena_ring { u16 next_to_use; u16 next_to_clean; @@ -116,6 +144,11 @@ struct ena_ring { struct ena_adapter *adapter; uint64_t offloads; u16 sgl_size; + + union { + struct ena_stats_rx rx_stats; + struct ena_stats_tx tx_stats; + }; } __rte_cache_aligned; enum ena_adapter_state { @@ -143,33 +176,6 @@ struct ena_stats_dev { u64 admin_q_pause; }; -struct ena_stats_tx { - u64 cnt; - u64 bytes; - u64 queue_stop; - u64 prepare_ctx_err; - u64 queue_wakeup; - u64 dma_mapping_err; - u64 linearize; - u64 linearize_failed; - u64 tx_poll; - u64 doorbells; - u64 missing_tx_comp; - u64 bad_req_id; -}; - -struct ena_stats_rx { - u64 cnt; - u64 bytes; - u64 refil_partial; - u64 bad_csum; - u64 page_alloc_fail; - u64 skb_alloc_fail; - u64 dma_mapping_err; - u64 bad_desc_num; - u64 small_copy_len_pkt; -}; - /* board specific private data structure */ struct ena_adapter { /* OS defined structs */ From patchwork Fri Dec 14 13:18:42 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michal Krawczyk X-Patchwork-Id: 48874 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 391DD1BC1A; Fri, 14 Dec 2018 14:19:38 +0100 (CET) Received: from mail-lf1-f66.google.com (mail-lf1-f66.google.com [209.85.167.66]) by dpdk.org (Postfix) with ESMTP id BF8151BBD5 for ; Fri, 14 Dec 2018 14:19:18 +0100 (CET) Received: by mail-lf1-f66.google.com with SMTP id l10so4213103lfh.9 for ; Fri, 14 Dec 2018 05:19:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=semihalf-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=Oi7AkfGautnrYJZBAcqKLZj+Z3VroaFDZFsJhIv2UKs=; b=oy/+7rhvWz99690o8WxQCLgyl4WQZykrAmMejjAYJUsAnPROBJsdMqcSsVR6RMXCKC GfuOuQhQ+TdoBHiUD/Ta8H4pItIRtO4yRLZXZMoblZvX285YEHiLNbgx7fnPZICszVID MfkvYeh3ZxmojGBaspJhPGu2eEmEMKCXLHkSWi/bOVi8vfdKnnV5ww8Edicls6zYvZ/g iRlS8PRMC4GSrQ1MW6syl114Oj4b44QIElLGv5aGJm8ATev+SBMMRK7LuMXBWjd11qQr lasVd8l1wO0bUZNoDMFq9QYfE4ObPFmSCPRGz/DFwxMT7Roqe4sz+ndi7H2MFJjikwbV k+Bw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=Oi7AkfGautnrYJZBAcqKLZj+Z3VroaFDZFsJhIv2UKs=; b=NjXHqN/0iKdB7t4oBVYodcR1Z5k5G0X2tC83uvJ4vHuRoXpWuC1teKHmGW27SbMEXQ CfwxBITey3wqJ3tJy6BI3qRFi7JhRD+hVlp+5v3tpWfp9OPxObA7Bu+XeBXM9luaAiL0 wTwjpR4GzavZP21Wwer5JOfy6b8PMMmYkeisBqspo5HsG1Cyq2kdXvLGvult8LlS/mdj H4e8jMT5WZdZf/yNMnweVLcKerhmJOjmPC235W9pnkMxr7etJQqXGtE+ZI/16kl1Zg2n /xWmm0xbPpSUZJXzJ000ZtSxHGNmsNNjhQYgzuVLK8/M8EK5s2TfaKBQY2f1CHJvGBUn MBAQ== X-Gm-Message-State: AA+aEWaXiYvCaaq+JGflBYyoeW22C3NBzCgljqaDzbzlP+Ov2tpwPxBI x0VHO9pCfrDyqbDTCgkGh2ZF4xCJXc4= X-Google-Smtp-Source: AFSGD/UYJrH6bGLx4luJk6YJH8c5DE7EWvSRBM0LP0mv61AWfFhWktw5XbGiRcdVQv5DW7f5QzXkpg== X-Received: by 2002:a19:a149:: with SMTP id k70mr1795773lfe.5.1544793557825; Fri, 14 Dec 2018 05:19:17 -0800 (PST) Received: from mkPC.semihalf.local (31-172-191-173.noc.fibertech.net.pl. [31.172.191.173]) by smtp.gmail.com with ESMTPSA id o25sm873884lfd.29.2018.12.14.05.19.16 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 14 Dec 2018 05:19:16 -0800 (PST) From: Michal Krawczyk To: dev@dpdk.org Cc: gtzalik@dpdk.org, mw@dpdk.org, matua@amazon.com, rk@semihalf.com, Solganik Alexander , Michal Krawczyk Date: Fri, 14 Dec 2018 14:18:42 +0100 Message-Id: <20181214131846.22439-17-mk@semihalf.com> X-Mailer: git-send-email 2.14.1 In-Reply-To: <20181214131846.22439-1-mk@semihalf.com> References: <20181214131846.22439-1-mk@semihalf.com> Subject: [dpdk-dev] [PATCH 16/20] net/ena: expose ENA-specific statistics in extended stats X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Solganik Alexander ENA PMD is having it's own custom statistics counters. They are exposed to the application by using the xstats DPDK API. The deprecated and unused statistics are removed, together with old API. Signed-off-by: Solganik Alexander Signed-off-by: Michal Krawczyk --- drivers/net/ena/ena_ethdev.c | 274 +++++++++++++++++++++++++++++++++---------- drivers/net/ena/ena_ethdev.h | 22 ++-- 2 files changed, 220 insertions(+), 76 deletions(-) diff --git a/drivers/net/ena/ena_ethdev.c b/drivers/net/ena/ena_ethdev.c index 799dcefcd..c3a26d402 100644 --- a/drivers/net/ena/ena_ethdev.c +++ b/drivers/net/ena/ena_ethdev.c @@ -80,7 +80,6 @@ #define ENA_RX_RSS_TABLE_LOG_SIZE 7 #define ENA_RX_RSS_TABLE_SIZE (1 << ENA_RX_RSS_TABLE_LOG_SIZE) #define ENA_HASH_KEY_SIZE 40 -#define ENA_ETH_SS_STATS 0xFF #define ETH_GSTRING_LEN 32 #define ARRAY_SIZE(x) (sizeof(x) / sizeof((x)[0])) @@ -97,11 +96,6 @@ struct ena_stats { int stat_offset; }; -#define ENA_STAT_ENA_COM_ENTRY(stat) { \ - .name = #stat, \ - .stat_offset = offsetof(struct ena_com_stats_admin, stat) \ -} - #define ENA_STAT_ENTRY(stat, stat_type) { \ .name = #stat, \ .stat_offset = offsetof(struct ena_stats_##stat_type, stat) \ @@ -126,55 +120,36 @@ struct ena_stats { uint32_t ena_alloc_cnt; static const struct ena_stats ena_stats_global_strings[] = { - ENA_STAT_GLOBAL_ENTRY(tx_timeout), - ENA_STAT_GLOBAL_ENTRY(io_suspend), - ENA_STAT_GLOBAL_ENTRY(io_resume), ENA_STAT_GLOBAL_ENTRY(wd_expired), - ENA_STAT_GLOBAL_ENTRY(interface_up), - ENA_STAT_GLOBAL_ENTRY(interface_down), - ENA_STAT_GLOBAL_ENTRY(admin_q_pause), + ENA_STAT_GLOBAL_ENTRY(dev_start), + ENA_STAT_GLOBAL_ENTRY(dev_stop), }; static const struct ena_stats ena_stats_tx_strings[] = { ENA_STAT_TX_ENTRY(cnt), ENA_STAT_TX_ENTRY(bytes), - ENA_STAT_TX_ENTRY(queue_stop), - ENA_STAT_TX_ENTRY(queue_wakeup), - ENA_STAT_TX_ENTRY(dma_mapping_err), + ENA_STAT_TX_ENTRY(prepare_ctx_err), ENA_STAT_TX_ENTRY(linearize), ENA_STAT_TX_ENTRY(linearize_failed), ENA_STAT_TX_ENTRY(tx_poll), ENA_STAT_TX_ENTRY(doorbells), - ENA_STAT_TX_ENTRY(prepare_ctx_err), - ENA_STAT_TX_ENTRY(missing_tx_comp), ENA_STAT_TX_ENTRY(bad_req_id), + ENA_STAT_TX_ENTRY(available_desc), }; static const struct ena_stats ena_stats_rx_strings[] = { ENA_STAT_RX_ENTRY(cnt), ENA_STAT_RX_ENTRY(bytes), - ENA_STAT_RX_ENTRY(refil_partial), + ENA_STAT_RX_ENTRY(refill_partial), ENA_STAT_RX_ENTRY(bad_csum), - ENA_STAT_RX_ENTRY(page_alloc_fail), - ENA_STAT_RX_ENTRY(skb_alloc_fail), - ENA_STAT_RX_ENTRY(dma_mapping_err), + ENA_STAT_RX_ENTRY(mbuf_alloc_fail), ENA_STAT_RX_ENTRY(bad_desc_num), - ENA_STAT_RX_ENTRY(small_copy_len_pkt), - ENA_STAT_TX_ENTRY(bad_req_id), -}; - -static const struct ena_stats ena_stats_ena_com_strings[] = { - ENA_STAT_ENA_COM_ENTRY(aborted_cmd), - ENA_STAT_ENA_COM_ENTRY(submitted_cmd), - ENA_STAT_ENA_COM_ENTRY(completed_cmd), - ENA_STAT_ENA_COM_ENTRY(out_of_space), - ENA_STAT_ENA_COM_ENTRY(no_completion), + ENA_STAT_RX_ENTRY(bad_req_id), }; #define ENA_STATS_ARRAY_GLOBAL ARRAY_SIZE(ena_stats_global_strings) #define ENA_STATS_ARRAY_TX ARRAY_SIZE(ena_stats_tx_strings) #define ENA_STATS_ARRAY_RX ARRAY_SIZE(ena_stats_rx_strings) -#define ENA_STATS_ARRAY_ENA_COM ARRAY_SIZE(ena_stats_ena_com_strings) #define QUEUE_OFFLOADS (DEV_TX_OFFLOAD_TCP_CKSUM |\ DEV_TX_OFFLOAD_UDP_CKSUM |\ @@ -260,11 +235,20 @@ static int ena_rss_reta_update(struct rte_eth_dev *dev, static int ena_rss_reta_query(struct rte_eth_dev *dev, struct rte_eth_rss_reta_entry64 *reta_conf, uint16_t reta_size); -static int ena_get_sset_count(struct rte_eth_dev *dev, int sset); static void ena_interrupt_handler_rte(void *cb_arg); static void ena_timer_wd_callback(struct rte_timer *timer, void *arg); static void ena_destroy_device(struct rte_eth_dev *eth_dev); static int eth_ena_dev_init(struct rte_eth_dev *eth_dev); +static int ena_xstats_get_names(struct rte_eth_dev *dev, + struct rte_eth_xstat_name *xstats_names, + unsigned int n); +static int ena_xstats_get(struct rte_eth_dev *dev, + struct rte_eth_xstat *stats, + unsigned int n); +static int ena_xstats_get_by_id(struct rte_eth_dev *dev, + const uint64_t *ids, + uint64_t *values, + unsigned int n); static const struct eth_dev_ops ena_dev_ops = { .dev_configure = ena_dev_configure, @@ -275,6 +259,9 @@ static const struct eth_dev_ops ena_dev_ops = { .dev_stop = ena_stop, .link_update = ena_link_update, .stats_get = ena_stats_get, + .xstats_get_names = ena_xstats_get_names, + .xstats_get = ena_xstats_get, + .xstats_get_by_id = ena_xstats_get_by_id, .mtu_set = ena_mtu_set, .rx_queue_release = ena_rx_queue_release, .tx_queue_release = ena_tx_queue_release, @@ -412,6 +399,7 @@ static int validate_tx_req_id(struct ena_ring *tx_ring, u16 req_id) RTE_LOG(ERR, PMD, "Invalid req_id: %hu\n", req_id); /* Trigger device reset */ + ++tx_ring->tx_stats.bad_req_id; tx_ring->adapter->reset_reason = ENA_REGS_RESET_INV_TX_REQ_ID; tx_ring->adapter->trigger_reset = true; return -EFAULT; @@ -463,24 +451,12 @@ static void ena_config_host_info(struct ena_com_dev *ena_dev) ena_com_delete_host_info(ena_dev); } -static int -ena_get_sset_count(struct rte_eth_dev *dev, int sset) +/* This function calculates the number of xstats based on the current config */ +static unsigned int ena_xstats_calc_num(struct rte_eth_dev *dev) { - if (sset != ETH_SS_STATS) - return -EOPNOTSUPP; - - /* Workaround for clang: - * touch internal structures to prevent - * compiler error - */ - ENA_TOUCH(ena_stats_global_strings); - ENA_TOUCH(ena_stats_tx_strings); - ENA_TOUCH(ena_stats_rx_strings); - ENA_TOUCH(ena_stats_ena_com_strings); - - return dev->data->nb_tx_queues * - (ENA_STATS_ARRAY_TX + ENA_STATS_ARRAY_RX) + - ENA_STATS_ARRAY_GLOBAL + ENA_STATS_ARRAY_ENA_COM; + return ENA_STATS_ARRAY_GLOBAL + + (dev->data->nb_tx_queues * ENA_STATS_ARRAY_TX) + + (dev->data->nb_rx_queues * ENA_STATS_ARRAY_RX); } static void ena_config_debug_area(struct ena_adapter *adapter) @@ -488,11 +464,7 @@ static void ena_config_debug_area(struct ena_adapter *adapter) u32 debug_area_size; int rc, ss_count; - ss_count = ena_get_sset_count(adapter->rte_dev, ETH_SS_STATS); - if (ss_count <= 0) { - RTE_LOG(ERR, PMD, "SS count is negative\n"); - return; - } + ss_count = ena_xstats_calc_num(adapter->rte_dev); /* allocate 32 bytes for each string and 64bit for the value */ debug_area_size = ss_count * ETH_GSTRING_LEN + sizeof(u64) * ss_count; @@ -1089,6 +1061,7 @@ static int ena_start(struct rte_eth_dev *dev) rte_timer_reset(&adapter->timer_wd, ticks, PERIODICAL, rte_lcore_id(), ena_timer_wd_callback, adapter); + ++adapter->dev_stats.dev_start; adapter->state = ENA_ADAPTER_STATE_RUNNING; return 0; @@ -1117,6 +1090,7 @@ static void ena_stop(struct rte_eth_dev *dev) RTE_LOG(ERR, PMD, "Device reset failed rc=%d\n", rc); } + ++adapter->dev_stats.dev_stop; adapter->state = ENA_ADAPTER_STATE_STOPPED; } @@ -1228,8 +1202,11 @@ static int ena_queue_start(struct ena_ring *ring) ring->next_to_clean = 0; ring->next_to_use = 0; - if (ring->type == ENA_RING_TYPE_TX) + if (ring->type == ENA_RING_TYPE_TX) { + ring->tx_stats.available_desc = + ena_com_free_desc(ring->ena_com_io_sq); return 0; + } bufs_num = ring->ring_size - 1; rc = ena_populate_rx_queue(ring, bufs_num); @@ -1321,7 +1298,6 @@ static int ena_tx_queue_setup(struct rte_eth_dev *dev, txq->offloads = tx_conf->offloads | dev->data->dev_conf.txmode.offloads; } - /* Store pointer to this queue in upper layer */ txq->configured = 1; dev->data->tx_queues[queue_idx] = txq; @@ -1433,7 +1409,7 @@ static int ena_populate_rx_queue(struct ena_ring *rxq, unsigned int count) rc = rte_mempool_get_bulk(rxq->mb_pool, (void **)mbufs, count); if (unlikely(rc < 0)) { rte_atomic64_inc(&rxq->adapter->drv_stats->rx_nombuf); - ++rxq->rx_stats.page_alloc_fail; + ++rxq->rx_stats.mbuf_alloc_fail; PMD_RX_LOG(DEBUG, "there are no enough free buffers"); return 0; } @@ -1471,6 +1447,7 @@ static int ena_populate_rx_queue(struct ena_ring *rxq, unsigned int count) "buffers (from %d)\n", rxq->id, i, count); rte_mempool_put_bulk(rxq->mb_pool, (void **)(&mbufs[i]), count - i); + ++rxq->rx_stats.refill_partial; } /* When we submitted free recources to device... */ @@ -1601,6 +1578,7 @@ static void check_for_missing_keep_alive(struct ena_adapter *adapter) RTE_LOG(ERR, PMD, "Keep alive timeout\n"); adapter->reset_reason = ENA_REGS_RESET_KEEP_ALIVE_TO; adapter->trigger_reset = true; + ++adapter->dev_stats.wd_expired; } } @@ -2086,6 +2064,7 @@ static uint16_t eth_ena_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, rx_ring->adapter->reset_reason = ENA_REGS_RESET_TOO_MANY_RX_DESCS; rx_ring->adapter->trigger_reset = true; + ++rx_ring->rx_stats.bad_desc_num; return 0; } @@ -2129,6 +2108,11 @@ static uint16_t eth_ena_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, /* fill mbuf attributes if any */ ena_rx_mbuf_prepare(mbuf_head, &ena_rx_ctx); + + if (unlikely(mbuf_head->ol_flags & + (PKT_RX_IP_CKSUM_BAD | PKT_RX_L4_CKSUM_BAD))) + ++rx_ring->rx_stats.bad_csum; + mbuf_head->hash.rss = ena_rx_ctx.hash; /* pass to DPDK application head mbuf */ @@ -2259,9 +2243,14 @@ static int ena_check_and_linearize_mbuf(struct ena_ring *tx_ring, (header_len < tx_ring->tx_max_header_size)) return 0; + ++tx_ring->tx_stats.linearize; rc = rte_pktmbuf_linearize(mbuf); - if (unlikely(rc)) + if (unlikely(rc)) { RTE_LOG(WARNING, PMD, "Mbuf linearize failed\n"); + rte_atomic64_inc(&tx_ring->adapter->drv_stats->ierrors); + ++tx_ring->tx_stats.linearize_failed; + return rc; + } return rc; } @@ -2399,15 +2388,18 @@ static uint16_t eth_ena_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, /* prepare the packet's descriptors to dma engine */ rc = ena_com_prepare_tx(tx_ring->ena_com_io_sq, &ena_tx_ctx, &nb_hw_desc); - if (unlikely(rc)) + if (unlikely(rc)) { + ++tx_ring->tx_stats.prepare_ctx_err; break; - + } tx_info->tx_descs = nb_hw_desc; next_to_use++; tx_ring->tx_stats.cnt += tx_info->num_of_bufs; tx_ring->tx_stats.bytes += total_length; } + tx_ring->tx_stats.available_desc = + ena_com_free_desc(tx_ring->ena_com_io_sq); /* If there are ready packets to be xmitted... */ if (sent_idx > 0) { @@ -2441,6 +2433,8 @@ static uint16_t eth_ena_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, if (unlikely(total_tx_descs > ENA_RING_DESCS_RATIO(ring_size))) break; } + tx_ring->tx_stats.available_desc = + ena_com_free_desc(tx_ring->ena_com_io_sq); if (total_tx_descs > 0) { /* acknowledge completion of sent packets */ @@ -2448,9 +2442,167 @@ static uint16_t eth_ena_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, tx_ring->next_to_clean = next_to_clean; } + tx_ring->tx_stats.tx_poll++; + return sent_idx; } +/** + * DPDK callback to retrieve names of extended device statistics + * + * @param dev + * Pointer to Ethernet device structure. + * @param[out] xstats_names + * Buffer to insert names into. + * @param n + * Number of names. + * + * @return + * Number of xstats names. + */ +static int ena_xstats_get_names(struct rte_eth_dev *dev, + struct rte_eth_xstat_name *xstats_names, + unsigned int n) +{ + unsigned int xstats_count = ena_xstats_calc_num(dev); + unsigned int stat, i, count = 0; + + if (n < xstats_count || !xstats_names) + return xstats_count; + + for (stat = 0; stat < ENA_STATS_ARRAY_GLOBAL; stat++, count++) + strcpy(xstats_names[count].name, + ena_stats_global_strings[stat].name); + + for (stat = 0; stat < ENA_STATS_ARRAY_RX; stat++) + for (i = 0; i < dev->data->nb_rx_queues; i++, count++) + snprintf(xstats_names[count].name, + sizeof(xstats_names[count].name), + "rx_q%d_%s", i, + ena_stats_rx_strings[stat].name); + + for (stat = 0; stat < ENA_STATS_ARRAY_TX; stat++) + for (i = 0; i < dev->data->nb_tx_queues; i++, count++) + snprintf(xstats_names[count].name, + sizeof(xstats_names[count].name), + "tx_q%d_%s", i, + ena_stats_tx_strings[stat].name); + + return xstats_count; +} + +/** + * DPDK callback to get extended device statistics. + * + * @param dev + * Pointer to Ethernet device structure. + * @param[out] stats + * Stats table output buffer. + * @param n + * The size of the stats table. + * + * @return + * Number of xstats on success, negative on failure. + */ +static int ena_xstats_get(struct rte_eth_dev *dev, + struct rte_eth_xstat *xstats, + unsigned int n) +{ + struct ena_adapter *adapter = + (struct ena_adapter *)(dev->data->dev_private); + unsigned int xstats_count = ena_xstats_calc_num(dev); + unsigned int stat, i, count = 0; + int stat_offset; + void *stats_begin; + + if (n < xstats_count) + return xstats_count; + + if (!xstats) + return 0; + + for (stat = 0; stat < ENA_STATS_ARRAY_GLOBAL; stat++, count++) { + stat_offset = ena_stats_rx_strings[stat].stat_offset; + stats_begin = &adapter->dev_stats; + + xstats[count].id = count; + xstats[count].value = *((uint64_t *) + ((char *)stats_begin + stat_offset)); + } + + for (stat = 0; stat < ENA_STATS_ARRAY_RX; stat++) { + for (i = 0; i < dev->data->nb_rx_queues; i++, count++) { + stat_offset = ena_stats_rx_strings[stat].stat_offset; + stats_begin = &adapter->rx_ring[i].rx_stats; + + xstats[count].id = count; + xstats[count].value = *((uint64_t *) + ((char *)stats_begin + stat_offset)); + } + } + + for (stat = 0; stat < ENA_STATS_ARRAY_TX; stat++) { + for (i = 0; i < dev->data->nb_tx_queues; i++, count++) { + stat_offset = ena_stats_tx_strings[stat].stat_offset; + stats_begin = &adapter->tx_ring[i].rx_stats; + + xstats[count].id = count; + xstats[count].value = *((uint64_t *) + ((char *)stats_begin + stat_offset)); + } + } + + return count; +} + +static int ena_xstats_get_by_id(struct rte_eth_dev *dev, + const uint64_t *ids, + uint64_t *values, + unsigned int n) +{ + struct ena_adapter *adapter = + (struct ena_adapter *)(dev->data->dev_private); + uint64_t id; + uint64_t rx_entries, tx_entries; + unsigned int i; + int qid; + int valid = 0; + for (i = 0; i < n; ++i) { + id = ids[i]; + /* Check if id belongs to global statistics */ + if (id < ENA_STATS_ARRAY_GLOBAL) { + values[i] = *((uint64_t *)&adapter->dev_stats + id); + ++valid; + continue; + } + + /* Check if id belongs to rx queue statistics */ + id -= ENA_STATS_ARRAY_GLOBAL; + rx_entries = ENA_STATS_ARRAY_RX * dev->data->nb_rx_queues; + if (id < rx_entries) { + qid = id % dev->data->nb_rx_queues; + id /= dev->data->nb_rx_queues; + values[i] = *((uint64_t *) + &adapter->rx_ring[qid].rx_stats + id); + ++valid; + continue; + } + /* Check if id belongs to rx queue statistics */ + id -= rx_entries; + tx_entries = ENA_STATS_ARRAY_TX * dev->data->nb_tx_queues; + if (id < tx_entries) { + qid = id % dev->data->nb_tx_queues; + id /= dev->data->nb_tx_queues; + values[i] = *((uint64_t *) + &adapter->tx_ring[qid].tx_stats + id); + ++valid; + continue; + } + } + + return valid; +} + /********************************************************************* * PMD configuration *********************************************************************/ diff --git a/drivers/net/ena/ena_ethdev.h b/drivers/net/ena/ena_ethdev.h index 255e1c0ec..9eca6e63b 100644 --- a/drivers/net/ena/ena_ethdev.h +++ b/drivers/net/ena/ena_ethdev.h @@ -82,28 +82,22 @@ struct ena_calc_queue_size_ctx { struct ena_stats_tx { u64 cnt; u64 bytes; - u64 queue_stop; u64 prepare_ctx_err; - u64 queue_wakeup; - u64 dma_mapping_err; u64 linearize; u64 linearize_failed; u64 tx_poll; u64 doorbells; - u64 missing_tx_comp; u64 bad_req_id; + u64 available_desc; }; struct ena_stats_rx { u64 cnt; u64 bytes; - u64 refil_partial; + u64 refill_partial; u64 bad_csum; - u64 page_alloc_fail; - u64 skb_alloc_fail; - u64 dma_mapping_err; + u64 mbuf_alloc_fail; u64 bad_desc_num; - u64 small_copy_len_pkt; u64 bad_req_id; }; @@ -167,13 +161,9 @@ struct ena_driver_stats { }; struct ena_stats_dev { - u64 tx_timeout; - u64 io_suspend; - u64 io_resume; u64 wd_expired; - u64 interface_up; - u64 interface_down; - u64 admin_q_pause; + u64 dev_start; + u64 dev_stop; }; /* board specific private data structure */ @@ -222,6 +212,8 @@ struct ena_adapter { uint64_t timestamp_wd; uint64_t keep_alive_timeout; + struct ena_stats_dev dev_stats; + bool trigger_reset; bool wd_state; From patchwork Fri Dec 14 13:18:43 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michal Krawczyk X-Patchwork-Id: 48875 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 6EF381BC1E; Fri, 14 Dec 2018 14:19:40 +0100 (CET) Received: from mail-lj1-f193.google.com (mail-lj1-f193.google.com [209.85.208.193]) by dpdk.org (Postfix) with ESMTP id 9CA821BB6F for ; Fri, 14 Dec 2018 14:19:20 +0100 (CET) Received: by mail-lj1-f193.google.com with SMTP id s5-v6so4846871ljd.12 for ; Fri, 14 Dec 2018 05:19:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=semihalf-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=Pi8SLZKHzKandWenyDGvOb2HK3j3GpKY+B3mTtl4z+w=; b=kbpT6Gtfqk7qg2Ph0ohQBUUD60/WyNjp1mXk+jBHEF3AGhpxXUaeDPGHnr0tUpSsTy a83swMZeaULbMS2po+IWAb5KqQOJQgjoZv93nKrNGKT/Oyvq0LwUdX8hY0Dr9d5IeV2B z2z6pxoSl50o41QeEScFA5ojRK27ATk5fh/UbUju1rJDLe5Fq6ESnchfSBifPXugXoFY RaprBF1Zu+fH11qunWXraKmOuBeujFQytv/OWvHazKqPec7DgCPMW+2rKuAW4sEoAqYR EFCOd5W5Su6iNxTMbJskZ9n9+JMH9xKvi2oJGLTo71fNTa7tAvB/Dajjeyb4YdWZxnE5 Naxw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=Pi8SLZKHzKandWenyDGvOb2HK3j3GpKY+B3mTtl4z+w=; b=fHTD5SbIxqg/g6EujfATX1j1W3WNLRIz9zHzL1pmDCfmXtuEMo2pGgM7Zbm9TCt8GP 8nh/hwdw3z/mU8xg90TK0NCwRfcEPIP5xE532Yc1kvDTBmfwuwqQ255/wuAuAAOwoiMQ qnOWWs20LOXSdqN/hx+fybu7bJqVLknJD8eD3c8WIlBVQYj2CNqkVR0z0yjtvMXZ5SaK a0BST7y9WzMULauaAqXgShNWdPT8pUzUW/RTUN1dj07CZOwiXC6loi5wlFwV6sOvv3Ty KXrp39eiUElJItylyS4REl6jh/T1rB75aoTKzuFQ/P0owtJr4ibvuJkmSxeIXIk9drr3 2vlw== X-Gm-Message-State: AA+aEWa7WqxvG0zS8+c91p9cpDjXCifiR6SndPb+E04TfFB7HRyKBFlE iQgEAvj7F/Gwc4qlCia7Q9r/Ic2DmC4= X-Google-Smtp-Source: AFSGD/UeM2emdGydZ3cw6CwnnNUT1BLRmmyBJ6MWssLaEtkhXg20ENQ+pBknOA2oHV2UiLh8WNrtZw== X-Received: by 2002:a2e:2281:: with SMTP id i123-v6mr1636762lji.154.1544793559852; Fri, 14 Dec 2018 05:19:19 -0800 (PST) Received: from mkPC.semihalf.local (31-172-191-173.noc.fibertech.net.pl. [31.172.191.173]) by smtp.gmail.com with ESMTPSA id o25sm873884lfd.29.2018.12.14.05.19.17 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 14 Dec 2018 05:19:18 -0800 (PST) From: Michal Krawczyk To: dev@dpdk.org Cc: gtzalik@dpdk.org, mw@dpdk.org, matua@amazon.com, rk@semihalf.com Date: Fri, 14 Dec 2018 14:18:43 +0100 Message-Id: <20181214131846.22439-18-mk@semihalf.com> X-Mailer: git-send-email 2.14.1 In-Reply-To: <20181214131846.22439-1-mk@semihalf.com> References: <20181214131846.22439-1-mk@semihalf.com> Subject: [dpdk-dev] [PATCH 17/20] net/ena: new way of getting rx drops X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Rafal Kozik The Rx drops cannot be acquired using the older API. Now, it must be read in keep alive message. Signed-off-by: Rafal Kozik Acked-by: Michal Krawczyk --- drivers/net/ena/ena_ethdev.c | 10 ++++++++-- drivers/net/ena/ena_ethdev.h | 1 + 2 files changed, 9 insertions(+), 2 deletions(-) diff --git a/drivers/net/ena/ena_ethdev.c b/drivers/net/ena/ena_ethdev.c index c3a26d402..2b84cc79d 100644 --- a/drivers/net/ena/ena_ethdev.c +++ b/drivers/net/ena/ena_ethdev.c @@ -933,6 +933,7 @@ static void ena_stats_restart(struct rte_eth_dev *dev) rte_atomic64_init(&adapter->drv_stats->ierrors); rte_atomic64_init(&adapter->drv_stats->oerrors); rte_atomic64_init(&adapter->drv_stats->rx_nombuf); + rte_atomic64_init(&adapter->drv_stats->rx_drops); } static int ena_stats_get(struct rte_eth_dev *dev, @@ -965,10 +966,9 @@ static int ena_stats_get(struct rte_eth_dev *dev, ena_stats.rx_bytes_low); stats->obytes = __MERGE_64B_H_L(ena_stats.tx_bytes_high, ena_stats.tx_bytes_low); - stats->imissed = __MERGE_64B_H_L(ena_stats.rx_drops_high, - ena_stats.rx_drops_low); /* Driver related stats */ + stats->imissed = rte_atomic64_read(&adapter->drv_stats->rx_drops); stats->ierrors = rte_atomic64_read(&adapter->drv_stats->ierrors); stats->oerrors = rte_atomic64_read(&adapter->drv_stats->oerrors); stats->rx_nombuf = rte_atomic64_read(&adapter->drv_stats->rx_nombuf); @@ -2689,8 +2689,14 @@ static void ena_keep_alive(void *adapter_data, __rte_unused struct ena_admin_aenq_entry *aenq_e) { struct ena_adapter *adapter = (struct ena_adapter *)adapter_data; + struct ena_admin_aenq_keep_alive_desc *desc; + uint64_t rx_drops; adapter->timestamp_wd = rte_get_timer_cycles(); + + desc = (struct ena_admin_aenq_keep_alive_desc *)aenq_e; + rx_drops = ((uint64_t)desc->rx_drops_high << 32) | desc->rx_drops_low; + rte_atomic64_set(&adapter->drv_stats->rx_drops, rx_drops); } /** diff --git a/drivers/net/ena/ena_ethdev.h b/drivers/net/ena/ena_ethdev.h index 9eca6e63b..309b92f65 100644 --- a/drivers/net/ena/ena_ethdev.h +++ b/drivers/net/ena/ena_ethdev.h @@ -158,6 +158,7 @@ struct ena_driver_stats { rte_atomic64_t ierrors; rte_atomic64_t oerrors; rte_atomic64_t rx_nombuf; + rte_atomic64_t rx_drops; }; struct ena_stats_dev { From patchwork Fri Dec 14 13:18:44 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michal Krawczyk X-Patchwork-Id: 48876 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id B2B891BC6F; Fri, 14 Dec 2018 14:19:42 +0100 (CET) Received: from mail-lf1-f66.google.com (mail-lf1-f66.google.com [209.85.167.66]) by dpdk.org (Postfix) with ESMTP id B46151BB6F for ; Fri, 14 Dec 2018 14:19:21 +0100 (CET) Received: by mail-lf1-f66.google.com with SMTP id z13so4208605lfe.11 for ; Fri, 14 Dec 2018 05:19:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=semihalf-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=1YHLFG7lWUwpcmncfPR5dF9Zm66KjEAUP9Bpv+KhUpM=; b=nwqFJlgfpTuoomocYMTMJ3qs8c14Qux0BFDIyBQZHdtgfhJrblx1ZMeJDKzORgGsqo EMsPQRi/uc++J3jekBtqBWfcDhqFZcU3Bv1JKJv6aYYlQg2X1Vgb1q+Jw8fUcTw2Pj6J F2PeO9nCcWAPcadJDeh42FFtalKvl5UoBStAbt/cTt4z+Bj2cXcjnv3/bWcfH1Xmq69W epiERJlTqPNDd8U4ggarANWuVIxWZf4q/QLOA75QUBbvqh6Nry9XOCqMLySGUNE0YFDF jWxfY+ytJCam9IEmaI2l5iBzzzxeI7I6fiJGjZ6RAo4OsU1EaJ+U1f4LgEPUuoSyAz4B JlVw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=1YHLFG7lWUwpcmncfPR5dF9Zm66KjEAUP9Bpv+KhUpM=; b=TzA2JwNIsXL40KEft36hloXYAmfjClllTv+m/4s5M7z8VMcMV8GTLtqmdaKlwx1lzb ZGvuob+vJkJvqO6mdDsnNvkrdGbn5pR2xIDN6LHXu1Evny7SYnb8F7PBffRAx7uo0XRU IZ5aV1F64/yzBDxSaYYjslHUTejyoARlFvKw8SMirtisXiytggPBi6WDkVh6BfgKoZYv aOsj9PJrUYkVj0BoGSj7nSRST45MTto6oe5Ey/LW7JJSz0Ivq6QhcG8n4By6M8sUzKXw V/YK5qHBEbYLjfMd2rQrEihZeOlZ57w988KIt3qChGMAD908oqKG9pLhqE2ci00hNs4/ 2Xpg== X-Gm-Message-State: AA+aEWYnw+Zkq9yyNLPv8hHkPgMwfc44a6zc7r8Lk3s5udZQNOsGU7JV AOIszpZW0ytNhFdPhlDU+jahcCoWWls= X-Google-Smtp-Source: AFSGD/V2EZZNd5SLIDisXJm9GjrPCBzfQDYkljkG6wE8o1KO67FKHtcyZLd4CiMRzK3oKIiBLkNmtA== X-Received: by 2002:a19:200b:: with SMTP id g11mr1772830lfg.58.1544793561066; Fri, 14 Dec 2018 05:19:21 -0800 (PST) Received: from mkPC.semihalf.local (31-172-191-173.noc.fibertech.net.pl. [31.172.191.173]) by smtp.gmail.com with ESMTPSA id o25sm873884lfd.29.2018.12.14.05.19.19 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 14 Dec 2018 05:19:20 -0800 (PST) From: Michal Krawczyk To: dev@dpdk.org Cc: gtzalik@dpdk.org, mw@dpdk.org, matua@amazon.com, rk@semihalf.com Date: Fri, 14 Dec 2018 14:18:44 +0100 Message-Id: <20181214131846.22439-19-mk@semihalf.com> X-Mailer: git-send-email 2.14.1 In-Reply-To: <20181214131846.22439-1-mk@semihalf.com> References: <20181214131846.22439-1-mk@semihalf.com> Subject: [dpdk-dev] [PATCH 18/20] net/ena: update completion queue after cleanup X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Rafal Kozik After Rx or Tx cleanup update completion queue head by calling ena_com_update_dev_comp_head(). Fixes: 1daff5260ff8 ("net/ena: use unmasked head and tail") Signed-off-by: Rafal Kozik Acked-by: Michal Krawczyk --- drivers/net/ena/ena_ethdev.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/drivers/net/ena/ena_ethdev.c b/drivers/net/ena/ena_ethdev.c index 2b84cc79d..b336102ba 100644 --- a/drivers/net/ena/ena_ethdev.c +++ b/drivers/net/ena/ena_ethdev.c @@ -2126,8 +2126,10 @@ static uint16_t eth_ena_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, desc_in_use = desc_in_use - completed + 1; /* Burst refill to save doorbells, memory barriers, const interval */ - if (ring_size - desc_in_use > ENA_RING_DESCS_RATIO(ring_size)) + if (ring_size - desc_in_use > ENA_RING_DESCS_RATIO(ring_size)) { + ena_com_update_dev_comp_head(rx_ring->ena_com_io_cq); ena_populate_rx_queue(rx_ring, ring_size - desc_in_use); + } return recv_idx; } @@ -2438,8 +2440,9 @@ static uint16_t eth_ena_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, if (total_tx_descs > 0) { /* acknowledge completion of sent packets */ - ena_com_comp_ack(tx_ring->ena_com_io_sq, total_tx_descs); tx_ring->next_to_clean = next_to_clean; + ena_com_comp_ack(tx_ring->ena_com_io_sq, total_tx_descs); + ena_com_update_dev_comp_head(tx_ring->ena_com_io_cq); } tx_ring->tx_stats.tx_poll++; From patchwork Fri Dec 14 13:18:45 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michal Krawczyk X-Patchwork-Id: 48877 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 2949F1BC8A; Fri, 14 Dec 2018 14:19:44 +0100 (CET) Received: from mail-lj1-f196.google.com (mail-lj1-f196.google.com [209.85.208.196]) by dpdk.org (Postfix) with ESMTP id C3A691BBD1 for ; Fri, 14 Dec 2018 14:19:23 +0100 (CET) Received: by mail-lj1-f196.google.com with SMTP id x85-v6so4894406ljb.2 for ; Fri, 14 Dec 2018 05:19:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=semihalf-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=KDm0H/W/Wao7rq4zt7nzx2k1NJs1dGZAiqTlePR+FQo=; b=wI6nY4PVp4v1iauWBsWJ/kopStgqnJhpV4fwr/DcSF7kw90ju1Q6fiAKBfCorO9aAR /FhPRVqaRKYawZ2/QS5ti5poLNfz2shU8TQ1kLEoIHfVb457+cIeNcN4MqjpmXSng2gi jP/SMCsjvXh2JGEPbvB0gwTQ9+6pmOXpyrQMRgs7ohMjfKmTptJRLtiyuzhuxjdhyWZM oXWsHUyr7EVJ+kCqkVO7t1Su4F3b/8CSS6R1N4nuvi/Yh6pQjlP8tEZ8pbV+GuLkk8k/ sOzRoVnl5AT/aw20koHY4+emM8AH4++GsVowt6r8rg8UjpuUJPQt7ovykTCqbNQs0plg 9ECA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=KDm0H/W/Wao7rq4zt7nzx2k1NJs1dGZAiqTlePR+FQo=; b=JC7uQV2Ekvl89sJt8zrr2m40IpKiZi7cx76gvbuArtMSD9EpgpDZpQACN/jDpDdWbQ G2MREpy4gMTJiwMOgvy9UY9iK1Qy2wqmy3cSiejLdpfjb+mXiU22Hn3EQgPGOKMykN3H EJGSgSXJQ64cl4rGT/harW05nl3fTZ0HWW93/QSA9ND5N/9xHRThIqrE0+ibmCAk//qL oiNke8qXJw5hv+iXqiujW51Ra/jPuPCcvKsR4ISXQMYa/dhWNCLrlq0h/xfbpoNAQlIV IqEtLVd8k6io4NLurU0w2OpyGH/ugufrwVsBpZ+LRuY6K4f4xjC0xvb3WJsyY9UqThSL 8OKA== X-Gm-Message-State: AA+aEWb9fYUKehS1gHyUm74iHH4R6qmbZifHogIEr/nXbrPoGSppdtzS ePfUuGwJju0+H/GrzhrGAnfRuwiiClU= X-Google-Smtp-Source: AFSGD/Xhg+JQfve+OVrlGNNFSq6NIfbuOI/2AOmZJx6o1RnHu7jWA75z8/Tj84WUyQ6OXGdXaHzSaQ== X-Received: by 2002:a2e:834a:: with SMTP id l10-v6mr1808300ljh.42.1544793562960; Fri, 14 Dec 2018 05:19:22 -0800 (PST) Received: from mkPC.semihalf.local (31-172-191-173.noc.fibertech.net.pl. [31.172.191.173]) by smtp.gmail.com with ESMTPSA id o25sm873884lfd.29.2018.12.14.05.19.21 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 14 Dec 2018 05:19:21 -0800 (PST) From: Michal Krawczyk To: dev@dpdk.org Cc: gtzalik@dpdk.org, mw@dpdk.org, matua@amazon.com, rk@semihalf.com, Michal Krawczyk Date: Fri, 14 Dec 2018 14:18:45 +0100 Message-Id: <20181214131846.22439-20-mk@semihalf.com> X-Mailer: git-send-email 2.14.1 In-Reply-To: <20181214131846.22439-1-mk@semihalf.com> References: <20181214131846.22439-1-mk@semihalf.com> Subject: [dpdk-dev] [PATCH 19/20] docs: update ENA PMD documentation and features X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Previously, the features list was indicating unsupported ENA PMD features and were missing few ones, that were actually supported. The features file was updated, so it is now reflecting current driver state. The documentation was updated with the more actual example and features, especially ones which are ENA and not listed in the features file. Signed-off-by: Michal Krawczyk --- doc/guides/nics/ena.rst | 31 ++++++++++++++++--------------- doc/guides/nics/features/ena.ini | 12 ++++-------- 2 files changed, 20 insertions(+), 23 deletions(-) diff --git a/doc/guides/nics/ena.rst b/doc/guides/nics/ena.rst index 34c48575b..9f0a6509f 100644 --- a/doc/guides/nics/ena.rst +++ b/doc/guides/nics/ena.rst @@ -1,6 +1,6 @@ .. BSD LICENSE - Copyright (c) 2015-2016 Amazon.com, Inc. or its affiliates. + Copyright (c) 2015-2018 Amazon.com, Inc. or its affiliates. All rights reserved. Redistribution and use in source and binary forms, with or without @@ -164,20 +164,18 @@ section of :ref:`the DPDK documentation ` or refer to *DPDK Release N Supported features ------------------ +* MTU configuration * Jumbo frames up to 9K -* Port Hardware Statistics * IPv4/TCP/UDP checksum offload * TSO offload * Multiple receive and transmit queues -* RSS +* RSS hash +* RSS indirection table configuration * Low Latency Queue for Tx - -Unsupported features --------------------- - -The features supported by the device and not yet supported by this PMD include: - -* Asynchronous Event Notification Queue (AENQ) +* Basic and extended statistics +* LSC event notification +* Watchdog (requires handling of timers in the application) +* Device reset upon failure Prerequisites ------------- @@ -214,14 +212,17 @@ Example output: .. code-block:: console [...] - EAL: PCI device 0000:02:00.1 on NUMA socket -1 - EAL: probe driver: 1d0f:ec20 rte_ena_pmd - EAL: PCI memory mapped at 0x7f9b6c400000 - PMD: eth_ena_dev_init(): Initializing 0:2:0.1 + EAL: PCI device 0000:00:06.0 on NUMA socket -1 + EAL: Invalid NUMA socket, default to 0 + EAL: probe driver: 1d0f:ec20 net_ena + Interactive-mode selected + testpmd: create a new mbuf pool : n=171456, size=2176, socket=0 + testpmd: preferred mempool ops selected: ring_mp_mc + Warning! port-topology=paired and odd forward ports number, the last port will pair with itself. Configuring Port 0 (socket 0) Port 0: 00:00:00:11:00:01 Checking link statuses... - Port 0 Link Up - speed 10000 Mbps - full-duplex + Done testpmd> diff --git a/doc/guides/nics/features/ena.ini b/doc/guides/nics/features/ena.ini index aa6f05a71..4300dd012 100644 --- a/doc/guides/nics/features/ena.ini +++ b/doc/guides/nics/features/ena.ini @@ -4,25 +4,21 @@ ; Refer to default.ini for the full list of available PMD features. ; [Features] -Speed capabilities = P -Queue start/stop = Y +Link status = Y +Link status event = Y MTU update = Y Jumbo frame = Y Scattered Rx = Y TSO = Y RSS hash = Y -RSS key update = Y RSS reta update = Y -SR-IOV = Y -CRC offload = Y -VLAN offload = Y L3 checksum offload = Y L4 checksum offload = Y -Inner L3 checksum = Y -Inner L4 checksum = Y Basic stats = Y Extended stats = Y Linux UIO = Y Linux VFIO = Y x86-32 = Y x86-64 = Y +Usage doc = Y +Design doc = Y From patchwork Fri Dec 14 13:18:46 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michal Krawczyk X-Patchwork-Id: 48878 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 692341BC15; Fri, 14 Dec 2018 14:19:45 +0100 (CET) Received: from mail-lj1-f195.google.com (mail-lj1-f195.google.com [209.85.208.195]) by dpdk.org (Postfix) with ESMTP id 095B61BBD1 for ; Fri, 14 Dec 2018 14:19:24 +0100 (CET) Received: by mail-lj1-f195.google.com with SMTP id e5-v6so4878333lja.4 for ; Fri, 14 Dec 2018 05:19:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=semihalf-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=isZOwDkti5VGYwBG5C/Qw9PnGUvXYXNeyO9Me7QkaOg=; b=fgqnmNTaNhSR1LOOAAgROh3uR0KJKL+63YD8k7OuK7k0vkskTg5F4tdRAqZg5fSRda vtO9sIDXjtKb7NLq2B0B0cvz16EZVTV2rR11Ty7QGekcZR8YBX7nDPOnPer3xCJQNNsy l9pr+CJDEuWgeWGm6YBEMnD+MRxjouzUqS0M/728efRNmL5U6afpTRpTAFMxP0QRgD4Q vqUcvgPFoUFZV0Y9fdalDqZGqNFUwQMlRvdy8GY6XU94x0occMtMwVrRF2lBYzAMEtDN ktRMmUVJhauTumij9tGOsj1Aj5lAWeDn7/UhD2Tl+25tytZynzaf3s2Be5avNaylC1sA 8JFw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=isZOwDkti5VGYwBG5C/Qw9PnGUvXYXNeyO9Me7QkaOg=; b=fE96GVyKk9B58lPVWLw1Peo9uGY9n5065Y03Z9ZXmy66hma3RNZd/N5M/5AMAk6Ewx oVxWPeRU7H7rrs2HSfMyUHzDsDU5NZS2wygyAlXHJ76fyS7JhPboHTiFrVeO1im8f/to ogSOK+GxaK0wT9lNroJb8xjYsSH52RE6YU9pWLHehVvqzNvdPmYnRF4l+X5mpXZWxx1N JeIuQ7r2pYel14Z25ZciLBJvGLsE+uua6TlVgvOLRbM0OK2D+i2+78Sn+1TTCzYqv7Nc dp9W26885egx9P3rbycz3NLZbQijxKC6Mv9ecFVrdJuvZFMYJZ0s1I+xqOAsnkKsRg/G PWvA== X-Gm-Message-State: AA+aEWY7cVwIvhBPxBGqOAD+IlU16dfVfKE14Hi2y2HX/sDTfqOW3dbB hwQVKo811c/9HM89QyDPcnuidqK4tN8= X-Google-Smtp-Source: AFSGD/WtUiD7Xydm3BPejmAX/E8sL4kQajQZOv270cAHbgfOJNCz5nld2tMgWQ2bszaZF4wTiTRWzg== X-Received: by 2002:a2e:8256:: with SMTP id j22-v6mr1853755ljh.40.1544793564100; Fri, 14 Dec 2018 05:19:24 -0800 (PST) Received: from mkPC.semihalf.local (31-172-191-173.noc.fibertech.net.pl. [31.172.191.173]) by smtp.gmail.com with ESMTPSA id o25sm873884lfd.29.2018.12.14.05.19.22 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 14 Dec 2018 05:19:23 -0800 (PST) From: Michal Krawczyk To: dev@dpdk.org Cc: gtzalik@dpdk.org, mw@dpdk.org, matua@amazon.com, rk@semihalf.com, Michal Krawczyk Date: Fri, 14 Dec 2018 14:18:46 +0100 Message-Id: <20181214131846.22439-21-mk@semihalf.com> X-Mailer: git-send-email 2.14.1 In-Reply-To: <20181214131846.22439-1-mk@semihalf.com> References: <20181214131846.22439-1-mk@semihalf.com> Subject: [dpdk-dev] [PATCH 20/20] net/ena: release of the ENAv2 driver (v2.0.0) X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" The ENAv2 is introducing many new features, mainly the LLQ feature (Low Latency Queue) which allows the device to process packets faster and as a result, the latency is noticeably lower. The second major feature is configurable depth of hw queues where Rx and Tx can be reconfigured independently and maximum depth of Rx queue is 8k. The release also includes many bug fixes and minor new features, like improved statistics counters and extended statistics. Driver is still compatible with ENAv1 device. Signed-off-by: Michal Krawczyk --- drivers/net/ena/ena_ethdev.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/drivers/net/ena/ena_ethdev.c b/drivers/net/ena/ena_ethdev.c index b336102ba..eba3e4c16 100644 --- a/drivers/net/ena/ena_ethdev.c +++ b/drivers/net/ena/ena_ethdev.c @@ -53,9 +53,9 @@ #include #include -#define DRV_MODULE_VER_MAJOR 1 -#define DRV_MODULE_VER_MINOR 1 -#define DRV_MODULE_VER_SUBMINOR 1 +#define DRV_MODULE_VER_MAJOR 2 +#define DRV_MODULE_VER_MINOR 0 +#define DRV_MODULE_VER_SUBMINOR 0 #define ENA_IO_TXQ_IDX(q) (2 * (q)) #define ENA_IO_RXQ_IDX(q) (2 * (q) + 1)