From patchwork Tue Feb 4 13:58:32 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Lukas Bartosik [C]" X-Patchwork-Id: 65548 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id BA47DA0534; Tue, 4 Feb 2020 14:59:23 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 788DB1BFAC; Tue, 4 Feb 2020 14:59:04 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by dpdk.org (Postfix) with ESMTP id 3610EA69 for ; Tue, 4 Feb 2020 14:59:03 +0100 (CET) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 014DtBwq012233; Tue, 4 Feb 2020 05:59:02 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=pfpt0818; bh=9KKAYRoksuPolZiv0qTKs+rk3/L9Udw6b+/lha7nIqM=; b=ldKBBkY1nYk1fYuJPcdG0lpVCbxnxfuc3srNgtanxP/mut3h3C9yenTI9NbPrAxfiscL egLEwDcLPKMe6bdfrEGi564zKT8CMzY8xX1UzRd/Eb9Dema6+hQMBTPP+B5HWSvVzwsS nxaejDRaZzcxgMxPkb0t9rnRcMgeNJUZ7Sx3Wc2jcMgSQTZF1t652bl1cNEiY1iuBWXK vUxKya7o4K6GAeZ2M6kynAtYvd/66UNWBHPLcThxZPjhtQDzQU9TS6VhA3wx2UxHVRj2 w6mVSUpnAV0vUTG8qbMgMTV2RmYrE3YbzdHjNZCGBlObJR1cuO+UDN85jcI1Q+oQPk5c eQ== Received: from sc-exch04.marvell.com ([199.233.58.184]) by mx0b-0016f401.pphosted.com with ESMTP id 2xw9qum088-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Tue, 04 Feb 2020 05:59:02 -0800 Received: from SC-EXCH03.marvell.com (10.93.176.83) by SC-EXCH04.marvell.com (10.93.176.84) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Tue, 4 Feb 2020 05:59:00 -0800 Received: from maili.marvell.com (10.93.176.43) by SC-EXCH03.marvell.com (10.93.176.83) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Tue, 4 Feb 2020 05:59:00 -0800 Received: from luke.marvell.com (unknown [10.95.130.32]) by maili.marvell.com (Postfix) with ESMTP id 13CC13F70D6; Tue, 4 Feb 2020 05:58:56 -0800 (PST) From: Lukasz Bartosik To: Akhil Goyal , Radu Nicolau , Thomas Monjalon CC: Anoob Joseph , Jerin Jacob , Narayana Prasad , Ankur Dwivedi , Archana Muniganti , Tejasree Kondoj , Vamsi Attunuru , "Konstantin Ananyev" , Date: Tue, 4 Feb 2020 14:58:32 +0100 Message-ID: <1580824721-21527-5-git-send-email-lbartosik@marvell.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1580824721-21527-1-git-send-email-lbartosik@marvell.com> References: <1579527918-360-1-git-send-email-anoobj@marvell.com> <1580824721-21527-1-git-send-email-lbartosik@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.138, 18.0.572 definitions=2020-02-04_04:2020-02-04, 2020-02-04 signatures=0 Subject: [dpdk-dev] [PATCH v3 04/13] examples/ipsec-secgw: add Rx adapter support X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Anoob Joseph Add Rx adapter support. The event helper init routine will initialize the Rx adapter according to the configuration. If Rx adapter config is not present it will generate a default config. If there are enough event queues available it will map eth ports and event queues 1:1 (one eth port will be connected to one event queue). Otherwise it will map all eth ports to one event queue. Signed-off-by: Anoob Joseph Signed-off-by: Lukasz Bartosik --- examples/ipsec-secgw/event_helper.c | 273 +++++++++++++++++++++++++++++++++++- examples/ipsec-secgw/event_helper.h | 29 ++++ 2 files changed, 301 insertions(+), 1 deletion(-) diff --git a/examples/ipsec-secgw/event_helper.c b/examples/ipsec-secgw/event_helper.c index cf2dff0..1d06a45 100644 --- a/examples/ipsec-secgw/event_helper.c +++ b/examples/ipsec-secgw/event_helper.c @@ -4,10 +4,58 @@ #include #include #include +#include #include +#include #include "event_helper.h" +static int +eh_get_enabled_cores(struct rte_bitmap *eth_core_mask) +{ + int i, count = 0; + + RTE_LCORE_FOREACH(i) { + /* Check if this core is enabled in core mask*/ + if (rte_bitmap_get(eth_core_mask, i)) { + /* Found enabled core */ + count++; + } + } + return count; +} + +static inline unsigned int +eh_get_next_eth_core(struct eventmode_conf *em_conf) +{ + static unsigned int prev_core = -1; + unsigned int next_core; + + /* + * Make sure we have at least one eth core running, else the following + * logic would lead to an infinite loop. + */ + if (eh_get_enabled_cores(em_conf->eth_core_mask) == 0) { + EH_LOG_ERR("No enabled eth core found"); + return RTE_MAX_LCORE; + } + + /* Only some cores are marked as eth cores, skip others */ + do { + /* Get the next core */ + next_core = rte_get_next_lcore(prev_core, 0, 1); + + /* Check if we have reached max lcores */ + if (next_core == RTE_MAX_LCORE) + return next_core; + + /* Update prev_core */ + prev_core = next_core; + } while (!(rte_bitmap_get(em_conf->eth_core_mask, next_core))); + + return next_core; +} + static inline unsigned int eh_get_next_active_core(struct eventmode_conf *em_conf, unsigned int prev_core) { @@ -168,6 +216,82 @@ eh_set_default_conf_link(struct eventmode_conf *em_conf) } static int +eh_set_default_conf_rx_adapter(struct eventmode_conf *em_conf) +{ + struct rx_adapter_connection_info *conn; + struct eventdev_params *eventdev_config; + struct rx_adapter_conf *adapter; + bool single_ev_queue = false; + int eventdev_id; + int nb_eth_dev; + int adapter_id; + int conn_id; + int i; + + /* Create one adapter with eth queues mapped to event queue(s) */ + + if (em_conf->nb_eventdev == 0) { + EH_LOG_ERR("No event devs registered"); + return -EINVAL; + } + + /* Get the number of eth devs */ + nb_eth_dev = rte_eth_dev_count_avail(); + + /* Use the first event dev */ + eventdev_config = &(em_conf->eventdev_config[0]); + + /* Get eventdev ID */ + eventdev_id = eventdev_config->eventdev_id; + adapter_id = 0; + + /* Get adapter conf */ + adapter = &(em_conf->rx_adapter[adapter_id]); + + /* Set adapter conf */ + adapter->eventdev_id = eventdev_id; + adapter->adapter_id = adapter_id; + adapter->rx_core_id = eh_get_next_eth_core(em_conf); + + /* + * Map all queues of eth device (port) to an event queue. If there + * are more event queues than eth ports then create 1:1 mapping. + * Otherwise map all eth ports to a single event queue. + */ + if (nb_eth_dev > eventdev_config->nb_eventqueue) + single_ev_queue = true; + + for (i = 0; i < nb_eth_dev; i++) { + + /* Use only the ports enabled */ + if ((em_conf->eth_portmask & (1 << i)) == 0) + continue; + + /* Get the connection id */ + conn_id = adapter->nb_connections; + + /* Get the connection */ + conn = &(adapter->conn[conn_id]); + + /* Set mapping between eth ports & event queues*/ + conn->ethdev_id = i; + conn->eventq_id = single_ev_queue ? 0 : i; + + /* Add all eth queues eth port to event queue */ + conn->ethdev_rx_qid = -1; + + /* Update no of connections */ + adapter->nb_connections++; + + } + + /* We have setup one adapter */ + em_conf->nb_rx_adapter = 1; + + return 0; +} + +static int eh_validate_conf(struct eventmode_conf *em_conf) { int ret; @@ -192,6 +316,16 @@ eh_validate_conf(struct eventmode_conf *em_conf) return ret; } + /* + * Check if rx adapters are specified. Else generate a default config + * with one rx adapter and all eth queues - event queue mapped. + */ + if (em_conf->nb_rx_adapter == 0) { + ret = eh_set_default_conf_rx_adapter(em_conf); + if (ret != 0) + return ret; + } + return 0; } @@ -347,6 +481,104 @@ eh_initialize_eventdev(struct eventmode_conf *em_conf) return 0; } +static int +eh_rx_adapter_configure(struct eventmode_conf *em_conf, + struct rx_adapter_conf *adapter) +{ + struct rte_event_eth_rx_adapter_queue_conf queue_conf = {0}; + struct rte_event_dev_info evdev_default_conf = {0}; + struct rte_event_port_conf port_conf = {0}; + struct rx_adapter_connection_info *conn; + uint8_t eventdev_id; + uint32_t service_id; + int ret; + int j; + + /* Get event dev ID */ + eventdev_id = adapter->eventdev_id; + + /* Get default configuration of event dev */ + ret = rte_event_dev_info_get(eventdev_id, &evdev_default_conf); + if (ret < 0) { + EH_LOG_ERR("Failed to get event dev info %d", ret); + return ret; + } + + /* Setup port conf */ + port_conf.new_event_threshold = 1200; + port_conf.dequeue_depth = + evdev_default_conf.max_event_port_dequeue_depth; + port_conf.enqueue_depth = + evdev_default_conf.max_event_port_enqueue_depth; + + /* Create Rx adapter */ + ret = rte_event_eth_rx_adapter_create(adapter->adapter_id, + adapter->eventdev_id, &port_conf); + if (ret < 0) { + EH_LOG_ERR("Failed to create rx adapter %d", ret); + return ret; + } + + /* Setup various connections in the adapter */ + for (j = 0; j < adapter->nb_connections; j++) { + /* Get connection */ + conn = &(adapter->conn[j]); + + /* Setup queue conf */ + queue_conf.ev.queue_id = conn->eventq_id; + queue_conf.ev.sched_type = em_conf->ext_params.sched_type; + queue_conf.ev.event_type = RTE_EVENT_TYPE_ETHDEV; + + /* Add queue to the adapter */ + ret = rte_event_eth_rx_adapter_queue_add(adapter->adapter_id, + conn->ethdev_id, conn->ethdev_rx_qid, + &queue_conf); + if (ret < 0) { + EH_LOG_ERR("Failed to add eth queue to rx adapter %d", + ret); + return ret; + } + } + + /* Get the service ID used by rx adapter */ + ret = rte_event_eth_rx_adapter_service_id_get(adapter->adapter_id, + &service_id); + if (ret != -ESRCH && ret < 0) { + EH_LOG_ERR("Failed to get service id used by rx adapter %d", + ret); + return ret; + } + + rte_service_set_runstate_mapped_check(service_id, 0); + + /* Start adapter */ + ret = rte_event_eth_rx_adapter_start(adapter->adapter_id); + if (ret < 0) { + EH_LOG_ERR("Failed to start rx adapter %d", ret); + return ret; + } + + return 0; +} + +static int +eh_initialize_rx_adapter(struct eventmode_conf *em_conf) +{ + struct rx_adapter_conf *adapter; + int i, ret; + + /* Configure rx adapters */ + for (i = 0; i < em_conf->nb_rx_adapter; i++) { + adapter = &(em_conf->rx_adapter[i]); + ret = eh_rx_adapter_configure(em_conf, adapter); + if (ret < 0) { + EH_LOG_ERR("Failed to configure rx adapter %d", ret); + return ret; + } + } + return 0; +} + int32_t eh_devs_init(struct eh_conf *conf) { @@ -370,6 +602,9 @@ eh_devs_init(struct eh_conf *conf) /* Get eventmode conf */ em_conf = conf->mode_params; + /* Eventmode conf would need eth portmask */ + em_conf->eth_portmask = conf->eth_portmask; + /* Validate the requested config */ ret = eh_validate_conf(em_conf); if (ret < 0) { @@ -394,6 +629,13 @@ eh_devs_init(struct eh_conf *conf) return ret; } + /* Setup Rx adapter */ + ret = eh_initialize_rx_adapter(em_conf); + if (ret < 0) { + EH_LOG_ERR("Failed to initialize rx adapter %d", ret); + return ret; + } + /* Start eth devices after setting up adapter */ RTE_ETH_FOREACH_DEV(port_id) { @@ -416,8 +658,8 @@ int32_t eh_devs_uninit(struct eh_conf *conf) { struct eventmode_conf *em_conf; + int ret, i, j; uint16_t id; - int ret, i; if (conf == NULL) { EH_LOG_ERR("Invalid event helper configuration"); @@ -435,6 +677,35 @@ eh_devs_uninit(struct eh_conf *conf) /* Get eventmode conf */ em_conf = conf->mode_params; + /* Stop and release rx adapters */ + for (i = 0; i < em_conf->nb_rx_adapter; i++) { + + id = em_conf->rx_adapter[i].adapter_id; + ret = rte_event_eth_rx_adapter_stop(id); + if (ret < 0) { + EH_LOG_ERR("Failed to stop rx adapter %d", ret); + return ret; + } + + for (j = 0; j < em_conf->rx_adapter[i].nb_connections; j++) { + + ret = rte_event_eth_rx_adapter_queue_del(id, + em_conf->rx_adapter[i].conn[j].ethdev_id, -1); + if (ret < 0) { + EH_LOG_ERR( + "Failed to remove rx adapter queues %d", + ret); + return ret; + } + } + + ret = rte_event_eth_rx_adapter_free(id); + if (ret < 0) { + EH_LOG_ERR("Failed to free rx adapter %d", ret); + return ret; + } + } + /* Stop and release event devices */ for (i = 0; i < em_conf->nb_eventdev; i++) { diff --git a/examples/ipsec-secgw/event_helper.h b/examples/ipsec-secgw/event_helper.h index 16b03b3..baf93e1 100644 --- a/examples/ipsec-secgw/event_helper.h +++ b/examples/ipsec-secgw/event_helper.h @@ -20,6 +20,12 @@ extern "C" { /* Max event devices supported */ #define EVENT_MODE_MAX_EVENT_DEVS RTE_EVENT_MAX_DEVS +/* Max Rx adapters supported */ +#define EVENT_MODE_MAX_RX_ADAPTERS RTE_EVENT_MAX_DEVS + +/* Max Rx adapter connections */ +#define EVENT_MODE_MAX_CONNECTIONS_PER_ADAPTER 16 + /* Max event queues supported per event device */ #define EVENT_MODE_MAX_EVENT_QUEUES_PER_DEV RTE_EVENT_MAX_QUEUES_PER_DEV @@ -57,12 +63,33 @@ struct eh_event_link_info { /**< Lcore to be polling on this port */ }; +/* Rx adapter connection info */ +struct rx_adapter_connection_info { + uint8_t ethdev_id; + uint8_t eventq_id; + int32_t ethdev_rx_qid; +}; + +/* Rx adapter conf */ +struct rx_adapter_conf { + int32_t eventdev_id; + int32_t adapter_id; + uint32_t rx_core_id; + uint8_t nb_connections; + struct rx_adapter_connection_info + conn[EVENT_MODE_MAX_CONNECTIONS_PER_ADAPTER]; +}; + /* Eventmode conf data */ struct eventmode_conf { int nb_eventdev; /**< No of event devs */ struct eventdev_params eventdev_config[EVENT_MODE_MAX_EVENT_DEVS]; /**< Per event dev conf */ + uint8_t nb_rx_adapter; + /**< No of Rx adapters */ + struct rx_adapter_conf rx_adapter[EVENT_MODE_MAX_RX_ADAPTERS]; + /**< Rx adapter conf */ uint8_t nb_link; /**< No of links */ struct eh_event_link_info @@ -70,6 +97,8 @@ struct eventmode_conf { /**< Per link conf */ struct rte_bitmap *eth_core_mask; /**< Core mask of cores to be used for software Rx and Tx */ + uint32_t eth_portmask; + /**< Mask of the eth ports to be used */ union { RTE_STD_C11 struct {