From patchwork Thu Feb 20 08:01:57 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Lukas Bartosik [C]" X-Patchwork-Id: 65945 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 24573A0555; Thu, 20 Feb 2020 09:03:11 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 3B2FA1BF95; Thu, 20 Feb 2020 09:02:33 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by dpdk.org (Postfix) with ESMTP id 6BDC31BFD1 for ; Thu, 20 Feb 2020 09:02:30 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 01K80f50016138; Thu, 20 Feb 2020 00:02:29 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=pfpt0818; bh=1KbWFVd0d50d8jp5eOOJWZ1/PPE1c0hhTariP2mMG+o=; b=wD9bFwo1GSvT2koNj/Z2DwvouvoGlhibv/Zfm/Cm3GzZZo1x8orFZArLF5/RFPi86Q8g khaHl5F/pJV8o62kdp6xIFqwIpeZHY1rVGe3zGNTpx31iC/0s/P+WYUgqOunHP8zu/OB u1SmluV0QE1Tqq9dVH/8tVoI85bqEn2D822psAe0GF0sjUhiUnndcs6gC6LJ1s0CuKsY A4BVHkFzrm960wjHx54vXnb+lNbO/j8mD8ymOKSyw+bGI+CveaXoXDwjnCPfI4zvis+1 rul1qT7U+W4ILNF1yQpm5VTA5LnMQjX+0CjpWk1J9yahXEihpJTuStrS1ZHmHFaU/d07 bg== Received: from sc-exch03.marvell.com ([199.233.58.183]) by mx0a-0016f401.pphosted.com with ESMTP id 2y8ubhefg4-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Thu, 20 Feb 2020 00:02:29 -0800 Received: from SC-EXCH03.marvell.com (10.93.176.83) by SC-EXCH03.marvell.com (10.93.176.83) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Thu, 20 Feb 2020 00:02:27 -0800 Received: from maili.marvell.com (10.93.176.43) by SC-EXCH03.marvell.com (10.93.176.83) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Thu, 20 Feb 2020 00:02:27 -0800 Received: from luke.marvell.com (unknown [10.95.130.24]) by maili.marvell.com (Postfix) with ESMTP id C0C623F703F; Thu, 20 Feb 2020 00:02:24 -0800 (PST) From: Lukasz Bartosik To: Akhil Goyal , Radu Nicolau , Thomas Monjalon CC: Anoob Joseph , Jerin Jacob , Narayana Prasad , Ankur Dwivedi , Archana Muniganti , Tejasree Kondoj , Vamsi Attunuru , "Konstantin Ananyev" , Date: Thu, 20 Feb 2020 09:01:57 +0100 Message-ID: <1582185727-6749-6-git-send-email-lbartosik@marvell.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1582185727-6749-1-git-send-email-lbartosik@marvell.com> References: <1580824721-21527-1-git-send-email-lbartosik@marvell.com> <1582185727-6749-1-git-send-email-lbartosik@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.138, 18.0.572 definitions=2020-02-20_02:2020-02-19, 2020-02-20 signatures=0 Subject: [dpdk-dev] [PATCH v4 05/15] examples/ipsec-secgw: add Tx adapter support X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Anoob Joseph Add Tx adapter support. The event helper init routine will initialize the Tx adapter according to the configuration. If Tx adapter config is not present it will generate a default config. Signed-off-by: Anoob Joseph Signed-off-by: Lukasz Bartosik --- examples/ipsec-secgw/event_helper.c | 313 ++++++++++++++++++++++++++++++++++++ examples/ipsec-secgw/event_helper.h | 48 ++++++ 2 files changed, 361 insertions(+) diff --git a/examples/ipsec-secgw/event_helper.c b/examples/ipsec-secgw/event_helper.c index 2653e86..fca1e08 100644 --- a/examples/ipsec-secgw/event_helper.c +++ b/examples/ipsec-secgw/event_helper.c @@ -5,6 +5,7 @@ #include #include #include +#include #include #include @@ -76,6 +77,22 @@ eh_get_next_active_core(struct eventmode_conf *em_conf, unsigned int prev_core) return next_core; } +static struct eventdev_params * +eh_get_eventdev_params(struct eventmode_conf *em_conf, uint8_t eventdev_id) +{ + int i; + + for (i = 0; i < em_conf->nb_eventdev; i++) { + if (em_conf->eventdev_config[i].eventdev_id == eventdev_id) + break; + } + + /* No match */ + if (i == em_conf->nb_eventdev) + return NULL; + + return &(em_conf->eventdev_config[i]); +} static int eh_set_default_conf_eventdev(struct eventmode_conf *em_conf) { @@ -288,6 +305,95 @@ eh_set_default_conf_rx_adapter(struct eventmode_conf *em_conf) } static int +eh_set_default_conf_tx_adapter(struct eventmode_conf *em_conf) +{ + struct tx_adapter_connection_info *conn; + struct eventdev_params *eventdev_config; + struct tx_adapter_conf *tx_adapter; + int eventdev_id; + int adapter_id; + int nb_eth_dev; + int conn_id; + int i; + + /* + * Create one Tx adapter with all eth queues mapped to event queues + * 1:1. + */ + + if (em_conf->nb_eventdev == 0) { + EH_LOG_ERR("No event devs registered"); + return -EINVAL; + } + + /* Get the number of eth devs */ + nb_eth_dev = rte_eth_dev_count_avail(); + + /* Use the first event dev */ + eventdev_config = &(em_conf->eventdev_config[0]); + + /* Get eventdev ID */ + eventdev_id = eventdev_config->eventdev_id; + adapter_id = 0; + + /* Get adapter conf */ + tx_adapter = &(em_conf->tx_adapter[adapter_id]); + + /* Set adapter conf */ + tx_adapter->eventdev_id = eventdev_id; + tx_adapter->adapter_id = adapter_id; + + /* TODO: Tx core is required only when internal port is not present */ + tx_adapter->tx_core_id = eh_get_next_eth_core(em_conf); + + /* + * Application uses one event queue per adapter for submitting + * packets for Tx. Reserve the last queue available and decrement + * the total available event queues for this + */ + + /* Queue numbers start at 0 */ + tx_adapter->tx_ev_queue = eventdev_config->nb_eventqueue - 1; + + /* + * Map all Tx queues of the eth device (port) to the event device. + */ + + /* Set defaults for connections */ + + /* + * One eth device (port) is one connection. Map all Tx queues + * of the device to the Tx adapter. + */ + + for (i = 0; i < nb_eth_dev; i++) { + + /* Use only the ports enabled */ + if ((em_conf->eth_portmask & (1 << i)) == 0) + continue; + + /* Get the connection id */ + conn_id = tx_adapter->nb_connections; + + /* Get the connection */ + conn = &(tx_adapter->conn[conn_id]); + + /* Add ethdev to connections */ + conn->ethdev_id = i; + + /* Add all eth tx queues to adapter */ + conn->ethdev_tx_qid = -1; + + /* Update no of connections */ + tx_adapter->nb_connections++; + } + + /* We have setup one adapter */ + em_conf->nb_tx_adapter = 1; + return 0; +} + +static int eh_validate_conf(struct eventmode_conf *em_conf) { int ret; @@ -322,6 +428,16 @@ eh_validate_conf(struct eventmode_conf *em_conf) return ret; } + /* + * Check if tx adapters are specified. Else generate a default config + * with one tx adapter. + */ + if (em_conf->nb_tx_adapter == 0) { + ret = eh_set_default_conf_tx_adapter(em_conf); + if (ret != 0) + return ret; + } + return 0; } @@ -573,6 +689,133 @@ eh_initialize_rx_adapter(struct eventmode_conf *em_conf) return 0; } +static int +eh_tx_adapter_configure(struct eventmode_conf *em_conf, + struct tx_adapter_conf *adapter) +{ + struct rte_event_dev_info evdev_default_conf = {0}; + struct rte_event_port_conf port_conf = {0}; + struct tx_adapter_connection_info *conn; + struct eventdev_params *eventdev_config; + uint8_t tx_port_id = 0; + uint8_t eventdev_id; + uint32_t service_id; + int ret, j; + + /* Get event dev ID */ + eventdev_id = adapter->eventdev_id; + + /* Get event device conf */ + eventdev_config = eh_get_eventdev_params(em_conf, eventdev_id); + + /* Create Tx adapter */ + + /* Get default configuration of event dev */ + ret = rte_event_dev_info_get(eventdev_id, &evdev_default_conf); + if (ret < 0) { + EH_LOG_ERR("Failed to get event dev info %d", ret); + return ret; + } + + /* Setup port conf */ + port_conf.new_event_threshold = + evdev_default_conf.max_num_events; + port_conf.dequeue_depth = + evdev_default_conf.max_event_port_dequeue_depth; + port_conf.enqueue_depth = + evdev_default_conf.max_event_port_enqueue_depth; + + /* Create adapter */ + ret = rte_event_eth_tx_adapter_create(adapter->adapter_id, + adapter->eventdev_id, &port_conf); + if (ret < 0) { + EH_LOG_ERR("Failed to create tx adapter %d", ret); + return ret; + } + + /* Setup various connections in the adapter */ + for (j = 0; j < adapter->nb_connections; j++) { + + /* Get connection */ + conn = &(adapter->conn[j]); + + /* Add queue to the adapter */ + ret = rte_event_eth_tx_adapter_queue_add(adapter->adapter_id, + conn->ethdev_id, conn->ethdev_tx_qid); + if (ret < 0) { + EH_LOG_ERR("Failed to add eth queue to tx adapter %d", + ret); + return ret; + } + } + + /* Setup Tx queue & port */ + + /* Get event port used by the adapter */ + ret = rte_event_eth_tx_adapter_event_port_get( + adapter->adapter_id, &tx_port_id); + if (ret) { + EH_LOG_ERR("Failed to get tx adapter port id %d", ret); + return ret; + } + + /* + * Tx event queue is reserved for Tx adapter. Unlink this queue + * from all other ports + * + */ + for (j = 0; j < eventdev_config->nb_eventport; j++) { + rte_event_port_unlink(eventdev_id, j, + &(adapter->tx_ev_queue), 1); + } + + /* Link Tx event queue to Tx port */ + ret = rte_event_port_link(eventdev_id, tx_port_id, + &(adapter->tx_ev_queue), NULL, 1); + if (ret != 1) { + EH_LOG_ERR("Failed to link event queue to port"); + return ret; + } + + /* Get the service ID used by Tx adapter */ + ret = rte_event_eth_tx_adapter_service_id_get(adapter->adapter_id, + &service_id); + if (ret != -ESRCH && ret < 0) { + EH_LOG_ERR("Failed to get service id used by tx adapter %d", + ret); + return ret; + } + + rte_service_set_runstate_mapped_check(service_id, 0); + + /* Start adapter */ + ret = rte_event_eth_tx_adapter_start(adapter->adapter_id); + if (ret < 0) { + EH_LOG_ERR("Failed to start tx adapter %d", ret); + return ret; + } + + return 0; +} + +static int +eh_initialize_tx_adapter(struct eventmode_conf *em_conf) +{ + struct tx_adapter_conf *adapter; + int i, ret; + + /* Configure Tx adapters */ + for (i = 0; i < em_conf->nb_tx_adapter; i++) { + adapter = &(em_conf->tx_adapter[i]); + ret = eh_tx_adapter_configure(em_conf, adapter); + if (ret < 0) { + EH_LOG_ERR("Failed to configure tx adapter %d", ret); + return ret; + } + } + return 0; +} + int32_t eh_devs_init(struct eh_conf *conf) { @@ -630,6 +873,13 @@ eh_devs_init(struct eh_conf *conf) return ret; } + /* Setup Tx adapter */ + ret = eh_initialize_tx_adapter(em_conf); + if (ret < 0) { + EH_LOG_ERR("Failed to initialize tx adapter %d", ret); + return ret; + } + /* Start eth devices after setting up adapter */ RTE_ETH_FOREACH_DEV(port_id) { @@ -713,5 +963,68 @@ eh_devs_uninit(struct eh_conf *conf) } } + /* Stop and release tx adapters */ + for (i = 0; i < em_conf->nb_tx_adapter; i++) { + + id = em_conf->tx_adapter[i].adapter_id; + ret = rte_event_eth_tx_adapter_stop(id); + if (ret < 0) { + EH_LOG_ERR("Failed to stop tx adapter %d", ret); + return ret; + } + + for (j = 0; j < em_conf->tx_adapter[i].nb_connections; j++) { + + ret = rte_event_eth_tx_adapter_queue_del(id, + em_conf->tx_adapter[i].conn[j].ethdev_id, -1); + if (ret < 0) { + EH_LOG_ERR( + "Failed to remove tx adapter queues %d", + ret); + return ret; + } + } + + ret = rte_event_eth_tx_adapter_free(id); + if (ret < 0) { + EH_LOG_ERR("Failed to free tx adapter %d", ret); + return ret; + } + } + return 0; } + +uint8_t +eh_get_tx_queue(struct eh_conf *conf, uint8_t eventdev_id) +{ + struct eventdev_params *eventdev_config; + struct eventmode_conf *em_conf; + + if (conf == NULL) { + EH_LOG_ERR("Invalid event helper configuration"); + return -EINVAL; + } + + if (conf->mode_params == NULL) { + EH_LOG_ERR("Invalid event mode parameters"); + return -EINVAL; + } + + /* Get eventmode conf */ + em_conf = conf->mode_params; + + /* Get event device conf */ + eventdev_config = eh_get_eventdev_params(em_conf, eventdev_id); + + if (eventdev_config == NULL) { + EH_LOG_ERR("Failed to read eventdev config"); + return -EINVAL; + } + + /* + * The last queue is reserved to be used as atomic queue for the + * last stage (eth packet tx stage) + */ + return eventdev_config->nb_eventqueue - 1; +} diff --git a/examples/ipsec-secgw/event_helper.h b/examples/ipsec-secgw/event_helper.h index 00ce14e..913b172 100644 --- a/examples/ipsec-secgw/event_helper.h +++ b/examples/ipsec-secgw/event_helper.h @@ -19,9 +19,15 @@ /* Max Rx adapters supported */ #define EVENT_MODE_MAX_RX_ADAPTERS RTE_EVENT_MAX_DEVS +/* Max Tx adapters supported */ +#define EVENT_MODE_MAX_TX_ADAPTERS RTE_EVENT_MAX_DEVS + /* Max Rx adapter connections */ #define EVENT_MODE_MAX_CONNECTIONS_PER_ADAPTER 16 +/* Max Tx adapter connections */ +#define EVENT_MODE_MAX_CONNECTIONS_PER_TX_ADAPTER 16 + /* Max event queues supported per event device */ #define EVENT_MODE_MAX_EVENT_QUEUES_PER_DEV RTE_EVENT_MAX_QUEUES_PER_DEV @@ -29,6 +35,9 @@ #define EVENT_MODE_MAX_LCORE_LINKS \ (EVENT_MODE_MAX_EVENT_DEVS * EVENT_MODE_MAX_EVENT_QUEUES_PER_DEV) +/* Max adapters that one Tx core can handle */ +#define EVENT_MODE_MAX_ADAPTERS_PER_TX_CORE EVENT_MODE_MAX_TX_ADAPTERS + /** * Packet transfer mode of the application */ @@ -76,6 +85,23 @@ struct rx_adapter_conf { conn[EVENT_MODE_MAX_CONNECTIONS_PER_ADAPTER]; }; +/* Tx adapter connection info */ +struct tx_adapter_connection_info { + uint8_t ethdev_id; + int32_t ethdev_tx_qid; +}; + +/* Tx adapter conf */ +struct tx_adapter_conf { + int32_t eventdev_id; + int32_t adapter_id; + uint32_t tx_core_id; + uint8_t nb_connections; + struct tx_adapter_connection_info + conn[EVENT_MODE_MAX_CONNECTIONS_PER_TX_ADAPTER]; + uint8_t tx_ev_queue; +}; + /* Eventmode conf data */ struct eventmode_conf { int nb_eventdev; @@ -86,6 +112,10 @@ struct eventmode_conf { /**< No of Rx adapters */ struct rx_adapter_conf rx_adapter[EVENT_MODE_MAX_RX_ADAPTERS]; /**< Rx adapter conf */ + uint8_t nb_tx_adapter; + /**< No of Tx adapters */ + struct tx_adapter_conf tx_adapter[EVENT_MODE_MAX_TX_ADAPTERS]; + /** Tx adapter conf */ uint8_t nb_link; /**< No of links */ struct eh_event_link_info @@ -166,4 +196,22 @@ eh_devs_init(struct eh_conf *conf); int32_t eh_devs_uninit(struct eh_conf *conf); +/** + * Get eventdev tx queue + * + * If the application uses event device which does not support internal port + * then it needs to submit the events to a Tx queue before final transmission. + * This Tx queue will be created internally by the eventmode helper subsystem, + * and application will need its queue ID when it runs the execution loop. + * + * @param mode_conf + * Event helper configuration + * @param eventdev_id + * Event device ID + * @return + * Tx queue ID + */ +uint8_t +eh_get_tx_queue(struct eh_conf *conf, uint8_t eventdev_id); + #endif /* _EVENT_HELPER_H_ */