From patchwork Tue Jan 14 14:25:14 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Andrzej Ostruszka [C]" X-Patchwork-Id: 64667 Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id CAAB8A04FF; Tue, 14 Jan 2020 15:25:25 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id EB57D1C0DD; Tue, 14 Jan 2020 15:25:24 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by dpdk.org (Postfix) with ESMTP id 2D51A1C0D6 for ; Tue, 14 Jan 2020 15:25:23 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 00EEKUIm013363 for ; Tue, 14 Jan 2020 06:25:22 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : mime-version : content-transfer-encoding : content-type; s=pfpt0818; bh=xuXcd58IgGzKJifWxjuV6ScfKyOCyS2WFjpSMnmcZ0A=; b=vSTKBHeYi1fXxj0CHuKfmtN8dcg7N6D0UPbTBUNCYbhdTcnj/KG8pI+C+YtkDO53en0F aLYyVRqRQJ5+DVb1UsIQd3RNB1Im9CgLIrFn/Lwff8YnKS1VOky3k/aGQnGY/92OgX5E tH1Nqo0A0fSmARP2ZtiRuJC+P3I4Tz9glOFae7NSj0xrvluH3KUZG/gMws+Yti1/RR+N 353pO6uamgJAkezjh5aQmqwkfeNcTMI/McaBLsdFl+IyF6EzaCTjkfr6sXx2q4cQM4x9 eMIUfr5rD0makHD9v0AXQPbZK95GFi9u2bUGg6DkdDnyQFZYkzuzhGbnJXltlytmmqwf +w== Received: from sc-exch02.marvell.com ([199.233.58.182]) by mx0a-0016f401.pphosted.com with ESMTP id 2xhc6sgng8-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Tue, 14 Jan 2020 06:25:22 -0800 Received: from SC-EXCH03.marvell.com (10.93.176.83) by SC-EXCH02.marvell.com (10.93.176.82) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Tue, 14 Jan 2020 06:25:20 -0800 Received: from maili.marvell.com (10.93.176.43) by SC-EXCH03.marvell.com (10.93.176.83) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Tue, 14 Jan 2020 06:25:20 -0800 Received: from amok.marvell.com (unknown [10.95.130.253]) by maili.marvell.com (Postfix) with ESMTP id D1FD63F703F; Tue, 14 Jan 2020 06:25:18 -0800 (PST) From: Andrzej Ostruszka To: CC: Jerin Jacob Kollanukkaran , Nithin Kumar Dabilpuram , Pavan Nikhilesh Bhagavatula , Kiran Kumar Kokkilagadda , Krzysztof Kanas Date: Tue, 14 Jan 2020 15:25:14 +0100 Message-ID: <20200114142517.29522-1-aostruszka@marvell.com> X-Mailer: git-send-email 2.17.1 MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.138, 18.0.572 definitions=2020-01-14_04:2020-01-13, 2020-01-14 signatures=0 Subject: [dpdk-dev] [RFC PATCH 0/3] introduce IF proxy library X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" What is this useful for ======================= Usually, when an ethernet port is assigned to DPDK it vanishes from the system and user looses ability to control it via normal configuration utilities (e.g. those from iproute2 package). Moreover by default DPDK application is not aware of the network configuration of the system. To address both of these issues application needs to: - add some command line interface (or other mechanism) allowing for control of the port and its configuration - query the status of network configuration and monitor its changes The purpose of this library is to help with both of these tasks (as long as they remain in domain of configuration available to the system). In other words, if DPDK application has some special needs, that cannot be addressed by the normal system configuration utilities, then they need to be solved by the application itself. The connection between DPDK and system is based on the existence of ports that are visible to both DPDK and system (like Tap, KNI and possibly some other drivers). These ports serve as an interface proxies. Let's visualize the action of the library by the following example: Linux | DPDK ============================================================== | | +-------+ +-------+ | | Port1 | | Port2 | "ip link set dev tap1 mtu 1600" | +-------+ +-------+ | | ^ ^ | +------+ | mtu_change | `->| Tap1 |---' callback | +------+ | "ip addr add 198.51.100.14 \ | | dev tap2" | | | +------+ | `->| Tap2 |-------------------' +------+ addr_add callback | "ip route add 198.0.2.0/24 \ | dev eth0" | | | route_add callback `-------------> | So we have two ports Port1 and Port2 that are not visible to the system. We create two proxy interfaces (here based on Tap driver) and bind the ports to their proxies. When user issues a command changing MTU for Tap1 interface the library notes this and calls "mtu_change" callback for the Port1. Similarly when user adds an IPv4 address to the Tap2 interface "addr_add" callback is called for the Port2. Note also that that not only port related callbacks are available - for example you can also get information about routing table. See below for a complete list of available callbacks. Please note that nothing has been mentioned about forwarding of the packets between system and DPDK. Since the proxies are normal DPDK ports you can receive/send to them via usual RX/TX burst API. However since the library is not aware of the structure of packet processing used by the application it cannot automatically forward the packets - it is responsibility of the application to include proxy ports into its packet processing engine. As mentioned above the intention of the library is to: - provide information about network configuration that would allow application to decide what to do with the packets received on DPDK ports, - allow for control of the ports via standard configuration utilities Although the library only helps you to identify proxy for given port (and vice versa) and calls appropriate callbacks it does open some interesting possibilities. For example you can use the proxy ports to forward packets for protocols that you do not wish to handle in DPDK application to the system protocol stack and just listen to the configuration changes - so that way you can "offload" handling of those protocols to the system. Why this RFC ============ We would like to solicit some input from the community: - regarding usefulness of this library - what is missing or what needs to be changed - about currently proposed API - any other suggestions and/or improvements are also welcome How to use it ============= Usage of this library is rather simple. You have to: 1. Create proxy (if you don't have port suitable for being proxy or you have one but do not wish to use it as a proxy). 2. Bind port to proxy. 3. Register callbacks. 4. Start listening to the network configuration. The only mandatory requirement for DPDK port to be able to act as a proxy is that it is visible in the system - this is checked during port to proxy binding by calling rte_eth_dev_info_get() on proxy port and inspecting 'if_index' field (it has to be non-zero). One can create such port in the application by calling: proxy_id = rte_ifpx_create(RTE_IFPX_DEFAULT); Upon success this returns id of DPDK proxy port created (RTE_MAX_ETHPORTS on failure). The argument selects type of proxy port to create (currently Tap/KNI only). This function actually is just a wrapper around: uint16_t rte_ifpx_create_by_devarg(const char *devarg); creating valid 'devarg' string for the chosen type of proxy. If you have other driver capable of acting as a proxy you can call rte_ifpx_create_by_devarg() directly passing appropriate argument. Once you have id of both port and proxy you can bind the two via: rte_ifpx_port_bind(port_id, proxy_id); This creates logical binding - as mentioned above there is no automatic packet forwarding. With this binding whenever user changes the state of proxy interface in the system (link up/down, change mac/mtu, add/remove IPv4/IPv6) you get appropriate callback called for the bound port. So far we've mentioned several times that the library calls callbacks. They are grouped in 'struct rte_ifpx_callbacks' and user provides them to the library via: rte_ifpx_callbacks_register(&cbs); It is worth mentioning that the context (lcore/thread) in which these callbacks are called is implementation defined. It might differ between different platforms, so the application needs to assume that some kind of inter lcore/thread synchronization/communication is required. Once we have bindings in place and callbacks registered, the only essential part that remains is to get the current network configuration and start listening to its changes. This is accomplished via a call to: rte_ifpx_listen(); And basically this is all one needs to understand how to use this library. Other less essential parts include: - ability to query what callbacks are available for given platform - getting mapping between proxy and port - unbinding the ports from proxy - destroying proxy port - closing the listening service - getting basic information about proxy Currently available features and implementation =============================================== The library's API is system independent but it obviously needs some system dependent parts. We provide exemplary Linux implementation (based on netlink sockets). Very similar implementation is possible for FreeBSD (with the usage of PF_ROUTE sockets). Windows implementation would need to differ much (probably IP Helper library would be of some help). Here is the list of currently implemented callbacks: struct rte_ifpx_callbacks { void (*mac_change)(uint16_t port_id, const struct rte_ether_addr *mac); void (*mtu_change)(uint16_t port_id, uint16_t mtu); void (*link_change)(uint16_t port_id, int is_up); void (*addr_add)(uint16_t port_id, uint32_t ip); void (*addr_del)(uint16_t port_id, uint32_t ip); void (*addr6_add)(uint16_t port_id, const uint8_t *ip); void (*addr6_del)(uint16_t port_id, const uint8_t *ip); void (*route_add)(uint32_t ip, uint8_t depth); void (*route_del)(uint32_t ip, uint8_t depth); void (*route6_add)(const uint8_t *ip, uint8_t depth); void (*route6_del)(const uint8_t *ip, uint8_t depth); void (*cfg_finished)(void); }; They are all rather self-descriptive with the exception of the last one. When the user calls rte_ifpx_listen() the library first queries the system for its current configuration. That might require several request/reply exchanges between DPDK and system and once it is finished this callback is called to let application know that all info has been gathered. BTW at the moment all IPv4 addresses are passed in host order. It is worth to mention also that while typical case would be a 1-to-1 mapping between port and proxy, the 1-to-many mapping is also supported. In that case port related callbacks will be called for each port bound to given proxy interface - in that case it is application responsibility to define semantic of such mapping (e.g. all changes apply to all ports, or link changes apply to all but other are accepted in "round robin" fashion, or ...). As mentioned above Linux implementation is based on netlink socket. This socket is registered as file descriptor in EAL interrupts (similarly to how EAL alarms are implemented). What is inside this RFC ======================= - 1 commit for API - 1 commit for implementation - this is just to show PoC, and allow for early playing around with the idea (e.g. run the test/example from the next commit) - 1 commit for test/example - just to show how this can be used Next steps ========== - gather community feedback - polish the implementation: * call the notification callbacks without lock held (at the moment attempts to modify callbacks from within the callback would deadlock) * separate the system dependent parts from the rest so that it is easy to figure out what needs to be reimplemented on different platforms * apply community suggestions - if any - add neighbour callbacks (ARP table) Best regards Andrzej Ostruszka Andrzej Ostruszka (3): lib: introduce IF proxy library (API) if_proxy: add preliminary Linux implementation if_proxy: add example, test and documentation app/test/Makefile | 5 + app/test/meson.build | 1 + app/test/test_if_proxy.c | 431 ++++++++++ config/common_base | 5 + doc/guides/prog_guide/if_proxy_lib.rst | 103 +++ doc/guides/prog_guide/index.rst | 1 + examples/Makefile | 1 + examples/if_proxy/Makefile | 58 ++ examples/if_proxy/main.c | 203 +++++ examples/if_proxy/meson.build | 12 + examples/meson.build | 2 +- lib/Makefile | 2 + .../common/include/rte_eal_interrupts.h | 2 + lib/librte_eal/linux/eal/eal_interrupts.c | 14 +- lib/librte_if_proxy/Makefile | 25 + lib/librte_if_proxy/meson.build | 7 + lib/librte_if_proxy/rte_if_proxy.c | 803 ++++++++++++++++++ lib/librte_if_proxy/rte_if_proxy.h | 364 ++++++++ lib/meson.build | 2 +- 19 files changed, 2035 insertions(+), 6 deletions(-) create mode 100644 app/test/test_if_proxy.c create mode 100644 doc/guides/prog_guide/if_proxy_lib.rst create mode 100644 examples/if_proxy/Makefile create mode 100644 examples/if_proxy/main.c create mode 100644 examples/if_proxy/meson.build create mode 100644 lib/librte_if_proxy/Makefile create mode 100644 lib/librte_if_proxy/meson.build create mode 100644 lib/librte_if_proxy/rte_if_proxy.c create mode 100644 lib/librte_if_proxy/rte_if_proxy.h