From patchwork Fri Oct 19 00:23:54 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Gora X-Patchwork-Id: 47056 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 693EF7D4E; Fri, 19 Oct 2018 02:24:23 +0200 (CEST) Received: from mail-qt1-f196.google.com (mail-qt1-f196.google.com [209.85.160.196]) by dpdk.org (Postfix) with ESMTP id 5F3877D4E for ; Fri, 19 Oct 2018 02:24:22 +0200 (CEST) Received: by mail-qt1-f196.google.com with SMTP id x24-v6so4931697qtx.11 for ; Thu, 18 Oct 2018 17:24:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=GU+Ae+jbMqhbgYDk08eZMjAQViAixUhJ/KkiUwuLcxA=; b=gJrWcIAkkYyW3UD4P5e8qin3gPse5M0IQFa0IE2EGaHgS32oOslwSls/MxaUNaH1xh sMFniZ9/PKGdRegmHLO3jF/DTelbM64cstG26fPXMXgRMIf5q1tNLqebVktgwAIvj19O a6JgW9h0XX7ICDlZYV/R4oC8BwROUgddwtUXUYPHShjXev8uIFWpdEMQt4lZspYgC6hY S3BlrvK3wmpCKjxpUymIeP8TKCfH+6OnNkQNaGsdLjKrjJmW46Th7XMK8MOCNcQrDsMQ ++SoYhAwaJO06a2Zwbgu9XOwgbNvORRMDddHjzSAuUDW4ixyKj9+bKvhx6nW1OdXET2R TPHg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references:mime-version:content-transfer-encoding; bh=GU+Ae+jbMqhbgYDk08eZMjAQViAixUhJ/KkiUwuLcxA=; b=sDn2UN0VthylQkY81AbwWGbUzt0OtA6TzdDjmgwjsxJkoQJyqtEsY5rnlR8DxN89Pf XU+Ykc5gZ51QiQkH9aozGe9v0uJJSqVC3uiqTWxQ18sc++AdfUdTHD+DxU0VE6cop/Cz x9M0TwNr0jsojD9QmQ2yIu8cI9M9S5U7EE+uOKnn9rB/zuRTLUW111vE/avain0BiUEl 3rpC7TgMggHvTntyiZxmW6xzd0h7pQE8PVx3hYmi2Y7X8F14ISNCYjUyujZm3j3gntnL +Hs3tghzGbKwsDhIeuSQIkq/sc+lvOdxXdsJG2prkAqHZLZfkvmVvddRTNnsqXaWQ1bw DyJQ== X-Gm-Message-State: ABuFfogg38deSu2swcxyr88E6HHTXnQdMsMIIWGiIdgQK0NBK8eGalg9 jNqyvaKjGU0dpIuvWw/Bbxto0nyT X-Google-Smtp-Source: ACcGV631DQ9HsE5g6E9Tfe2Kyxm/X99RhH+69S1MpZ4mrUlWlfWfuCFpPGFrzmo/zw2P7HqnCKFTVQ== X-Received: by 2002:a0c:d023:: with SMTP id u32mr12450572qvg.187.1539908661416; Thu, 18 Oct 2018 17:24:21 -0700 (PDT) Received: from snappy.local.lan ([191.205.43.134]) by smtp.gmail.com with ESMTPSA id y124-v6sm12963926qke.22.2018.10.18.17.24.18 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 18 Oct 2018 17:24:20 -0700 (PDT) From: Dan Gora To: dev@dpdk.org Cc: Igor Ryzhov , Stephen Hemminger , Ferruh Yigit , Dan Gora Date: Thu, 18 Oct 2018 21:23:54 -0300 Message-Id: <20181019002358.17132-2-dg@adax.com> X-Mailer: git-send-email 2.19.0 In-Reply-To: <20181019002358.17132-1-dg@adax.com> References: <20180911232906.18352-1-dg@adax.com> <20181019002358.17132-1-dg@adax.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v5 1/5] kni: add API to set link status on kernel interface X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add a new API function to KNI, rte_kni_update_link() to allow DPDK applications to update the link status for KNI network interfaces in the linux kernel. Signed-off-by: Dan Gora --- lib/librte_kni/rte_kni.c | 41 ++++++++++++++++ lib/librte_kni/rte_kni.h | 20 ++++++++ lib/librte_kni/rte_kni_version.map | 6 +++ test/test/test_kni.c | 77 ++++++++++++++++++++++++++++++ 4 files changed, 144 insertions(+) diff --git a/lib/librte_kni/rte_kni.c b/lib/librte_kni/rte_kni.c index b8edd40f4..c9726d4f8 100644 --- a/lib/librte_kni/rte_kni.c +++ b/lib/librte_kni/rte_kni.c @@ -717,6 +717,47 @@ rte_kni_unregister_handlers(struct rte_kni *kni) return 0; } + +int __rte_experimental +rte_kni_update_link(struct rte_kni *kni, unsigned int linkup) +{ + char path[64]; + char old_carrier[2]; + const char *new_carrier; + int old_linkup; + int fd, ret; + + if (kni == NULL) + return -1; + + snprintf(path, sizeof(path), "/sys/devices/virtual/net/%s/carrier", + kni->name); + + fd = open(path, O_RDWR); + if (fd == -1) { + RTE_LOG(ERR, KNI, "Failed to open file: %s.\n", path); + return -1; + } + + ret = read(fd, old_carrier, 2); + if (ret < 1) { + close(fd); + return -1; + } + old_linkup = (old_carrier[0] == '1'); + + new_carrier = linkup ? "1" : "0"; + ret = write(fd, new_carrier, 1); + if (ret < 1) { + RTE_LOG(ERR, KNI, "Failed to write file: %s.\n", path); + close(fd); + return -1; + } + + close(fd); + return old_linkup; +} + void rte_kni_close(void) { diff --git a/lib/librte_kni/rte_kni.h b/lib/librte_kni/rte_kni.h index 601abdfc6..02ca43b4b 100644 --- a/lib/librte_kni/rte_kni.h +++ b/lib/librte_kni/rte_kni.h @@ -232,6 +232,26 @@ int rte_kni_register_handlers(struct rte_kni *kni, struct rte_kni_ops *ops); */ int rte_kni_unregister_handlers(struct rte_kni *kni); +/** + * Update link carrier state for KNI port. + * + * Update the linkup/linkdown state of a KNI interface in the kernel. + * + * @param kni + * pointer to struct rte_kni. + * @param linkup + * New link state: + * 0 for linkdown. + * > 0 for linkup. + * + * @return + * On failure: -1 + * Previous link state == linkdown: 0 + * Previous link state == linkup: 1 + */ +int __rte_experimental +rte_kni_update_link(struct rte_kni *kni, unsigned int linkup); + /** * Close KNI device. */ diff --git a/lib/librte_kni/rte_kni_version.map b/lib/librte_kni/rte_kni_version.map index acd515eb0..c877dc6aa 100644 --- a/lib/librte_kni/rte_kni_version.map +++ b/lib/librte_kni/rte_kni_version.map @@ -15,3 +15,9 @@ DPDK_2.0 { local: *; }; + +EXPERIMENTAL { + global: + + rte_kni_update_link; +}; diff --git a/test/test/test_kni.c b/test/test/test_kni.c index cf64c0b27..e38206905 100644 --- a/test/test/test_kni.c +++ b/test/test/test_kni.c @@ -118,6 +118,79 @@ kni_change_mtu(uint16_t port_id, unsigned int new_mtu) port_id, kni_pkt_mtu); return 0; } + +static int +test_kni_link_change(void) +{ + int ret; + int pid; + + pid = fork(); + if (pid < 0) { + printf("Error: Failed to fork a process\n"); + return -1; + } + + if (pid == 0) { + printf("Starting KNI Link status change tests.\n"); + if (system(IFCONFIG TEST_KNI_PORT" up") == -1) { + ret = -1; + goto error; + } + + ret = rte_kni_update_link(test_kni_ctx, 1); + if (ret < 0) { + printf("Failed to change link state to Up ret=%d.\n", + ret); + goto error; + } + rte_delay_ms(1000); + printf("KNI: Set LINKUP, previous state=%d\n", ret); + + ret = rte_kni_update_link(test_kni_ctx, 0); + if (ret != 1) { + printf( + "Failed! Previous link state should be 1, returned %d.\n", + ret); + goto error; + } + rte_delay_ms(1000); + printf("KNI: Set LINKDOWN, previous state=%d\n", ret); + + ret = rte_kni_update_link(test_kni_ctx, 1); + if (ret != 0) { + printf( + "Failed! Previous link state should be 0, returned %d.\n", + ret); + goto error; + } + printf("KNI: Set LINKUP, previous state=%d\n", ret); + + ret = 0; + rte_delay_ms(1000); + +error: + if (system(IFCONFIG TEST_KNI_PORT" down") == -1) + ret = -1; + + printf("KNI: Link status change tests: %s.\n", + (ret == 0) ? "Passed" : "Failed"); + exit(ret); + } else { + int p_ret, status; + + while (1) { + p_ret = waitpid(pid, &status, WNOHANG); + if (p_ret != 0) { + if (WIFEXITED(status)) + return WEXITSTATUS(status); + return -1; + } + rte_delay_ms(10); + rte_kni_handle_request(test_kni_ctx); + } + } +} /** * This loop fully tests the basic functions of KNI. e.g. transmitting, * receiving to, from kernel space, and kernel requests. @@ -401,6 +474,10 @@ test_kni_processing(uint16_t port_id, struct rte_mempool *mp) goto fail_kni; } + ret = test_kni_link_change(); + if (ret != 0) + goto fail_kni; + rte_eal_mp_remote_launch(test_kni_loop, NULL, CALL_MASTER); RTE_LCORE_FOREACH_SLAVE(i) { if (rte_eal_wait_lcore(i) < 0) { From patchwork Fri Oct 19 00:23:55 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Gora X-Patchwork-Id: 47057 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id B6EAA1B0FE; Fri, 19 Oct 2018 02:24:31 +0200 (CEST) Received: from mail-qt1-f196.google.com (mail-qt1-f196.google.com [209.85.160.196]) by dpdk.org (Postfix) with ESMTP id 429107D05 for ; Fri, 19 Oct 2018 02:24:30 +0200 (CEST) Received: by mail-qt1-f196.google.com with SMTP id l41-v6so36464148qtl.8 for ; Thu, 18 Oct 2018 17:24:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=PAHfaF7bBA2pwTKnNyBLBZ7cAsqfGg9KDm4B9jLj1rw=; b=D0Uz9K1p4B57lxYiAh9m0B/fB3MO+5vFiDOf4g5rNoneVlpE2M/uwyPL2UtgZe7yC9 8Q0//po1rwrqKb5OzQ+6Ll3IBGg1ZzpQr/By47lpXQkxEpP0QHXlQoz7n8B2g+nhvNmH SCkrVsAsE8UXEeamUVtzsh7rmLn+dZn4fOZgDgs+uP+pnzfERTjpqxK96qZpTCU5604J ODQujQ3enOKThk7u2rVWwucj8sfJZDD8FGgMAJJ7oiGaO9SX8m8Nj/p4houWFSx6kHfx 1y/WNDrvDYoVZPHSVyI83JxRjJ+smWI3/gdGRy4twpeZzT2g137mlKhSjk++hheGd+e2 CFtQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references:mime-version:content-transfer-encoding; bh=PAHfaF7bBA2pwTKnNyBLBZ7cAsqfGg9KDm4B9jLj1rw=; b=uhyHyGHNhFPKKzuTQ/dkl46NEf8un7HbpwD51jT9x1lF3bM3diAhcMgg6YdZCjxemI VX+4M7BtLJE0veBsyQBfRECuYB8pg4z8/AfwBhRgTKt8UlN+l1esYv28kv+ebbwyzoTu 5yhgDwheNL573fs7CmVuVCJ5C6vi7edqeJKmCgPAYMRYbBvpNZbIGpJMlya+BRdPGHNy iN2Ren3t4H9OUUWptAKfqBTFLBCHveDoNjaxGJnYMm+OFfNqCx8AFlVEVlm6Z+RRkf4X 3YPkpzuZd5+if91+pIoaEJcXGOubI3YEb3WfDmZnnSAc1mQR14ehitbfbnN2x/kDG+xR pEHQ== X-Gm-Message-State: ABuFfogWLBK2bkg6NgR77KDaTdY6nkZD+lGQGNwpqgebC2sFp+DEj3/v lXalNrcaB2kVwEO2BpoBEP6jOAL+ X-Google-Smtp-Source: ACcGV63h8qpHFczcwvz0OPpjWbv8XySXY3DI3CWhNb0Qxe/NtpeQoy/byjNTipNgTdH/7Mg3GC3tCA== X-Received: by 2002:a0c:d8d4:: with SMTP id w20mr32997106qvj.214.1539908669100; Thu, 18 Oct 2018 17:24:29 -0700 (PDT) Received: from snappy.local.lan ([191.205.43.134]) by smtp.gmail.com with ESMTPSA id y124-v6sm12963926qke.22.2018.10.18.17.24.25 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 18 Oct 2018 17:24:28 -0700 (PDT) From: Dan Gora To: dev@dpdk.org Cc: Igor Ryzhov , Stephen Hemminger , Ferruh Yigit , Dan Gora , John McNamara , Marko Kovacevic Date: Thu, 18 Oct 2018 21:23:55 -0300 Message-Id: <20181019002358.17132-3-dg@adax.com> X-Mailer: git-send-email 2.19.0 In-Reply-To: <20181019002358.17132-1-dg@adax.com> References: <20180911232906.18352-1-dg@adax.com> <20181019002358.17132-1-dg@adax.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v5 2/5] kni: set default carrier state of interface X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add module parameter 'carrier='on|off' to set the default carrier state for linux network interfaces created by the KNI module. The default carrier state is 'off'. For KNI interfaces which need to reflect the carrier state of a physical Ethernet port controlled by the DPDK application, the default carrier state should be left set to 'off'. The application can set the carrier state of the KNI interface to reflect the state of the physical Ethernet port using rte_kni_update_link(). For KNI interfaces which are purely virtual, the default carrier state can be set to 'on'. This enables the KNI interface to be used without having to explicity set the carrier state to 'on' using rte_kni_update_link(). Signed-off-by: Dan Gora --- .../prog_guide/kernel_nic_interface.rst | 203 ++++++++++++++---- kernel/linux/kni/kni_dev.h | 3 + kernel/linux/kni/kni_misc.c | 57 ++++- kernel/linux/kni/kni_net.c | 5 + 4 files changed, 213 insertions(+), 55 deletions(-) diff --git a/doc/guides/prog_guide/kernel_nic_interface.rst b/doc/guides/prog_guide/kernel_nic_interface.rst index 8fa13fa1a..9173f2b78 100644 --- a/doc/guides/prog_guide/kernel_nic_interface.rst +++ b/doc/guides/prog_guide/kernel_nic_interface.rst @@ -29,58 +29,182 @@ The components of an application using the DPDK Kernel NIC Interface are shown i The DPDK KNI Kernel Module -------------------------- -The KNI kernel loadable module provides support for two types of devices: +The KNI kernel loadable module ``rte_kni`` provides the kernel interface +for DPDK applications. -* A Miscellaneous device (/dev/kni) that: +When the ``rte_kni`` module is loaded, it will create a device ``/dev/kni`` +that is used by the DPDK KNI API functions to control and communicate with +the kernel module. - * Creates net devices (via ioctl calls). +The ``rte_kni`` kernel module contains several optional parameters which +can be specified when the module is loaded to control its behavior: - * Maintains a kernel thread context shared by all KNI instances - (simulating the RX side of the net driver). +.. code-block:: console - * For single kernel thread mode, maintains a kernel thread context shared by all KNI instances - (simulating the RX side of the net driver). + # modinfo rte_kni.ko + + parm: lo_mode: KNI loopback mode (default=lo_mode_none): + lo_mode_none Kernel loopback disabled + lo_mode_fifo Enable kernel loopback with fifo + lo_mode_fifo_skb Enable kernel loopback with fifo and skb buffer + (charp) + parm: kthread_mode: Kernel thread mode (default=single): + single Single kernel thread mode enabled. + multiple Multiple kernel thread mode enabled. + (charp) + parm: carrier: Default carrier state for KNI interface (default=off): + off Interfaces will be created with carrier state set to off. + on Interfaces will be created with carrier state set to on. + (charp) - * For multiple kernel thread mode, maintains a kernel thread context for each KNI instance - (simulating the RX side of the net driver). +Loading the ``rte_kni`` kernel module without any optional parameters is +the typical way a DPDK application gets packets into and out of the kernel +network stack. Without any parameters, only one kernel thread is created +for all KNI devices for packet receiving in kernel side, loopback mode is +disabled, and the default carrier state of KNI interfaces is set to *off*. -* Net device: +.. code-block:: console - * Net functionality provided by implementing several operations such as netdev_ops, - header_ops, ethtool_ops that are defined by struct net_device, - including support for DPDK mbufs and FIFOs. + # insmod kmod/rte_kni.ko - * The interface name is provided from userspace. +Loopback Mode +~~~~~~~~~~~~~ - * The MAC address can be the real NIC MAC address or random. +For testing, the ``rte_kni`` kernel module can be loaded in loopback mode +by specifying the ``lo_mode`` parameter: -KNI Creation and Deletion -------------------------- +.. code-block:: console + + # insmod kmod/rte_kni.ko lo_mode=lo_mode_fifo + +The ``lo_mode_fifo`` loopback option will loop back ring enqueue/dequeue +operations in kernel space. + +.. code-block:: console + + # insmod kmod/rte_kni.ko lo_mode=lo_mode_fifo_skb + +The ``lo_mode_fifo_skb`` loopback option will loop back ring enqueue/dequeue +operations and sk buffer copies in kernel space. + +If the ``lo_mode`` parameter is not specified, loopback mode is disabled. + +Kernel Thread Mode +~~~~~~~~~~~~~~~~~~ + +To provide flexibility of performance, the ``rte_kni`` KNI kernel module +can be loaded with the ``kthread_mode`` parameter. The ``rte_kni`` kernel +module supports two options: "single kernel thread" mode and "multiple +kernel thread" mode. + +Single kernel thread mode is enabled as follows: + +.. code-block:: console + + # insmod kmod/rte_kni.ko kthread_mode=single + +This mode will create only one kernel thread for all KNI interfaces to +receive data on the kernel side. By default, this kernel thread is not +bound to any particular core, but the user can set the core affinity for +this kernel thread by setting the ``core_id`` and ``force_bind`` parameters +in ``struct rte_kni_conf`` when the first KNI interface is created: + +For optimum performance, the kernel thread should be bound to a core in +on the same socket as the DPDK lcores used in the application. -The KNI interfaces are created by a DPDK application dynamically. -The interface name and FIFO details are provided by the application through an ioctl call -using the rte_kni_device_info struct which contains: +The KNI kernel module can also be configured to start a separate kernel +thread for each KNI interface created by the DPDK application. Multiple +kernel thread mode is enabled as follows: -* The interface name. +.. code-block:: console -* Physical addresses of the corresponding memzones for the relevant FIFOs. + # insmod kmod/rte_kni.ko kthread_mode=multiple -* Mbuf mempool details, both physical and virtual (to calculate the offset for mbuf pointers). +This mode will create a separate kernel thread for each KNI interface to +receive data on the kernel side. The core affinity of each ``kni_thread`` +kernel thread can be specified by setting the ``core_id`` and ``force_bind`` +parameters in ``struct rte_kni_conf`` when each KNI interface is created. -* PCI information. +Multiple kernel thread mode can provide scalable higher performance if +sufficient unused cores are available on the host system. -* Core affinity. +If the ``kthread_mode`` parameter is not specified, the "single kernel +thread" mode is used. -Refer to rte_kni_common.h in the DPDK source code for more details. +Default Carrier State +~~~~~~~~~~~~~~~~~~~~~ -The physical addresses will be re-mapped into the kernel address space and stored in separate KNI contexts. +The default carrier state of KNI interfaces created by the ``rte_kni`` +kernel module is controlled via the ``carrier`` option when the module +is loaded. -The affinity of kernel RX thread (both single and multi-threaded modes) is controlled by force_bind and -core_id config parameters. +If ``carrier=off`` is specified, the kernel module will leave the carrier +state of the interface *down* when the interface is management enabled. +The DPDK application can set the carrier state of the KNI interface using the +``rte_kni_update_link()`` function. This is useful for DPDK applications +which require that the carrier state of the KNI interface reflect the +actual link state of the corresponding physical NIC port. -The KNI interfaces can be deleted by a DPDK application dynamically after being created. -Furthermore, all those KNI interfaces not deleted will be deleted on the release operation -of the miscellaneous device (when the DPDK application is closed). +If ``carrier=on`` is specified, the kernel module will automatically set +the carrier state of the interface to *up* when the interface is management +enabled. This is useful for DPDK applications which use the KNI interface as +a purely virtual interface that does not correspond to any physical hardware +and do not wish to explicitly set the carrier state of the interface with +``rte_kni_update_link()``. It is also useful for testing in loopback mode +where the NIC port may not be physically connected to anything. + +To set the default carrier state to *on*: + +.. code-block:: console + + # insmod kmod/rte_kni.ko carrier=on + +To set the default carrier state to *off*: + +.. code-block:: console + + # insmod kmod/rte_kni.ko carrier=off + +If the ``carrier`` parameter is not specified, the default carrier state +of KNI interfaces will be set to *off*. + +KNI Creation and Deletion +------------------------- + +Before any KNI interfaces can be created, the ``rte_kni`` kernel module must +be loaded into the kernel and configured withe ``rte_kni_init()`` function. + +The KNI interfaces are created by a DPDK application dynamically via the +``rte_kni_alloc()`` function. + +The ``struct rte_kni_conf`` structure contains fields which allow the +user to specify the interface name, set the MTU size, set an explicit or +random MAC address and control the affinity of the kernel Rx thread(s) +(both single and multi-threaded modes). + +The ``struct rte_kni_ops`` structure contains pointers to functions to +handle requests from the ``rte_kni`` kernel module. These functions +allow DPDK applications to perform actions when the KNI interfaces are +manipulated by control commands or functions external to the application. + +For example, the DPDK application may wish to enabled/disable a physical +NIC port when a user enabled/disables a KNI interface with ``ip link set +[up|down] dev ``. The DPDK application can register a callback for +``config_network_if`` which will be called when the interface management +state changes. + +In order to run these callbacks, the application must periodically call +the ``rte_kni_handle_request()`` function. Any user callback function +registered will be called directly from ``rte_kni_handle_request()`` +so care must be taken to prevent deadlock or not block any DPDK fastpath +tasks. Typically DPDK applications which use these callbacks will need +to create a separate thread or secondary process to periodically call +``rte_kni_handle_request()``. + +The KNI interfaces can be deleted by a DPDK application with +``rte_kni_release()``. All KNI interfaces not explicitly deleted will be +deleted when the the ``/dev/kni`` device is closed, either explicitly with +``rte_kni_close()`` or when the DPDK application is closed. DPDK mbuf Flow -------------- @@ -118,7 +242,7 @@ The packet is received from the Linux net stack, by calling the kni_net_tx() cal The mbuf is dequeued (without waiting due the cache) and filled with data from sk_buff. The sk_buff is then freed and the mbuf sent in the tx_q FIFO. -The DPDK TX thread dequeues the mbuf and sends it to the PMD (via rte_eth_tx_burst()). +The DPDK TX thread dequeues the mbuf and sends it to the PMD via ``rte_eth_tx_burst()``. It then puts the mbuf back in the cache. Ethtool @@ -128,16 +252,3 @@ Ethtool is a Linux-specific tool with corresponding support in the kernel where each net device must register its own callbacks for the supported operations. The current implementation uses the igb/ixgbe modified Linux drivers for ethtool support. Ethtool is not supported in i40e and VMs (VF or EM devices). - -Link state and MTU change -------------------------- - -Link state and MTU change are network interface specific operations usually done via ifconfig. -The request is initiated from the kernel side (in the context of the ifconfig process) -and handled by the user space DPDK application. -The application polls the request, calls the application handler and returns the response back into the kernel space. - -The application handlers can be registered upon interface creation or explicitly registered/unregistered in runtime. -This provides flexibility in multiprocess scenarios -(where the KNI is created in the primary process but the callbacks are handled in the secondary one). -The constraint is that a single process can register and handle the requests. diff --git a/kernel/linux/kni/kni_dev.h b/kernel/linux/kni/kni_dev.h index 6275ef27f..688f574a4 100644 --- a/kernel/linux/kni/kni_dev.h +++ b/kernel/linux/kni/kni_dev.h @@ -29,6 +29,9 @@ #define MBUF_BURST_SZ 32 +/* Default carrier state for created KNI network interfaces */ +extern uint32_t dflt_carrier; + /** * A structure describing the private information for a kni device. */ diff --git a/kernel/linux/kni/kni_misc.c b/kernel/linux/kni/kni_misc.c index fa69f8e63..0d2e85c30 100644 --- a/kernel/linux/kni/kni_misc.c +++ b/kernel/linux/kni/kni_misc.c @@ -39,6 +39,10 @@ static char *lo_mode; static char *kthread_mode; static uint32_t multiple_kthread_on; +/* Default carrier state for created KNI network interfaces */ +static char *carrier; +uint32_t dflt_carrier; + #define KNI_DEV_IN_USE_BIT_NUM 0 /* Bit number for device in use */ static int kni_net_id; @@ -466,6 +470,8 @@ kni_ioctl_create(struct net *net, uint32_t ioctl_num, return -ENODEV; } + netif_carrier_off(net_dev); + ret = kni_run_thread(knet, kni, dev_info.force_bind); if (ret != 0) return ret; @@ -590,6 +596,24 @@ kni_parse_kthread_mode(void) return 0; } +static int __init +kni_parse_carrier_state(void) +{ + if (!carrier) { + dflt_carrier = 0; + return 0; + } + + if (strcmp(carrier, "off") == 0) + dflt_carrier = 0; + else if (strcmp(carrier, "on") == 0) + dflt_carrier = 1; + else + return -1; + + return 0; +} + static int __init kni_init(void) { @@ -605,6 +629,16 @@ kni_init(void) else pr_debug("Multiple kernel thread mode enabled\n"); + if (kni_parse_carrier_state() < 0) { + pr_err("Invalid parameter for carrier\n"); + return -EINVAL; + } + + if (dflt_carrier == 0) + pr_debug("Default carrier state set to off.\n"); + else + pr_debug("Default carrier state set to on.\n"); + #ifdef HAVE_SIMPLIFIED_PERNET_OPERATIONS rc = register_pernet_subsys(&kni_net_ops); #else @@ -647,19 +681,24 @@ kni_exit(void) module_init(kni_init); module_exit(kni_exit); -module_param(lo_mode, charp, S_IRUGO | S_IWUSR); +module_param(lo_mode, charp, 0644); MODULE_PARM_DESC(lo_mode, "KNI loopback mode (default=lo_mode_none):\n" -" lo_mode_none Kernel loopback disabled\n" -" lo_mode_fifo Enable kernel loopback with fifo\n" -" lo_mode_fifo_skb Enable kernel loopback with fifo and skb buffer\n" -"\n" +"\t\tlo_mode_none Kernel loopback disabled\n" +"\t\tlo_mode_fifo Enable kernel loopback with fifo\n" +"\t\tlo_mode_fifo_skb Enable kernel loopback with fifo and skb buffer\n" ); -module_param(kthread_mode, charp, S_IRUGO); +module_param(kthread_mode, charp, 0644); MODULE_PARM_DESC(kthread_mode, "Kernel thread mode (default=single):\n" -" single Single kernel thread mode enabled.\n" -" multiple Multiple kernel thread mode enabled.\n" -"\n" +"\t\tsingle Single kernel thread mode enabled.\n" +"\t\tmultiple Multiple kernel thread mode enabled.\n" +); + +module_param(carrier, charp, 0644); +MODULE_PARM_DESC(carrier, +"Default carrier state for KNI interface (default=off):\n" +"\t\toff Interfaces will be created with carrier state set to off.\n" +"\t\ton Interfaces will be created with carrier state set to on.\n" ); diff --git a/kernel/linux/kni/kni_net.c b/kernel/linux/kni/kni_net.c index 3a542d737..7371b6d58 100644 --- a/kernel/linux/kni/kni_net.c +++ b/kernel/linux/kni/kni_net.c @@ -133,6 +133,10 @@ kni_net_open(struct net_device *dev) struct kni_dev *kni = netdev_priv(dev); netif_start_queue(dev); + if (dflt_carrier == 1) + netif_carrier_on(dev); + else + netif_carrier_off(dev); memset(&req, 0, sizeof(req)); req.req_id = RTE_KNI_REQ_CFG_NETWORK_IF; @@ -152,6 +156,7 @@ kni_net_release(struct net_device *dev) struct kni_dev *kni = netdev_priv(dev); netif_stop_queue(dev); /* can't transmit any more */ + netif_carrier_off(dev); memset(&req, 0, sizeof(req)); req.req_id = RTE_KNI_REQ_CFG_NETWORK_IF; From patchwork Fri Oct 19 00:23:56 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Gora X-Patchwork-Id: 47058 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 85CEE1B118; Fri, 19 Oct 2018 02:24:37 +0200 (CEST) Received: from mail-qt1-f193.google.com (mail-qt1-f193.google.com [209.85.160.193]) by dpdk.org (Postfix) with ESMTP id 41C211B118 for ; Fri, 19 Oct 2018 02:24:36 +0200 (CEST) Received: by mail-qt1-f193.google.com with SMTP id l41-v6so36464340qtl.8 for ; Thu, 18 Oct 2018 17:24:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=QrLUK46ue0GAzvJc00QmyPIQuNeQQM+LDiPulcmMH3w=; b=fH9eyYJO4zaoQNrLSDs7kqZcB/Qo6S+m9aYCTwHuSeGseMj8xRbRQpEp6tBj8m7b2K SD+VpVVveRtGzfyeaa+hX05O/y+4qu/jy1kFU/QmFOs6iVJzy2ng5brekO3rzzSP2mIh z3fbcXwy0lKf+NoLZuDl/Dd6ybwXiHE6Stmdv4ViwUy+sEJ8FConnmyNUhw8KsSciKqG St+46rYrt876sGX+tqhUBDEIPLqVVBlEQhZ3Zq9wsbrY6RLkJD/iPiULdzsbmbbG2i6n EXMyiWAINyYHRaSvxFC0lOK1uzyvwcyhhMnV5R74nFIA85+sDuryyePhvVYTm+iOyFDw UJqA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references:mime-version:content-transfer-encoding; bh=QrLUK46ue0GAzvJc00QmyPIQuNeQQM+LDiPulcmMH3w=; b=l4b9XRXGBEGPYPQA0vL7SwcpOuKVQTHnmmLMqPWp0i7+S08luhm6vcTvteoGlCpGaE OdUST9eTBJrJ4pEPlaPZLrjSgMrWPajcD2I9JYJznTzqkvt82yQwOIsBKhnNqm8LjE4z 6CMtQJpVn6Au6Oyz+xMcuASsnbJTBWctA9GHIPrQHOEvT3At7uimQRUiUUaVmiyk7CFF 4b1NwU2p6dj5kKXJndZsWekxdeBlHJzkG8MfJQf4YUzF060GQrLHAhUsTVpQFpufTisN qI2kjPaBz0PuGMEgxHCTREd+wlpQuTkedH5SPzt2UGVgptPfc2+3k8jDhcjFQXSVUK8G G8YA== X-Gm-Message-State: ABuFfogLg5elOWWACFoOs3q5PmShhXVhvss6GNSGIVICXhwJzcfl4uJW UDC0OOpZLSuBvXTu9VvokOMbh5eg X-Google-Smtp-Source: ACcGV63CISE+NnSvrTJ36T7+ua4OEqaEldN7QyiiMS2Eioyat5vFpMkgPhVoRRoO9A50qInWf3tkcg== X-Received: by 2002:ac8:392e:: with SMTP id s43-v6mr31778939qtb.294.1539908675111; Thu, 18 Oct 2018 17:24:35 -0700 (PDT) Received: from snappy.local.lan ([191.205.43.134]) by smtp.gmail.com with ESMTPSA id y124-v6sm12963926qke.22.2018.10.18.17.24.31 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 18 Oct 2018 17:24:34 -0700 (PDT) From: Dan Gora To: dev@dpdk.org Cc: Igor Ryzhov , Stephen Hemminger , Ferruh Yigit , Dan Gora , John McNamara , Marko Kovacevic Date: Thu, 18 Oct 2018 21:23:56 -0300 Message-Id: <20181019002358.17132-4-dg@adax.com> X-Mailer: git-send-email 2.19.0 In-Reply-To: <20181019002358.17132-1-dg@adax.com> References: <20180911232906.18352-1-dg@adax.com> <20181019002358.17132-1-dg@adax.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v5 3/5] examples/kni: monitor and update link status continually X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Update KNI example to add the command line flag '-m' to enable a function to continuously monitor the Ethernet link status of the physical link and update the link status of the corresponding interfaces with rte_kni_update_link(). Signed-off-by: Dan Gora --- .../sample_app_ug/kernel_nic_interface.rst | 249 +++++++++++------- examples/kni/Makefile | 2 + examples/kni/main.c | 70 ++++- 3 files changed, 224 insertions(+), 97 deletions(-) diff --git a/doc/guides/sample_app_ug/kernel_nic_interface.rst b/doc/guides/sample_app_ug/kernel_nic_interface.rst index 1b3ee9a5e..8de61bed6 100644 --- a/doc/guides/sample_app_ug/kernel_nic_interface.rst +++ b/doc/guides/sample_app_ug/kernel_nic_interface.rst @@ -31,18 +31,27 @@ This is done by creating one or more kernel net devices for each of the DPDK por The application allows the use of standard Linux tools (ethtool, ifconfig, tcpdump) with the DPDK ports and also the exchange of packets between the DPDK application and the Linux* kernel. +The Kernel NIC Interface sample application requires that the +KNI kernel module ``rte_kni`` be loaded into the kernel. See +:doc:`../prog_guide/kernel_nic_interface` for more information on loading +the ``rte_kni`` kernel module. + Overview -------- -The Kernel NIC Interface sample application uses two threads in user space for each physical NIC port being used, -and allocates one or more KNI device for each physical NIC port with kernel module's support. -For a physical NIC port, one thread reads from the port and writes to KNI devices, -and another thread reads from KNI devices and writes the data unmodified to the physical NIC port. -It is recommended to configure one KNI device for each physical NIC port. -If configured with more than one KNI devices for a physical NIC port, -it is just for performance testing, or it can work together with VMDq support in future. +The Kernel NIC Interface sample application ``kni`` allocates one or more +KNI interfaces for each physical NIC port. For each physical NIC port, +``kni`` uses two DPDK threads in user space; one thread reads from the port and +writes to the corresponding KNI interfaces and the other thread reads from +the KNI interfaces and writes the data unmodified to the physical NIC port. + +It is recommended to configure one KNI interface for each physical NIC port. +The application can be configured with more than one KNI interface for +each physical NIC port for performance testing or it can work together with +VMDq support in future. -The packet flow through the Kernel NIC Interface application is as shown in the following figure. +The packet flow through the Kernel NIC Interface application is as shown +in the following figure. .. _figure_kernel_nic: @@ -50,145 +59,208 @@ The packet flow through the Kernel NIC Interface application is as shown in the Kernel NIC Application Packet Flow +If link monitoring is enabled with the ``-m`` command line flag, one +additional pthread is launched which will check the link status of each +physical NIC port and will update the carrier status of the corresponding +KNI interface(s) to match the physical NIC port's state. This means that +the KNI interface(s) will be disabled automatically when the Ethernet link +goes down and enabled when the Ethernet link goes up. + +If link monitoring is enabled, the ``rte_kni`` kernel module should be +loaded such that the default carrier state is set to *off*. This ensures +that the KNI interface is only enabled *after* the Ethernet link of the +corresponding NIC port has reached the linkup state. + +If link monitoring is not enabled, the ``rte_kni`` kernel module should be +loaded with the default carrier state set to *on*. This sets the carrier +state of the KNI interfaces to *on* when the KNI interfaces are enabled +without regard to the actual link state of the corresponding NIC port. +This is useful for testing in loopback mode where the NIC port may not be +physically connected to anything. + Compiling the Application ------------------------- To compile the sample application see :doc:`compiling`. -The application is located in the ``kni`` sub-directory. +The application is located in the ``examples/kni`` sub-directory. .. note:: This application is intended as a linuxapp only. -Loading the Kernel Module -------------------------- +Running the kni Example Application +----------------------------------- -Loading the KNI kernel module without any parameter is the typical way a DPDK application -gets packets into and out of the kernel net stack. -This way, only one kernel thread is created for all KNI devices for packet receiving in kernel side: +The ``kni`` example application requires a number of command line options: .. code-block:: console - #insmod rte_kni.ko + kni [EAL options] -- -P -p PORTMASK --config="(port,lcore_rx,lcore_tx[,lcore_kthread,...])[,(port,lcore_rx,lcore_tx[,lcore_kthread,...])]" -Pinning the kernel thread to a specific core can be done using a taskset command such as following: +Where: -.. code-block:: console +* ``-P``: - #taskset -p 100000 `pgrep --fl kni_thread | awk '{print $1}'` + Set all ports to promiscuous mode so that packets are accepted regardless + of the packet's Ethernet MAC destination address. Without this option, + only packets with the Ethernet MAC destination address set to the + Ethernet address of the port are accepted. -This command line tries to pin the specific kni_thread on the 20th lcore (lcore numbering starts at 0), -which means it needs to check if that lcore is available on the board. -This command must be sent after the application has been launched, as insmod does not start the kni thread. +* ``-p PORTMASK``: -For optimum performance, -the lcore in the mask must be selected to be on the same socket as the lcores used in the KNI application. + Hexadecimal bitmask of ports to configure. -To provide flexibility of performance, the kernel module of the KNI, -located in the kmod sub-directory of the DPDK target directory, -can be loaded with parameter of kthread_mode as follows: +* ``-m``: -* #insmod rte_kni.ko kthread_mode=single + Enable monitoring and updating of the Ethernet carrier state. With this + option set, a thread will be started which will periodically check + the Ethernet link status of the physical Ethernet ports and set the + carrier state of the corresponding KNI network interface to match it. + This means that the KNI interface will be disabled automatically when + the Ethernet link goes down and enabled when the Ethernet link goes up. - This mode will create only one kernel thread for all KNI devices for packet receiving in kernel side. - By default, it is in this single kernel thread mode. - It can set core affinity for this kernel thread by using Linux command taskset. +* ``--config="(port,lcore_rx,lcore_tx[,lcore_kthread,...])[,(port,lcore_rx,lcore_tx[,lcore_kthread,...])]"``: -* #insmod rte_kni.ko kthread_mode =multiple + Determines which lcores the Rx and Tx DPDK tasks, and (optionally) + the KNI kernel thread(s) are bound to for each physical port. - This mode will create a kernel thread for each KNI device for packet receiving in kernel side. - The core affinity of each kernel thread is set when creating the KNI device. - The lcore ID for each kernel thread is provided in the command line of launching the application. - Multiple kernel thread mode can provide scalable higher performance. +Refer to *DPDK Getting Started Guide* for general information on running +applications and the Environment Abstraction Layer (EAL) options. -To measure the throughput in a loopback mode, the kernel module of the KNI, -located in the kmod sub-directory of the DPDK target directory, -can be loaded with parameters as follows: +The ``-c coremask`` or ``-l corelist`` parameter of the EAL options must +include the lcores specified by ``lcore_rx`` and ``lcore_tx`` for each port, +but does not need to include lcores specified by ``lcore_kthread`` as those +cores are used to pin the kernel threads in the ``rte_kni`` kernel module. -* #insmod rte_kni.ko lo_mode=lo_mode_fifo +The ``--config`` parameter must include a set of +``(port,lcore_rx,lcore_tx,[lcore_kthread,...])`` values for each physical +port specified in the ``-p PORTMASK`` parameter. - This loopback mode will involve ring enqueue/dequeue operations in kernel space. +The optional ``lcore_kthread`` lcore ID parameter in ``--config`` can be +specified zero, one or more times for each physical port. -* #insmod rte_kni.ko lo_mode=lo_mode_fifo_skb +If no lcore ID is specified for ``lcore_kthread``, one KNI interface will +be created for the physical port ``port`` and the KNI kernel thread(s) +will have no specific core affinity. - This loopback mode will involve ring enqueue/dequeue operations and sk buffer copies in kernel space. +If one or more lcore IDs are specified for ``lcore_kthread``, a KNI interface +will be created for each lcore ID specified, bound to the physical port +``port``. If the ``rte_kni`` kernel module is loaded in "multiple kernel +thread" mode, a kernel thread will be created for each KNI interface and +bound to the specified core. If the ``rte_kni`` kernel module is loaded in +"single kernel thread" mode, only one kernel thread is started for all KNI +interfaces. The kernel thread will be bound to the first ``lcore_kthread`` +lcore ID specified. -Running the Application ------------------------ +Example Configurations +~~~~~~~~~~~~~~~~~~~~~~~ -The application requires a number of command line options: +The following commands will first load the ``rte_kni`` kernel module in +"multiple kernel thread" mode. The ``kni`` application is then started +using two ports; Port 0 uses lcore 4 for the Rx task, lcore 6 for the Tx +task, and will create a single KNI interface ``vEth0_0`` with the kernel +thread bound to lcore 8. Port 1 uses lcore 5 for the Rx task, lcore 7 +for the Tx task, and will create a single KNI interface ``vEth1_0`` with +the kernel thread bound to lcore 9. .. code-block:: console - kni [EAL options] -- -P -p PORTMASK --config="(port,lcore_rx,lcore_tx[,lcore_kthread,...])[,port,lcore_rx,lcore_tx[,lcore_kthread,...]]" + # rmmod rte_kni + # insmod kmod/rte_kni.ko kthread_mode=multiple + # ./build/kni -l 4-7 -n 4 -- -P -p 0x3 -m --config="(0,4,6,8),(1,5,7,9)" -Where: - -* -P: Set all ports to promiscuous mode so that packets are accepted regardless of the packet's Ethernet MAC destination address. - Without this option, only packets with the Ethernet MAC destination address set to the Ethernet address of the port are accepted. +The following example is identical, except an additional ``lcore_kthread`` +core is specified per physical port. In this case, ``kni`` will create +four KNI interfaces: ``vEth0_0``/``vEth0_1`` bound to physical port 0 and +``vEth1_0``/``vEth1_1`` bound to physical port 1. -* -p PORTMASK: Hexadecimal bitmask of ports to configure. +The kernel thread for each interface will be bound as follows: -* --config="(port,lcore_rx, lcore_tx[,lcore_kthread, ...]) [, port,lcore_rx, lcore_tx[,lcore_kthread, ...]]": - Determines which lcores of RX, TX, kernel thread are mapped to which ports. + * ``vEth0_0`` - bound to lcore 8. + * ``vEth0_1`` - bound to lcore 10. + * ``vEth1_0`` - bound to lcore 9. + * ``vEth1_1`` - bound to lcore 11 -Refer to *DPDK Getting Started Guide* for general information on running applications and the Environment Abstraction Layer (EAL) options. +.. code-block:: console -The -c coremask or -l corelist parameter of the EAL options should include the lcores indicated by the lcore_rx and lcore_tx, -but does not need to include lcores indicated by lcore_kthread as they are used to pin the kernel thread on. -The -p PORTMASK parameter should include the ports indicated by the port in --config, neither more nor less. + # rmmod rte_kni + # insmod kmod/rte_kni.ko kthread_mode=multiple + # ./build/kni -l 4-7 -n 4 -- -P -p 0x3 -m --config="(0,4,6,8,10),(1,5,7,9,11)" -The lcore_kthread in --config can be configured none, one or more lcore IDs. -In multiple kernel thread mode, if configured none, a KNI device will be allocated for each port, -while no specific lcore affinity will be set for its kernel thread. -If configured one or more lcore IDs, one or more KNI devices will be allocated for each port, -while specific lcore affinity will be set for its kernel thread. -In single kernel thread mode, if configured none, a KNI device will be allocated for each port. -If configured one or more lcore IDs, -one or more KNI devices will be allocated for each port while -no lcore affinity will be set as there is only one kernel thread for all KNI devices. +The following example can be used to test the interface between the ``kni`` +test application and the ``rte_kni`` kernel module. In this example, the +``rte_kni`` kernel module is loaded in single kernel thread mode, loopback +mode enabled, and the default carrier state is set to *on* so that the +corresponding physical NIC port does not have to be connected in order to +use the KNI interface. One KNI interface ``vEth0_0`` is created for port 0 +and one KNI interface ``vEth1_0`` is created for port 1. Since ``rte_kni`` +is loaded in "single kernel thread" mode, the one kernel thread is bound +to lcore 8. -For example, to run the application with two ports served by six lcores, one lcore of RX, one lcore of TX, -and one lcore of kernel thread for each port: +Since the physical NIC ports are not being used, link monitoring can be +disabled by **not** specifying the ``-m`` flag to ``kni``: .. code-block:: console - ./build/kni -l 4-7 -n 4 -- -P -p 0x3 --config="(0,4,6,8),(1,5,7,9)" + # rmmod rte_kni + # insmod kmod/rte_kni.ko lo_mode=lo_mode_fifo carrier=on + # ./build/kni -l 4-7 -n 4 -- -P -p 0x3 --config="(0,4,6,8),(1,5,7,9)" KNI Operations -------------- -Once the KNI application is started, one can use different Linux* commands to manage the net interfaces. -If more than one KNI devices configured for a physical port, -only the first KNI device will be paired to the physical device. -Operations on other KNI devices will not affect the physical port handled in user space application. +Once the ``kni`` application is started, the user can use the normal +Linux commands to manage the KNI interfaces as if they were any other +Linux network interface. -Assigning an IP address: +Enable KNI interface and assign an IP address: .. code-block:: console - #ifconfig vEth0_0 192.168.0.1 + # ifconfig vEth0_0 192.168.0.1 -Displaying the NIC registers: +Show KNI interface configuration and statistics: .. code-block:: console - #ethtool -d vEth0_0 + # ifconfig vEth0_0 -Dumping the network traffic: +Dump network traffic: .. code-block:: console - #tcpdump -i vEth0_0 + # tcpdump -i vEth0_0 + +The normal Linux commands can also be used to change the MAC address and +MTU size used by the physical NIC which corresponds to the KNI interface. +However, if more than one KNI interface is configured for a physical port, +these commands will only work on the first KNI interface for that port. Change the MAC address: .. code-block:: console - #ifconfig vEth0_0 hw ether 0C:01:02:03:04:08 + # ifconfig vEth0_0 hw ether 0C:01:02:03:04:08 + +Change the MTU size: + +.. code-block:: console + + # ifconfig vEth0_0 mtu 1450 + +If DPDK is compiled with ``CONFIG_RTE_KNI_KMOD_ETHTOOL=y`` and an Intel +NIC is used, the user can use ``ethtool`` on the KNI interface as if it +were a normal Linux kernel interface. + +Displaying the NIC registers: + +.. code-block:: console + + # ethtool -d vEth0_0 -When the DPDK userspace application is closed, all the KNI devices are deleted from Linux*. +When the ``kni`` application is closed, all the KNI interfaces are deleted +from the Linux kernel. Explanation ----------- @@ -227,7 +299,7 @@ to see if this lcore is reading from or writing to kernel NIC interfaces. For the case that reads from a NIC port and writes to the kernel NIC interfaces (``kni_ingress``), the packet reception is the same as in L2 Forwarding sample application (see :ref:`l2_fwd_app_rx_tx_packets`). -The packet transmission is done by sending mbufs into the kernel NIC interfaces by rte_kni_tx_burst(). +The packet transmission is done by sending mbufs into the kernel NIC interfaces by ``rte_kni_tx_burst()``. The KNI library automatically frees the mbufs after the kernel successfully copied the mbufs. For the other case that reads from kernel NIC interfaces @@ -235,16 +307,3 @@ and writes to a physical NIC port (``kni_egress``), packets are retrieved by reading mbufs from kernel NIC interfaces by ``rte_kni_rx_burst()``. The packet transmission is the same as in the L2 Forwarding sample application (see :ref:`l2_fwd_app_rx_tx_packets`). - -Callbacks for Kernel Requests -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -To execute specific PMD operations in user space requested by some Linux* commands, -callbacks must be implemented and filled in the struct rte_kni_ops structure. -Currently, setting a new MTU, change in MAC address, configuring promiscusous mode and -configuring the network interface(up/down) re supported. -Default implementation for following is available in rte_kni library. -Application may choose to not implement following callbacks: - -- ``config_mac_address`` -- ``config_promiscusity`` diff --git a/examples/kni/Makefile b/examples/kni/Makefile index 7e19d2e2a..dd90d7d73 100644 --- a/examples/kni/Makefile +++ b/examples/kni/Makefile @@ -20,6 +20,7 @@ static: build/$(APP)-static PC_FILE := $(shell pkg-config --path libdpdk) CFLAGS += -O3 $(shell pkg-config --cflags libdpdk) +CFLAGS += -DALLOW_EXPERIMENTAL_API LDFLAGS_SHARED = $(shell pkg-config --libs libdpdk) LDFLAGS_STATIC = -Wl,-Bstatic $(shell pkg-config --static --libs libdpdk) @@ -54,6 +55,7 @@ please change the definition of the RTE_TARGET environment variable) endif CFLAGS += -O3 +CFLAGS += -DALLOW_EXPERIMENTAL_API CFLAGS += $(WERROR_FLAGS) include $(RTE_SDK)/mk/rte.extapp.mk diff --git a/examples/kni/main.c b/examples/kni/main.c index 80c401c51..4fda51713 100644 --- a/examples/kni/main.c +++ b/examples/kni/main.c @@ -106,6 +106,8 @@ static struct rte_mempool * pktmbuf_pool = NULL; static uint32_t ports_mask = 0; /* Ports set in promiscuous mode off by default. */ static int promiscuous_on = 0; +/* Monitor link status continually. off by default. */ +static int monitor_links; /* Structure type for recording kni interface specific stats */ struct kni_interface_stats { @@ -325,11 +327,12 @@ main_loop(__rte_unused void *arg) static void print_usage(const char *prgname) { - RTE_LOG(INFO, APP, "\nUsage: %s [EAL options] -- -p PORTMASK -P " + RTE_LOG(INFO, APP, "\nUsage: %s [EAL options] -- -p PORTMASK -P -m " "[--config (port,lcore_rx,lcore_tx,lcore_kthread...)" "[,(port,lcore_rx,lcore_tx,lcore_kthread...)]]\n" " -p PORTMASK: hex bitmask of ports to use\n" " -P : enable promiscuous mode\n" + " -m : enable monitoring of port carrier state\n" " --config (port,lcore_rx,lcore_tx,lcore_kthread...): " "port and lcore configurations\n", prgname); @@ -510,7 +513,7 @@ parse_args(int argc, char **argv) opterr = 0; /* Parse command line */ - while ((opt = getopt_long(argc, argv, "p:P", longopts, + while ((opt = getopt_long(argc, argv, "p:Pm", longopts, &longindex)) != EOF) { switch (opt) { case 'p': @@ -519,6 +522,9 @@ parse_args(int argc, char **argv) case 'P': promiscuous_on = 1; break; + case 'm': + monitor_links = 1; + break; case 0: if (!strncmp(longopts[longindex].name, CMDLINE_OPT_CONFIG, @@ -674,6 +680,55 @@ check_all_ports_link_status(uint32_t port_mask) } } +static void +log_link_state(struct rte_kni *kni, int prev, struct rte_eth_link *link) +{ + if (kni == NULL || link == NULL) + return; + + if (prev == ETH_LINK_DOWN && link->link_status == ETH_LINK_UP) { + RTE_LOG(INFO, APP, "%s NIC Link is Up %d Mbps %s %s.\n", + rte_kni_get_name(kni), + link->link_speed, + link->link_autoneg ? "(AutoNeg)" : "(Fixed)", + link->link_duplex ? "Full Duplex" : "Half Duplex"); + } else if (prev == ETH_LINK_UP && link->link_status == ETH_LINK_DOWN) { + RTE_LOG(INFO, APP, "%s NIC Link is Down.\n", + rte_kni_get_name(kni)); + } +} + +/* + * Monitor the link status of all ports and update the + * corresponding KNI interface(s) + */ +static void * +monitor_all_ports_link_status(void *arg) +{ + uint16_t portid; + struct rte_eth_link link; + unsigned int i; + struct kni_port_params **p = kni_port_params_array; + int prev; + (void) arg; + + while (monitor_links) { + rte_delay_ms(500); + RTE_ETH_FOREACH_DEV(portid) { + if ((ports_mask & (1 << portid)) == 0) + continue; + memset(&link, 0, sizeof(link)); + rte_eth_link_get_nowait(portid, &link); + for (i = 0; i < p[portid]->nb_kni; i++) { + prev = rte_kni_update_link(p[portid]->kni[i], + link.link_status); + log_link_state(p[portid]->kni[i], prev, &link); + } + } + } + return NULL; +} + /* Callback for request of changing MTU */ static int kni_change_mtu(uint16_t port_id, unsigned int new_mtu) @@ -893,6 +948,8 @@ main(int argc, char** argv) int ret; uint16_t nb_sys_ports, port; unsigned i; + void *retval; + pthread_t kni_link_tid; /* Associate signal_hanlder function with USR signals */ signal(SIGUSR1, signal_handler); @@ -949,12 +1006,21 @@ main(int argc, char** argv) } check_all_ports_link_status(ports_mask); + ret = rte_ctrl_thread_create(&kni_link_tid, + "KNI link status check", NULL, + monitor_all_ports_link_status, NULL); + if (ret < 0) + rte_exit(EXIT_FAILURE, + "Could not create link status thread!\n"); + /* Launch per-lcore function on every lcore */ rte_eal_mp_remote_launch(main_loop, NULL, CALL_MASTER); RTE_LCORE_FOREACH_SLAVE(i) { if (rte_eal_wait_lcore(i) < 0) return -1; } + monitor_links = 0; + pthread_join(kni_link_tid, &retval); /* Release resources */ RTE_ETH_FOREACH_DEV(port) { From patchwork Fri Oct 19 00:23:57 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Gora X-Patchwork-Id: 47059 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 95CFC1B142; Fri, 19 Oct 2018 02:24:43 +0200 (CEST) Received: from mail-qt1-f195.google.com (mail-qt1-f195.google.com [209.85.160.195]) by dpdk.org (Postfix) with ESMTP id 801307D05 for ; Fri, 19 Oct 2018 02:24:42 +0200 (CEST) Received: by mail-qt1-f195.google.com with SMTP id b4-v6so36472394qtc.7 for ; Thu, 18 Oct 2018 17:24:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=1W+gCUuhLSwf9fXApqIyFdqnIzxHZWjNFsIVtIx0sc8=; b=P7Ux9qcr50GVv35yREQXIrplHsTXudiF0xcI4ttJ4W7DfNp0nknMSoLkI2KKeVHTdJ MR2D8R83drVBO8j817dvCYKXs0JQYHhfZKEMpwj7ikAZp4fi4U6rANWWgZYQqODw7BLg aJ2MHVFHj6QTssNXHDM7n/757x3BFtdGVklWgl+Uka4I9iNg08FIj809xiw47q5AiPkd 6F0RxVPSmDmfJRWSLbTB3dyVO96Rmum+rOvrN4bugRiX/XnkcACnm5qkZDo5UaRjE9b5 JgS2lrDpV0xlb3nYnQtl5TqPnTyrltoQuIuV5zlakLQIusynS9F4M3GwxVdtffWyUw/r D5FA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references:mime-version:content-transfer-encoding; bh=1W+gCUuhLSwf9fXApqIyFdqnIzxHZWjNFsIVtIx0sc8=; b=HwZQt7YVQv3FU3sjK+PCIR5iMqbNdtSNz9mwRYJeXd6S3xICgcsWXiZhY0tE7Teq71 0WFSy0anN2a6kmv2fgsfsHcTb7VYF8rNfYUvI1pIg1QcC2IstxMjwdkyELzUgr4LzyRJ SzI+hXo7unZxotzDUzE1ZsdI6HrT2MCwssP1wgrRlVkZmEZynOCsks5Z+H606LgP2O3X 8Z1ktfu9pvnI/LvFCJTU70HGNV5HtjfIv39foudBPjM2XxPVCadhfVDbfW1NOwDFHGkZ quyU56HPXO/kRv23t1DG2sICePu9sIIQgj5sndlRULUxs06XYCzoF5ccx897IPdXcnid APsA== X-Gm-Message-State: ABuFfoh3g5SYs/Qb9wKud14RtDQVTeLc1v9UbK4ENKR7xUk4VU/WTJw/ wmKvaV8ndeaLQSFfPGfGXbCmx0Ri X-Google-Smtp-Source: ACcGV62fOKxwLVUlTcLc+5ZguotCLgLsm6A0+ihZ6rF0NzmfPQHD3BinZVAaq5yQp35siZY9pBTDkA== X-Received: by 2002:ac8:21fd:: with SMTP id 58-v6mr29887303qtz.306.1539908681572; Thu, 18 Oct 2018 17:24:41 -0700 (PDT) Received: from snappy.local.lan ([191.205.43.134]) by smtp.gmail.com with ESMTPSA id y124-v6sm12963926qke.22.2018.10.18.17.24.39 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 18 Oct 2018 17:24:40 -0700 (PDT) From: Dan Gora To: dev@dpdk.org Cc: Igor Ryzhov , Stephen Hemminger , Ferruh Yigit , Dan Gora Date: Thu, 18 Oct 2018 21:23:57 -0300 Message-Id: <20181019002358.17132-5-dg@adax.com> X-Mailer: git-send-email 2.19.0 In-Reply-To: <20181019002358.17132-1-dg@adax.com> References: <20180911232906.18352-1-dg@adax.com> <20181019002358.17132-1-dg@adax.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v5 4/5] examples/kni: add log msgs to show and clear stats X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add logging messages showing the commands necessary for the user to have the application display and zero the statistics. Signed-off-by: Dan Gora --- examples/kni/main.c | 16 +++++++++++++--- 1 file changed, 13 insertions(+), 3 deletions(-) diff --git a/examples/kni/main.c b/examples/kni/main.c index 4fda51713..0e3b2a2f7 100644 --- a/examples/kni/main.c +++ b/examples/kni/main.c @@ -171,14 +171,13 @@ signal_handler(int signum) /* When we receive a USR2 signal, reset stats */ if (signum == SIGUSR2) { memset(&kni_stats, 0, sizeof(kni_stats)); - printf("\n**Statistics have been reset**\n"); + printf("\n** Statistics have been reset **\n"); return; } /* When we receive a RTMIN or SIGINT signal, stop kni processing */ if (signum == SIGRTMIN || signum == SIGINT){ - printf("SIGRTMIN is received, and the KNI processing is " - "going to stop\n"); + printf("\nSIGRTMIN/SIGINT received. KNI processing stopping.\n"); rte_atomic32_inc(&kni_stop); return; } @@ -950,6 +949,7 @@ main(int argc, char** argv) unsigned i; void *retval; pthread_t kni_link_tid; + int pid; /* Associate signal_hanlder function with USR signals */ signal(SIGUSR1, signal_handler); @@ -1006,6 +1006,16 @@ main(int argc, char** argv) } check_all_ports_link_status(ports_mask); + pid = getpid(); + RTE_LOG(INFO, APP, "========================\n"); + RTE_LOG(INFO, APP, "KNI Running\n"); + RTE_LOG(INFO, APP, "kill -SIGUSR1 %d\n", pid); + RTE_LOG(INFO, APP, " Show KNI Statistics.\n"); + RTE_LOG(INFO, APP, "kill -SIGUSR2 %d\n", pid); + RTE_LOG(INFO, APP, " Zero KNI Statistics.\n"); + RTE_LOG(INFO, APP, "========================\n"); + fflush(stdout); + ret = rte_ctrl_thread_create(&kni_link_tid, "KNI link status check", NULL, monitor_all_ports_link_status, NULL); From patchwork Fri Oct 19 00:23:58 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Gora X-Patchwork-Id: 47060 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 365321B136; Fri, 19 Oct 2018 02:24:51 +0200 (CEST) Received: from mail-qk1-f195.google.com (mail-qk1-f195.google.com [209.85.222.195]) by dpdk.org (Postfix) with ESMTP id 4E0501B136 for ; Fri, 19 Oct 2018 02:24:49 +0200 (CEST) Received: by mail-qk1-f195.google.com with SMTP id x8-v6so20027921qka.4 for ; Thu, 18 Oct 2018 17:24:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=+6LWhTYsYoXPIT940Q4sXLq6n9xxmxPAOZVAMSdxNcE=; b=jROYTLdS9ZbxCgodQ0gwlNg0/0Cl3FtobVn2mClMJfR+vlnuuwS1Phj7yQ79OpdgRY djVFABAVtvLO08cLOk0EngoBWuYlF/7eYazpSE9RANVGA9tBlhKyDGg75EhkjTs+ZZjP B9BqON2W6pLdEt3y9CqT6QBKLWt6pq0To3QEP4PZYefR81mR45z7ti8ESEnRTq/+UFVl LMBV+R/2KMwgPyK8jB1UY8t5h/TC9Sq9hIyUYz7DpQhvAwz2Wxa2n1h+OZxylh0496yV 1DpAOUCCEkVnyCnwxrYG9e9anLM4/NigOJWijxfb36PPCQYVUiZxPdeqo4/BnKXO2RNK FxJA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references:mime-version:content-transfer-encoding; bh=+6LWhTYsYoXPIT940Q4sXLq6n9xxmxPAOZVAMSdxNcE=; b=sEIeWThAqhb58Fv/KRSPx/WPoCxwiWdlRy9MHnDGAvrNR+W5eJs7kK7zur+mtok+Yr FjeChatSHT1rKmci6KSegsjPEQ+jWbEEZliQK1tjUPG//ENi0Aq5jeJgTIczg+VylASm xFyXK7r9HHtB0bmWNjPh64dkiCTWweeNkVBfoID/6qqb0HNvYbvHt0+etHTR//kpbkSr +RIyAhaiuPVVTVbndGq3bF6h6fnRJLPvEBaux7VkuIf1gHirtnDKaKbji7EUoM//wjNT YHPztUR2YwnlC8PQtz15q2PbTY/QqtV49Y8uXydaLzrf4080AWzi4EWPSD2waWoid3sG Cj4g== X-Gm-Message-State: AGRZ1gLklT1dzVidIGZFXlk5fK4GGiDzqG7jC4MU+aEE51dN3rHc0VLi cEjIdUv/L6Uz3HGtu24gqZXuQhEY X-Google-Smtp-Source: ACcGV60nUScUbNbXGFUliaoluxz7AVnSk4RpoYmuAgNCCtA7z7i+NxLjqteTuYHwMLUUTBPQY8j3fQ== X-Received: by 2002:ae9:ee02:: with SMTP id i2-v6mr1277957qkg.0.1539908688307; Thu, 18 Oct 2018 17:24:48 -0700 (PDT) Received: from snappy.local.lan ([191.205.43.134]) by smtp.gmail.com with ESMTPSA id y124-v6sm12963926qke.22.2018.10.18.17.24.45 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 18 Oct 2018 17:24:47 -0700 (PDT) From: Dan Gora To: dev@dpdk.org Cc: Igor Ryzhov , Stephen Hemminger , Ferruh Yigit , Dan Gora Date: Thu, 18 Oct 2018 21:23:58 -0300 Message-Id: <20181019002358.17132-6-dg@adax.com> X-Mailer: git-send-email 2.19.0 In-Reply-To: <20181019002358.17132-1-dg@adax.com> References: <20180911232906.18352-1-dg@adax.com> <20181019002358.17132-1-dg@adax.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v5 5/5] examples/kni: improve zeroing statistics X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" The worker threads incrementing the rx/tx_packets race with the signal handler from the main thread zeroing the entire statistics structure. This can cause the statistics to fail to be zeroed, even when there is no traffic on those interfaces. Improve zeroing the statistics by only incrementing rx/tx_packets in worker threads by a non-zero amount. This limits the race to the periods in which traffic is actually being received or transmitted. Signed-off-by: Dan Gora --- examples/kni/main.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/examples/kni/main.c b/examples/kni/main.c index 0e3b2a2f7..e37b1ad36 100644 --- a/examples/kni/main.c +++ b/examples/kni/main.c @@ -223,7 +223,8 @@ kni_ingress(struct kni_port_params *p) } /* Burst tx to kni */ num = rte_kni_tx_burst(p->kni[i], pkts_burst, nb_rx); - kni_stats[port_id].rx_packets += num; + if (num) + kni_stats[port_id].rx_packets += num; rte_kni_handle_request(p->kni[i]); if (unlikely(num < nb_rx)) { @@ -260,7 +261,8 @@ kni_egress(struct kni_port_params *p) } /* Burst tx to eth */ nb_tx = rte_eth_tx_burst(port_id, 0, pkts_burst, (uint16_t)num); - kni_stats[port_id].tx_packets += nb_tx; + if (nb_tx) + kni_stats[port_id].tx_packets += nb_tx; if (unlikely(nb_tx < num)) { /* Free mbufs not tx to NIC */ kni_burst_free_mbufs(&pkts_burst[nb_tx], num - nb_tx);