From patchwork Wed Oct 24 22:26:27 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Gora X-Patchwork-Id: 47383 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id EA40F4C9C; Thu, 25 Oct 2018 00:26:46 +0200 (CEST) Received: from mail-qt1-f196.google.com (mail-qt1-f196.google.com [209.85.160.196]) by dpdk.org (Postfix) with ESMTP id 1E3994C8A for ; Thu, 25 Oct 2018 00:26:45 +0200 (CEST) Received: by mail-qt1-f196.google.com with SMTP id j46-v6so7575693qtc.9 for ; Wed, 24 Oct 2018 15:26:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=GU+Ae+jbMqhbgYDk08eZMjAQViAixUhJ/KkiUwuLcxA=; b=J6N3uXOfhKF7D+eFNdUC/T5V+6gNuhQZdMOskcDmJJvRDsXNCUKm+FQSyyT5lXn/rw DE7ZKbSsKTvlCCI3x486z2eHQQZgWXjAPIIdxTfFQVdicoPnFUdZoab0QgAMUQZykn0s f9hw+hR0YxUoz0hkE6e+uRtgkAfLd62xbDoutINmT6k+eAPAurKdrt9qdKSze2znCWpC HlI6DX4boWBwZXikWLgj77AGB6a3wNYIKNTRoUprAHnwrpYqwwObQxq5TJkCfJusOqgN 09oGDRNim1OaHUSW+PSpKGJruxXsV1IRHld/TcW2ft4ZJflKlNuDWDq0pa56/dLyk7Dg U9IA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references:mime-version:content-transfer-encoding; bh=GU+Ae+jbMqhbgYDk08eZMjAQViAixUhJ/KkiUwuLcxA=; b=pQG9GSVwLfxdWl4JfnX/QQHGk1HVYNRrE3F3czlRJwbR9tdwcQK0Un5305mpAZ/u8Y PSo1ophfuB1+iURHMlcmglSVM/KXr0nEQONqpFZUmbd5HIdY9otg9Aj10sjn3vAII+Um aWvmwzgOr2yH9EnxnVJs+3EkqCIjKMYuvq7c+5qXQxpQifvb7QGQ//7g99nrpH6EVeJH bebly24OUO3J1NFVusOpEv1sYmfG2GF+QVrf4pJdxYAIH4HTJhAKuy8XHla9YbMQ8GZ9 K+fMslO1vBSVesQvbEEAm458AKuv1WJS2GH0aAnhrGWLxNAP8xB1iVi/c9zXf6hdPokm NxwQ== X-Gm-Message-State: AGRZ1gLwtd81Xf2spwa8DsFjlBpvr5UUi1oVkKvkMQWNQrlUv3b/9269 wgKNugooAVve0wJAX56RYsoFToqF X-Google-Smtp-Source: AJdET5eFtYefweq4D0Ui0AWMfv3nwtCgkeXIVbKT+tA5jc6owxXkYVlG/t4lcYm5KT47jdTADNYKew== X-Received: by 2002:ac8:22ed:: with SMTP id g42-v6mr4466365qta.105.1540420003957; Wed, 24 Oct 2018 15:26:43 -0700 (PDT) Received: from snappy.local.lan ([179.228.213.114]) by smtp.gmail.com with ESMTPSA id s52-v6sm5506026qts.5.2018.10.24.15.26.40 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Wed, 24 Oct 2018 15:26:42 -0700 (PDT) From: Dan Gora To: dev@dpdk.org Cc: Igor Ryzhov , Stephen Hemminger , Ferruh Yigit , Dan Gora Date: Wed, 24 Oct 2018 19:26:27 -0300 Message-Id: <20181024222631.30461-2-dg@adax.com> X-Mailer: git-send-email 2.19.0 In-Reply-To: <20181024222631.30461-1-dg@adax.com> References: <20180911232906.18352-1-dg@adax.com> <20181024222631.30461-1-dg@adax.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v7 1/5] kni: add API to set link status on kernel interface X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add a new API function to KNI, rte_kni_update_link() to allow DPDK applications to update the link status for KNI network interfaces in the linux kernel. Signed-off-by: Dan Gora --- lib/librte_kni/rte_kni.c | 41 ++++++++++++++++ lib/librte_kni/rte_kni.h | 20 ++++++++ lib/librte_kni/rte_kni_version.map | 6 +++ test/test/test_kni.c | 77 ++++++++++++++++++++++++++++++ 4 files changed, 144 insertions(+) diff --git a/lib/librte_kni/rte_kni.c b/lib/librte_kni/rte_kni.c index b8edd40f4..c9726d4f8 100644 --- a/lib/librte_kni/rte_kni.c +++ b/lib/librte_kni/rte_kni.c @@ -717,6 +717,47 @@ rte_kni_unregister_handlers(struct rte_kni *kni) return 0; } + +int __rte_experimental +rte_kni_update_link(struct rte_kni *kni, unsigned int linkup) +{ + char path[64]; + char old_carrier[2]; + const char *new_carrier; + int old_linkup; + int fd, ret; + + if (kni == NULL) + return -1; + + snprintf(path, sizeof(path), "/sys/devices/virtual/net/%s/carrier", + kni->name); + + fd = open(path, O_RDWR); + if (fd == -1) { + RTE_LOG(ERR, KNI, "Failed to open file: %s.\n", path); + return -1; + } + + ret = read(fd, old_carrier, 2); + if (ret < 1) { + close(fd); + return -1; + } + old_linkup = (old_carrier[0] == '1'); + + new_carrier = linkup ? "1" : "0"; + ret = write(fd, new_carrier, 1); + if (ret < 1) { + RTE_LOG(ERR, KNI, "Failed to write file: %s.\n", path); + close(fd); + return -1; + } + + close(fd); + return old_linkup; +} + void rte_kni_close(void) { diff --git a/lib/librte_kni/rte_kni.h b/lib/librte_kni/rte_kni.h index 601abdfc6..02ca43b4b 100644 --- a/lib/librte_kni/rte_kni.h +++ b/lib/librte_kni/rte_kni.h @@ -232,6 +232,26 @@ int rte_kni_register_handlers(struct rte_kni *kni, struct rte_kni_ops *ops); */ int rte_kni_unregister_handlers(struct rte_kni *kni); +/** + * Update link carrier state for KNI port. + * + * Update the linkup/linkdown state of a KNI interface in the kernel. + * + * @param kni + * pointer to struct rte_kni. + * @param linkup + * New link state: + * 0 for linkdown. + * > 0 for linkup. + * + * @return + * On failure: -1 + * Previous link state == linkdown: 0 + * Previous link state == linkup: 1 + */ +int __rte_experimental +rte_kni_update_link(struct rte_kni *kni, unsigned int linkup); + /** * Close KNI device. */ diff --git a/lib/librte_kni/rte_kni_version.map b/lib/librte_kni/rte_kni_version.map index acd515eb0..c877dc6aa 100644 --- a/lib/librte_kni/rte_kni_version.map +++ b/lib/librte_kni/rte_kni_version.map @@ -15,3 +15,9 @@ DPDK_2.0 { local: *; }; + +EXPERIMENTAL { + global: + + rte_kni_update_link; +}; diff --git a/test/test/test_kni.c b/test/test/test_kni.c index cf64c0b27..e38206905 100644 --- a/test/test/test_kni.c +++ b/test/test/test_kni.c @@ -118,6 +118,79 @@ kni_change_mtu(uint16_t port_id, unsigned int new_mtu) port_id, kni_pkt_mtu); return 0; } + +static int +test_kni_link_change(void) +{ + int ret; + int pid; + + pid = fork(); + if (pid < 0) { + printf("Error: Failed to fork a process\n"); + return -1; + } + + if (pid == 0) { + printf("Starting KNI Link status change tests.\n"); + if (system(IFCONFIG TEST_KNI_PORT" up") == -1) { + ret = -1; + goto error; + } + + ret = rte_kni_update_link(test_kni_ctx, 1); + if (ret < 0) { + printf("Failed to change link state to Up ret=%d.\n", + ret); + goto error; + } + rte_delay_ms(1000); + printf("KNI: Set LINKUP, previous state=%d\n", ret); + + ret = rte_kni_update_link(test_kni_ctx, 0); + if (ret != 1) { + printf( + "Failed! Previous link state should be 1, returned %d.\n", + ret); + goto error; + } + rte_delay_ms(1000); + printf("KNI: Set LINKDOWN, previous state=%d\n", ret); + + ret = rte_kni_update_link(test_kni_ctx, 1); + if (ret != 0) { + printf( + "Failed! Previous link state should be 0, returned %d.\n", + ret); + goto error; + } + printf("KNI: Set LINKUP, previous state=%d\n", ret); + + ret = 0; + rte_delay_ms(1000); + +error: + if (system(IFCONFIG TEST_KNI_PORT" down") == -1) + ret = -1; + + printf("KNI: Link status change tests: %s.\n", + (ret == 0) ? "Passed" : "Failed"); + exit(ret); + } else { + int p_ret, status; + + while (1) { + p_ret = waitpid(pid, &status, WNOHANG); + if (p_ret != 0) { + if (WIFEXITED(status)) + return WEXITSTATUS(status); + return -1; + } + rte_delay_ms(10); + rte_kni_handle_request(test_kni_ctx); + } + } +} /** * This loop fully tests the basic functions of KNI. e.g. transmitting, * receiving to, from kernel space, and kernel requests. @@ -401,6 +474,10 @@ test_kni_processing(uint16_t port_id, struct rte_mempool *mp) goto fail_kni; } + ret = test_kni_link_change(); + if (ret != 0) + goto fail_kni; + rte_eal_mp_remote_launch(test_kni_loop, NULL, CALL_MASTER); RTE_LCORE_FOREACH_SLAVE(i) { if (rte_eal_wait_lcore(i) < 0) { From patchwork Wed Oct 24 22:26:28 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Gora X-Patchwork-Id: 47384 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 0F92A4CE4; Thu, 25 Oct 2018 00:26:50 +0200 (CEST) Received: from mail-qt1-f193.google.com (mail-qt1-f193.google.com [209.85.160.193]) by dpdk.org (Postfix) with ESMTP id AD2FF4CA7 for ; Thu, 25 Oct 2018 00:26:48 +0200 (CEST) Received: by mail-qt1-f193.google.com with SMTP id o17-v6so1898259qtq.11 for ; Wed, 24 Oct 2018 15:26:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=qGxIO6HcSR2K8WNFCTP2d2Q8QcUom+A780a6KywQzKc=; b=bE1S0O8AUf9oCeYR4aYhVsFvzWk5TMvgNxObXzZd3Qd+7cL4T2Gcr6QDJI9LzB/DOk QiMbPNT9WaSrvtHwwFVBn3TQsxux8ZsusIVUlKvZP/BOFdtUuth4BdwxUEu2m1G+8MH3 48jv14Dl3o3if1bBocqgdlXyEdVhw9s5pGj0IXXl8Y0T6RnuNGonxkiiySQwqvnV6cu5 xvcLqplMJE770zU7mW6+CEHeIGbFw5yBOM8Q2RtoTxIR7fOwfyoUJdVan35r3Eie7NQr Ma9ZlbPoVusw+R9qMrKoLcVqWnXLcR8ZNK0lyMZiu7qtDTbRKWfw07udmWhPJW6gX96V p2vw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references:mime-version:content-transfer-encoding; bh=qGxIO6HcSR2K8WNFCTP2d2Q8QcUom+A780a6KywQzKc=; b=sad3hU+Wn67XmYSwInSrlJmEctCRvjZHQXTYaMXw8lECrzs5gn6zywomT0UZ4Zsk+P 1zhKmW8ki8mp6mtk0YcoF4iaUuDUxuzR4H17R5YkoEfnHNKnkPB0Yzvy/dFF3XpZV38w dPCzlDl+qpNKcrEvuPUIO3kLBwoDRsb841+i4iv4v7Q1kX17K+i7YdA3CASZhB0xWUDc 3fzioEdDuDLn9BJRqv/iVcoXs+JWMMiVTGrRgVY8TKHICiczWuoHU0xB8fXt1tLgDCFp s8rfvrXpon/UBlS4T5cIiWjLzTlulwOZbXEcz4936GBfFmD45gKjlHf0cZughAtSmZ2n LHFw== X-Gm-Message-State: AGRZ1gIEcyb5dYZdGo8ynn+gbCBB19in+JhlAdLnxBR/an9lliwsKi8c gfZEdUonEdhNabE9sXvJD/AXkhsn X-Google-Smtp-Source: AJdET5f1deHeL175etzog+483md6uvD7nhIWSAu5gNs1cO07Z0LhfJ35v8wpHGU+ROkm9LOqKetvVQ== X-Received: by 2002:aed:26c7:: with SMTP id q65-v6mr4417430qtd.282.1540420007654; Wed, 24 Oct 2018 15:26:47 -0700 (PDT) Received: from snappy.local.lan ([179.228.213.114]) by smtp.gmail.com with ESMTPSA id s52-v6sm5506026qts.5.2018.10.24.15.26.44 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Wed, 24 Oct 2018 15:26:46 -0700 (PDT) From: Dan Gora To: dev@dpdk.org Cc: Igor Ryzhov , Stephen Hemminger , Ferruh Yigit , Dan Gora , John McNamara , Marko Kovacevic Date: Wed, 24 Oct 2018 19:26:28 -0300 Message-Id: <20181024222631.30461-3-dg@adax.com> X-Mailer: git-send-email 2.19.0 In-Reply-To: <20181024222631.30461-1-dg@adax.com> References: <20180911232906.18352-1-dg@adax.com> <20181024222631.30461-1-dg@adax.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v7 2/5] kni: set default carrier state of interface X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add module parameter 'carrier='on|off' to set the default carrier state for linux network interfaces created by the KNI module. The default carrier state is 'off'. For KNI interfaces which need to reflect the carrier state of a physical Ethernet port controlled by the DPDK application, the default carrier state should be left set to 'off'. The application can set the carrier state of the KNI interface to reflect the state of the physical Ethernet port using rte_kni_update_link(). For KNI interfaces which are purely virtual, the default carrier state can be set to 'on'. This enables the KNI interface to be used without having to explicity set the carrier state to 'on' using rte_kni_update_link(). Signed-off-by: Dan Gora --- .../prog_guide/kernel_nic_interface.rst | 239 ++++++++++++++---- kernel/linux/kni/kni_dev.h | 3 + kernel/linux/kni/kni_misc.c | 60 ++++- kernel/linux/kni/kni_net.c | 5 + 4 files changed, 254 insertions(+), 53 deletions(-) diff --git a/doc/guides/prog_guide/kernel_nic_interface.rst b/doc/guides/prog_guide/kernel_nic_interface.rst index 8fa13fa1a..33ea980e5 100644 --- a/doc/guides/prog_guide/kernel_nic_interface.rst +++ b/doc/guides/prog_guide/kernel_nic_interface.rst @@ -29,58 +29,222 @@ The components of an application using the DPDK Kernel NIC Interface are shown i The DPDK KNI Kernel Module -------------------------- -The KNI kernel loadable module provides support for two types of devices: +The KNI kernel loadable module ``rte_kni`` provides the kernel interface +for DPDK applications. -* A Miscellaneous device (/dev/kni) that: +When the ``rte_kni`` module is loaded, it will create a device ``/dev/kni`` +that is used by the DPDK KNI API functions to control and communicate with +the kernel module. - * Creates net devices (via ioctl calls). +The ``rte_kni`` kernel module contains several optional parameters which +can be specified when the module is loaded to control its behavior: - * Maintains a kernel thread context shared by all KNI instances - (simulating the RX side of the net driver). +.. code-block:: console - * For single kernel thread mode, maintains a kernel thread context shared by all KNI instances - (simulating the RX side of the net driver). + # modinfo rte_kni.ko + + parm: lo_mode: KNI loopback mode (default=lo_mode_none): + lo_mode_none Kernel loopback disabled + lo_mode_fifo Enable kernel loopback with fifo + lo_mode_fifo_skb Enable kernel loopback with fifo and skb buffer + (charp) + parm: kthread_mode: Kernel thread mode (default=single): + single Single kernel thread mode enabled. + multiple Multiple kernel thread mode enabled. + (charp) + parm: carrier: Default carrier state for KNI interface (default=off): + off Interfaces will be created with carrier state set to off. + on Interfaces will be created with carrier state set to on. + (charp) - * For multiple kernel thread mode, maintains a kernel thread context for each KNI instance - (simulating the RX side of the net driver). +Loading the ``rte_kni`` kernel module without any optional parameters is +the typical way a DPDK application gets packets into and out of the kernel +network stack. Without any parameters, only one kernel thread is created +for all KNI devices for packet receiving in kernel side, loopback mode is +disabled, and the default carrier state of KNI interfaces is set to *off*. -* Net device: +.. code-block:: console - * Net functionality provided by implementing several operations such as netdev_ops, - header_ops, ethtool_ops that are defined by struct net_device, - including support for DPDK mbufs and FIFOs. + # insmod kmod/rte_kni.ko - * The interface name is provided from userspace. +.. _kni_loopback_mode: - * The MAC address can be the real NIC MAC address or random. +Loopback Mode +~~~~~~~~~~~~~ + +For testing, the ``rte_kni`` kernel module can be loaded in loopback mode +by specifying the ``lo_mode`` parameter: + +.. code-block:: console + + # insmod kmod/rte_kni.ko lo_mode=lo_mode_fifo + +The ``lo_mode_fifo`` loopback option will loop back ring enqueue/dequeue +operations in kernel space. + +.. code-block:: console + + # insmod kmod/rte_kni.ko lo_mode=lo_mode_fifo_skb + +The ``lo_mode_fifo_skb`` loopback option will loop back ring enqueue/dequeue +operations and sk buffer copies in kernel space. + +If the ``lo_mode`` parameter is not specified, loopback mode is disabled. + +.. _kni_kernel_thread_mode: + +Kernel Thread Mode +~~~~~~~~~~~~~~~~~~ + +To provide flexibility of performance, the ``rte_kni`` KNI kernel module +can be loaded with the ``kthread_mode`` parameter. The ``rte_kni`` kernel +module supports two options: "single kernel thread" mode and "multiple +kernel thread" mode. + +Single kernel thread mode is enabled as follows: + +.. code-block:: console + + # insmod kmod/rte_kni.ko kthread_mode=single + +This mode will create only one kernel thread for all KNI interfaces to +receive data on the kernel side. By default, this kernel thread is not +bound to any particular core, but the user can set the core affinity for +this kernel thread by setting the ``core_id`` and ``force_bind`` parameters +in ``struct rte_kni_conf`` when the first KNI interface is created: + +For optimum performance, the kernel thread should be bound to a core in +on the same socket as the DPDK lcores used in the application. + +The KNI kernel module can also be configured to start a separate kernel +thread for each KNI interface created by the DPDK application. Multiple +kernel thread mode is enabled as follows: + +.. code-block:: console + + # insmod kmod/rte_kni.ko kthread_mode=multiple + +This mode will create a separate kernel thread for each KNI interface to +receive data on the kernel side. The core affinity of each ``kni_thread`` +kernel thread can be specified by setting the ``core_id`` and ``force_bind`` +parameters in ``struct rte_kni_conf`` when each KNI interface is created. + +Multiple kernel thread mode can provide scalable higher performance if +sufficient unused cores are available on the host system. + +If the ``kthread_mode`` parameter is not specified, the "single kernel +thread" mode is used. + +.. _kni_default_carrier_state: + +Default Carrier State +~~~~~~~~~~~~~~~~~~~~~ + +The default carrier state of KNI interfaces created by the ``rte_kni`` +kernel module is controlled via the ``carrier`` option when the module +is loaded. + +If ``carrier=off`` is specified, the kernel module will leave the carrier +state of the interface *down* when the interface is management enabled. +The DPDK application can set the carrier state of the KNI interface using the +``rte_kni_update_link()`` function. This is useful for DPDK applications +which require that the carrier state of the KNI interface reflect the +actual link state of the corresponding physical NIC port. + +If ``carrier=on`` is specified, the kernel module will automatically set +the carrier state of the interface to *up* when the interface is management +enabled. This is useful for DPDK applications which use the KNI interface as +a purely virtual interface that does not correspond to any physical hardware +and do not wish to explicitly set the carrier state of the interface with +``rte_kni_update_link()``. It is also useful for testing in loopback mode +where the NIC port may not be physically connected to anything. + +To set the default carrier state to *on*: + +.. code-block:: console + + # insmod kmod/rte_kni.ko carrier=on + +To set the default carrier state to *off*: + +.. code-block:: console + + # insmod kmod/rte_kni.ko carrier=off + +If the ``carrier`` parameter is not specified, the default carrier state +of KNI interfaces will be set to *off*. KNI Creation and Deletion ------------------------- -The KNI interfaces are created by a DPDK application dynamically. -The interface name and FIFO details are provided by the application through an ioctl call -using the rte_kni_device_info struct which contains: +Before any KNI interfaces can be created, the ``rte_kni`` kernel module must +be loaded into the kernel and configured withe ``rte_kni_init()`` function. + +The KNI interfaces are created by a DPDK application dynamically via the +``rte_kni_alloc()`` function. + +The ``struct rte_kni_conf`` structure contains fields which allow the +user to specify the interface name, set the MTU size, set an explicit or +random MAC address and control the affinity of the kernel Rx thread(s) +(both single and multi-threaded modes). + +The ``struct rte_kni_ops`` structure contains pointers to functions to +handle requests from the ``rte_kni`` kernel module. These functions +allow DPDK applications to perform actions when the KNI interfaces are +manipulated by control commands or functions external to the application. + +For example, the DPDK application may wish to enabled/disable a physical +NIC port when a user enabled/disables a KNI interface with ``ip link set +[up|down] dev ``. The DPDK application can register a callback for +``config_network_if`` which will be called when the interface management +state changes. + +There are currently four callbacks for which the user can register +application functions: -* The interface name. +``config_network_if``: -* Physical addresses of the corresponding memzones for the relevant FIFOs. + Called when the management state of the KNI interface changes. + For example, when the user runs ``ip link set [up|down] dev ``. -* Mbuf mempool details, both physical and virtual (to calculate the offset for mbuf pointers). +``change_mtu``: -* PCI information. + Called when the user changes the MTU size of the KNI + interface. For example, when the user runs ``ip link set mtu + dev ``. -* Core affinity. +``config_mac_address``: -Refer to rte_kni_common.h in the DPDK source code for more details. + Called when the user changes the MAC address of the KNI interface. + For example, when the user runs ``ip link set address + dev ``. If the user sets this callback function to NULL, + but sets the ``port_id`` field to a value other than -1, a default + callback handler in the rte_kni library ``kni_config_mac_address()`` + will be called which calls ``rte_eth_dev_default_mac_addr_set()`` + on the specified ``port_id``. -The physical addresses will be re-mapped into the kernel address space and stored in separate KNI contexts. +``config_promiscusity``: -The affinity of kernel RX thread (both single and multi-threaded modes) is controlled by force_bind and -core_id config parameters. + Called when the user changes the promiscusity state of the KNI + interface. For example, when the user runs ``ip link set promisc + [on|off] dev ``. If the user sets this callback function to + NULL, but sets the ``port_id`` field to a value other than -1, a default + callback handler in the rte_kni library ``kni_config_promiscusity()`` + will be called which calls ``rte_eth_promiscuous_()`` + on the specified ``port_id``. -The KNI interfaces can be deleted by a DPDK application dynamically after being created. -Furthermore, all those KNI interfaces not deleted will be deleted on the release operation -of the miscellaneous device (when the DPDK application is closed). +In order to run these callbacks, the application must periodically call +the ``rte_kni_handle_request()`` function. Any user callback function +registered will be called directly from ``rte_kni_handle_request()`` so +care must be taken to prevent deadlock and to not block any DPDK fastpath +tasks. Typically DPDK applications which use these callbacks will need +to create a separate thread or secondary process to periodically call +``rte_kni_handle_request()``. + +The KNI interfaces can be deleted by a DPDK application with +``rte_kni_release()``. All KNI interfaces not explicitly deleted will be +deleted when the the ``/dev/kni`` device is closed, either explicitly with +``rte_kni_close()`` or when the DPDK application is closed. DPDK mbuf Flow -------------- @@ -118,7 +282,7 @@ The packet is received from the Linux net stack, by calling the kni_net_tx() cal The mbuf is dequeued (without waiting due the cache) and filled with data from sk_buff. The sk_buff is then freed and the mbuf sent in the tx_q FIFO. -The DPDK TX thread dequeues the mbuf and sends it to the PMD (via rte_eth_tx_burst()). +The DPDK TX thread dequeues the mbuf and sends it to the PMD via ``rte_eth_tx_burst()``. It then puts the mbuf back in the cache. Ethtool @@ -128,16 +292,3 @@ Ethtool is a Linux-specific tool with corresponding support in the kernel where each net device must register its own callbacks for the supported operations. The current implementation uses the igb/ixgbe modified Linux drivers for ethtool support. Ethtool is not supported in i40e and VMs (VF or EM devices). - -Link state and MTU change -------------------------- - -Link state and MTU change are network interface specific operations usually done via ifconfig. -The request is initiated from the kernel side (in the context of the ifconfig process) -and handled by the user space DPDK application. -The application polls the request, calls the application handler and returns the response back into the kernel space. - -The application handlers can be registered upon interface creation or explicitly registered/unregistered in runtime. -This provides flexibility in multiprocess scenarios -(where the KNI is created in the primary process but the callbacks are handled in the secondary one). -The constraint is that a single process can register and handle the requests. diff --git a/kernel/linux/kni/kni_dev.h b/kernel/linux/kni/kni_dev.h index 6275ef27f..688f574a4 100644 --- a/kernel/linux/kni/kni_dev.h +++ b/kernel/linux/kni/kni_dev.h @@ -29,6 +29,9 @@ #define MBUF_BURST_SZ 32 +/* Default carrier state for created KNI network interfaces */ +extern uint32_t dflt_carrier; + /** * A structure describing the private information for a kni device. */ diff --git a/kernel/linux/kni/kni_misc.c b/kernel/linux/kni/kni_misc.c index fa69f8e63..522ae23b9 100644 --- a/kernel/linux/kni/kni_misc.c +++ b/kernel/linux/kni/kni_misc.c @@ -39,6 +39,10 @@ static char *lo_mode; static char *kthread_mode; static uint32_t multiple_kthread_on; +/* Default carrier state for created KNI network interfaces */ +static char *carrier; +uint32_t dflt_carrier; + #define KNI_DEV_IN_USE_BIT_NUM 0 /* Bit number for device in use */ static int kni_net_id; @@ -466,6 +470,8 @@ kni_ioctl_create(struct net *net, uint32_t ioctl_num, return -ENODEV; } + netif_carrier_off(net_dev); + ret = kni_run_thread(knet, kni, dev_info.force_bind); if (ret != 0) return ret; @@ -590,6 +596,24 @@ kni_parse_kthread_mode(void) return 0; } +static int __init +kni_parse_carrier_state(void) +{ + if (!carrier) { + dflt_carrier = 0; + return 0; + } + + if (strcmp(carrier, "off") == 0) + dflt_carrier = 0; + else if (strcmp(carrier, "on") == 0) + dflt_carrier = 1; + else + return -1; + + return 0; +} + static int __init kni_init(void) { @@ -605,6 +629,16 @@ kni_init(void) else pr_debug("Multiple kernel thread mode enabled\n"); + if (kni_parse_carrier_state() < 0) { + pr_err("Invalid parameter for carrier\n"); + return -EINVAL; + } + + if (dflt_carrier == 0) + pr_debug("Default carrier state set to off.\n"); + else + pr_debug("Default carrier state set to on.\n"); + #ifdef HAVE_SIMPLIFIED_PERNET_OPERATIONS rc = register_pernet_subsys(&kni_net_ops); #else @@ -647,19 +681,27 @@ kni_exit(void) module_init(kni_init); module_exit(kni_exit); -module_param(lo_mode, charp, S_IRUGO | S_IWUSR); +module_param(lo_mode, charp, 0644); MODULE_PARM_DESC(lo_mode, "KNI loopback mode (default=lo_mode_none):\n" -" lo_mode_none Kernel loopback disabled\n" -" lo_mode_fifo Enable kernel loopback with fifo\n" -" lo_mode_fifo_skb Enable kernel loopback with fifo and skb buffer\n" -"\n" +"\t\tlo_mode_none Kernel loopback disabled\n" +"\t\tlo_mode_fifo Enable kernel loopback with fifo\n" +"\t\tlo_mode_fifo_skb Enable kernel loopback with fifo and skb buffer\n" +"\t\t" ); -module_param(kthread_mode, charp, S_IRUGO); +module_param(kthread_mode, charp, 0644); MODULE_PARM_DESC(kthread_mode, "Kernel thread mode (default=single):\n" -" single Single kernel thread mode enabled.\n" -" multiple Multiple kernel thread mode enabled.\n" -"\n" +"\t\tsingle Single kernel thread mode enabled.\n" +"\t\tmultiple Multiple kernel thread mode enabled.\n" +"\t\t" +); + +module_param(carrier, charp, 0644); +MODULE_PARM_DESC(carrier, +"Default carrier state for KNI interface (default=off):\n" +"\t\toff Interfaces will be created with carrier state set to off.\n" +"\t\ton Interfaces will be created with carrier state set to on.\n" +"\t\t" ); diff --git a/kernel/linux/kni/kni_net.c b/kernel/linux/kni/kni_net.c index 3a542d737..7371b6d58 100644 --- a/kernel/linux/kni/kni_net.c +++ b/kernel/linux/kni/kni_net.c @@ -133,6 +133,10 @@ kni_net_open(struct net_device *dev) struct kni_dev *kni = netdev_priv(dev); netif_start_queue(dev); + if (dflt_carrier == 1) + netif_carrier_on(dev); + else + netif_carrier_off(dev); memset(&req, 0, sizeof(req)); req.req_id = RTE_KNI_REQ_CFG_NETWORK_IF; @@ -152,6 +156,7 @@ kni_net_release(struct net_device *dev) struct kni_dev *kni = netdev_priv(dev); netif_stop_queue(dev); /* can't transmit any more */ + netif_carrier_off(dev); memset(&req, 0, sizeof(req)); req.req_id = RTE_KNI_REQ_CFG_NETWORK_IF; From patchwork Wed Oct 24 22:26:29 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Gora X-Patchwork-Id: 47385 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id EBF9B4CA7; Thu, 25 Oct 2018 00:26:53 +0200 (CEST) Received: from mail-qt1-f169.google.com (mail-qt1-f169.google.com [209.85.160.169]) by dpdk.org (Postfix) with ESMTP id CFCDB4CA7 for ; Thu, 25 Oct 2018 00:26:52 +0200 (CEST) Received: by mail-qt1-f169.google.com with SMTP id l9-v6so7565116qtj.12 for ; Wed, 24 Oct 2018 15:26:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Dj2S+QZeJSxbErUmmQ4dUR5UIalEAu1fCx8WmtFyras=; b=N+DrTgNwZYVrYt6nP9N3EvrPqn3D0Sh3VF+lEFZn+DbnJ1TP3uLRCDRcuXkg+bVlDE voXsZ4C7jcaBUc9zVj1BqITnd1JozHnBksAPlXCVI9pd82QAuOUxW5TPCGajWUFgGIKw VbElPLp/vkv/nbAJAQzSmQUlGO6b7N3Z2XRCWPktgazULwq0wIN14kh5TKp2Z5htSset +C2yLszsaKPvKUG6mzmZhzHpQQoaC4IaB14ZGmCEn93iCc0BiQWNdUh0CFP90rtzRnDl Wg4eOZF1LPLokzyt76kwv2Nely7i/8Mu/24y+uwEGhCVTStgwuqW9sGqPXjefMLabTyu Jg3A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references:mime-version:content-transfer-encoding; bh=Dj2S+QZeJSxbErUmmQ4dUR5UIalEAu1fCx8WmtFyras=; b=NfqELE1ttHtfsD5Hp2GuyjOVinUkQKJ8rbM26MlsFgVDWJ4pVu4IabD6l66rGN37tb 7DHhtlrH9/bFWvloLoQuS6xyLw+XqbcRp6OijA53iaN99PWXLPsmmbgcmS8WV2NHCdVY 7nUxY/CFrYlnnON5C9LCVsBvU9ekpt8UKRb326f2FKECG01V3KD86CHulCePkthce+0L WC7o7JpX17GGv6Utd5pKijGl5vnEn64jrUhPgY+KpFYMMxgFxieTeO+3c202MMY0Ry8p qnG0bfVjpq6XZdJtPLKWjSC/mUn+cb8Sw4a50t2HcRkc5TGjB2QgpvsTlXIZC5kpncsl EgvA== X-Gm-Message-State: AGRZ1gJB/c6czbfqFlZjAQR9pH9ckLRr4Yam4s1HDt5Nu4qWPZ4EBLqJ ul35cnQUON6PEToH9rGZ5Y3mFi6C X-Google-Smtp-Source: AJdET5crgNv8cVECfQUbdaZ3VuKhO+98fBoR5nui4Ub4VKs3TnnUh5B6d8aTR50ribH3t6jzEmO1hA== X-Received: by 2002:a0c:bb92:: with SMTP id i18mr4334347qvg.107.1540420011518; Wed, 24 Oct 2018 15:26:51 -0700 (PDT) Received: from snappy.local.lan ([179.228.213.114]) by smtp.gmail.com with ESMTPSA id s52-v6sm5506026qts.5.2018.10.24.15.26.47 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Wed, 24 Oct 2018 15:26:50 -0700 (PDT) From: Dan Gora To: dev@dpdk.org Cc: Igor Ryzhov , Stephen Hemminger , Ferruh Yigit , Dan Gora , John McNamara , Marko Kovacevic Date: Wed, 24 Oct 2018 19:26:29 -0300 Message-Id: <20181024222631.30461-4-dg@adax.com> X-Mailer: git-send-email 2.19.0 In-Reply-To: <20181024222631.30461-1-dg@adax.com> References: <20180911232906.18352-1-dg@adax.com> <20181024222631.30461-1-dg@adax.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v7 3/5] examples/kni: monitor and update link status continually X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Update KNI example to add the command line flag '-m' to enable a function to continuously monitor the Ethernet link status of the physical link and update the link status of the corresponding interfaces with rte_kni_update_link(). Signed-off-by: Dan Gora --- .../sample_app_ug/kernel_nic_interface.rst | 262 +++++++++++------- examples/kni/Makefile | 2 + examples/kni/main.c | 70 ++++- 3 files changed, 237 insertions(+), 97 deletions(-) diff --git a/doc/guides/sample_app_ug/kernel_nic_interface.rst b/doc/guides/sample_app_ug/kernel_nic_interface.rst index 1b3ee9a5e..6acdf0fff 100644 --- a/doc/guides/sample_app_ug/kernel_nic_interface.rst +++ b/doc/guides/sample_app_ug/kernel_nic_interface.rst @@ -31,18 +31,27 @@ This is done by creating one or more kernel net devices for each of the DPDK por The application allows the use of standard Linux tools (ethtool, ifconfig, tcpdump) with the DPDK ports and also the exchange of packets between the DPDK application and the Linux* kernel. +The Kernel NIC Interface sample application requires that the +KNI kernel module ``rte_kni`` be loaded into the kernel. See +:doc:`../prog_guide/kernel_nic_interface` for more information on loading +the ``rte_kni`` kernel module. + Overview -------- -The Kernel NIC Interface sample application uses two threads in user space for each physical NIC port being used, -and allocates one or more KNI device for each physical NIC port with kernel module's support. -For a physical NIC port, one thread reads from the port and writes to KNI devices, -and another thread reads from KNI devices and writes the data unmodified to the physical NIC port. -It is recommended to configure one KNI device for each physical NIC port. -If configured with more than one KNI devices for a physical NIC port, -it is just for performance testing, or it can work together with VMDq support in future. +The Kernel NIC Interface sample application ``kni`` allocates one or more +KNI interfaces for each physical NIC port. For each physical NIC port, +``kni`` uses two DPDK threads in user space; one thread reads from the port and +writes to the corresponding KNI interfaces and the other thread reads from +the KNI interfaces and writes the data unmodified to the physical NIC port. + +It is recommended to configure one KNI interface for each physical NIC port. +The application can be configured with more than one KNI interface for +each physical NIC port for performance testing or it can work together with +VMDq support in future. -The packet flow through the Kernel NIC Interface application is as shown in the following figure. +The packet flow through the Kernel NIC Interface application is as shown +in the following figure. .. _figure_kernel_nic: @@ -50,145 +59,221 @@ The packet flow through the Kernel NIC Interface application is as shown in the Kernel NIC Application Packet Flow +If link monitoring is enabled with the ``-m`` command line flag, one +additional pthread is launched which will check the link status of each +physical NIC port and will update the carrier status of the corresponding +KNI interface(s) to match the physical NIC port's state. This means that +the KNI interface(s) will be disabled automatically when the Ethernet link +goes down and enabled when the Ethernet link goes up. + +If link monitoring is enabled, the ``rte_kni`` kernel module should be loaded +such that the :ref:`default carrier state ` is +set to *off*. This ensures that the KNI interface is only enabled *after* +the Ethernet link of the corresponding NIC port has reached the linkup state. + +If link monitoring is not enabled, the ``rte_kni`` kernel module should be +loaded with the :ref:`default carrier state ` +set to *on*. This sets the carrier state of the KNI interfaces to *on* +when the KNI interfaces are enabled without regard to the actual link state +of the corresponding NIC port. This is useful for testing in loopback +mode where the NIC port may not be physically connected to anything. + Compiling the Application ------------------------- To compile the sample application see :doc:`compiling`. -The application is located in the ``kni`` sub-directory. +The application is located in the ``examples/kni`` sub-directory. .. note:: This application is intended as a linuxapp only. -Loading the Kernel Module -------------------------- +Running the kni Example Application +----------------------------------- -Loading the KNI kernel module without any parameter is the typical way a DPDK application -gets packets into and out of the kernel net stack. -This way, only one kernel thread is created for all KNI devices for packet receiving in kernel side: +The ``kni`` example application requires a number of command line options: .. code-block:: console - #insmod rte_kni.ko + kni [EAL options] -- -p PORTMASK --config="(port,lcore_rx,lcore_tx[,lcore_kthread,...])[,(port,lcore_rx,lcore_tx[,lcore_kthread,...])]" [-P] [-m] -Pinning the kernel thread to a specific core can be done using a taskset command such as following: +Where: -.. code-block:: console +* ``-p PORTMASK``: - #taskset -p 100000 `pgrep --fl kni_thread | awk '{print $1}'` + Hexadecimal bitmask of ports to configure. -This command line tries to pin the specific kni_thread on the 20th lcore (lcore numbering starts at 0), -which means it needs to check if that lcore is available on the board. -This command must be sent after the application has been launched, as insmod does not start the kni thread. +* ``--config="(port,lcore_rx,lcore_tx[,lcore_kthread,...])[,(port,lcore_rx,lcore_tx[,lcore_kthread,...])]"``: -For optimum performance, -the lcore in the mask must be selected to be on the same socket as the lcores used in the KNI application. + Determines which lcores the Rx and Tx DPDK tasks, and (optionally) + the KNI kernel thread(s) are bound to for each physical port. -To provide flexibility of performance, the kernel module of the KNI, -located in the kmod sub-directory of the DPDK target directory, -can be loaded with parameter of kthread_mode as follows: +* ``-P``: -* #insmod rte_kni.ko kthread_mode=single + Optional flag to set all ports to promiscuous mode so that packets are + accepted regardless of the packet's Ethernet MAC destination address. + Without this option, only packets with the Ethernet MAC destination + address set to the Ethernet address of the port are accepted. - This mode will create only one kernel thread for all KNI devices for packet receiving in kernel side. - By default, it is in this single kernel thread mode. - It can set core affinity for this kernel thread by using Linux command taskset. +* ``-m``: -* #insmod rte_kni.ko kthread_mode =multiple + Optional flag to enable monitoring and updating of the Ethernet + carrier state. With this option set, a thread will be started which + will periodically check the Ethernet link status of the physical + Ethernet ports and set the carrier state of the corresponding KNI + network interface to match it. This means that the KNI interface will + be disabled automatically when the Ethernet link goes down and enabled + when the Ethernet link goes up. - This mode will create a kernel thread for each KNI device for packet receiving in kernel side. - The core affinity of each kernel thread is set when creating the KNI device. - The lcore ID for each kernel thread is provided in the command line of launching the application. - Multiple kernel thread mode can provide scalable higher performance. +Refer to *DPDK Getting Started Guide* for general information on running +applications and the Environment Abstraction Layer (EAL) options. -To measure the throughput in a loopback mode, the kernel module of the KNI, -located in the kmod sub-directory of the DPDK target directory, -can be loaded with parameters as follows: +The ``-c coremask`` or ``-l corelist`` parameter of the EAL options must +include the lcores specified by ``lcore_rx`` and ``lcore_tx`` for each port, +but does not need to include lcores specified by ``lcore_kthread`` as those +cores are used to pin the kernel threads in the ``rte_kni`` kernel module. -* #insmod rte_kni.ko lo_mode=lo_mode_fifo +The ``--config`` parameter must include a set of +``(port,lcore_rx,lcore_tx,[lcore_kthread,...])`` values for each physical +port specified in the ``-p PORTMASK`` parameter. - This loopback mode will involve ring enqueue/dequeue operations in kernel space. +The optional ``lcore_kthread`` lcore ID parameter in ``--config`` can be +specified zero, one or more times for each physical port. -* #insmod rte_kni.ko lo_mode=lo_mode_fifo_skb +If no lcore ID is specified for ``lcore_kthread``, one KNI interface will +be created for the physical port ``port`` and the KNI kernel thread(s) +will have no specific core affinity. - This loopback mode will involve ring enqueue/dequeue operations and sk buffer copies in kernel space. +If one or more lcore IDs are specified for ``lcore_kthread``, a KNI interface +will be created for each lcore ID specified, bound to the physical port +``port``. If the ``rte_kni`` kernel module is loaded in :ref:`multiple +kernel thread ` mode, a kernel thread will be created +for each KNI interface and bound to the specified core. If the ``rte_kni`` +kernel module is loaded in :ref:`single kernel thread ` +mode, only one kernel thread is started for all KNI interfaces. The kernel +thread will be bound to the first ``lcore_kthread`` lcore ID specified. -Running the Application ------------------------ +Example Configurations +~~~~~~~~~~~~~~~~~~~~~~~ -The application requires a number of command line options: +The following commands will first load the ``rte_kni`` kernel module in +:ref:`multiple kernel thread ` mode. The ``kni`` +application is then started using two ports; Port 0 uses lcore 4 for the +Rx task, lcore 6 for the Tx task, and will create a single KNI interface +``vEth0_0`` with the kernel thread bound to lcore 8. Port 1 uses lcore +5 for the Rx task, lcore 7 for the Tx task, and will create a single KNI +interface ``vEth1_0`` with the kernel thread bound to lcore 9. .. code-block:: console - kni [EAL options] -- -P -p PORTMASK --config="(port,lcore_rx,lcore_tx[,lcore_kthread,...])[,port,lcore_rx,lcore_tx[,lcore_kthread,...]]" - -Where: + # rmmod rte_kni + # insmod kmod/rte_kni.ko kthread_mode=multiple + # ./build/kni -l 4-7 -n 4 -- -P -p 0x3 -m --config="(0,4,6,8),(1,5,7,9)" -* -P: Set all ports to promiscuous mode so that packets are accepted regardless of the packet's Ethernet MAC destination address. - Without this option, only packets with the Ethernet MAC destination address set to the Ethernet address of the port are accepted. +The following example is identical, except an additional ``lcore_kthread`` +core is specified per physical port. In this case, ``kni`` will create +four KNI interfaces: ``vEth0_0``/``vEth0_1`` bound to physical port 0 and +``vEth1_0``/``vEth1_1`` bound to physical port 1. -* -p PORTMASK: Hexadecimal bitmask of ports to configure. +The kernel thread for each interface will be bound as follows: -* --config="(port,lcore_rx, lcore_tx[,lcore_kthread, ...]) [, port,lcore_rx, lcore_tx[,lcore_kthread, ...]]": - Determines which lcores of RX, TX, kernel thread are mapped to which ports. + * ``vEth0_0`` - bound to lcore 8. + * ``vEth0_1`` - bound to lcore 10. + * ``vEth1_0`` - bound to lcore 9. + * ``vEth1_1`` - bound to lcore 11 -Refer to *DPDK Getting Started Guide* for general information on running applications and the Environment Abstraction Layer (EAL) options. +.. code-block:: console -The -c coremask or -l corelist parameter of the EAL options should include the lcores indicated by the lcore_rx and lcore_tx, -but does not need to include lcores indicated by lcore_kthread as they are used to pin the kernel thread on. -The -p PORTMASK parameter should include the ports indicated by the port in --config, neither more nor less. + # rmmod rte_kni + # insmod kmod/rte_kni.ko kthread_mode=multiple + # ./build/kni -l 4-7 -n 4 -- -P -p 0x3 -m --config="(0,4,6,8,10),(1,5,7,9,11)" -The lcore_kthread in --config can be configured none, one or more lcore IDs. -In multiple kernel thread mode, if configured none, a KNI device will be allocated for each port, -while no specific lcore affinity will be set for its kernel thread. -If configured one or more lcore IDs, one or more KNI devices will be allocated for each port, -while specific lcore affinity will be set for its kernel thread. -In single kernel thread mode, if configured none, a KNI device will be allocated for each port. -If configured one or more lcore IDs, -one or more KNI devices will be allocated for each port while -no lcore affinity will be set as there is only one kernel thread for all KNI devices. +The following example can be used to test the interface between the ``kni`` +test application and the ``rte_kni`` kernel module. In this example, +the ``rte_kni`` kernel module is loaded in :ref:`single kernel thread +mode `, :ref:`loopback mode ` +enabled, and the :ref:`default carrier state ` +is set to *on* so that the corresponding physical NIC port does not have +to be connected in order to use the KNI interface. One KNI interface +``vEth0_0`` is created for port 0 and one KNI interface ``vEth1_0`` is +created for port 1. Since ``rte_kni`` is loaded in "single kernel thread" +mode, the one kernel thread is bound to lcore 8. -For example, to run the application with two ports served by six lcores, one lcore of RX, one lcore of TX, -and one lcore of kernel thread for each port: +Since the physical NIC ports are not being used, link monitoring can be +disabled by **not** specifying the ``-m`` flag to ``kni``: .. code-block:: console - ./build/kni -l 4-7 -n 4 -- -P -p 0x3 --config="(0,4,6,8),(1,5,7,9)" + # rmmod rte_kni + # insmod kmod/rte_kni.ko lo_mode=lo_mode_fifo carrier=on + # ./build/kni -l 4-7 -n 4 -- -P -p 0x3 --config="(0,4,6,8),(1,5,7,9)" KNI Operations -------------- -Once the KNI application is started, one can use different Linux* commands to manage the net interfaces. -If more than one KNI devices configured for a physical port, -only the first KNI device will be paired to the physical device. -Operations on other KNI devices will not affect the physical port handled in user space application. +Once the ``kni`` application is started, the user can use the normal +Linux commands to manage the KNI interfaces as if they were any other +Linux network interface. -Assigning an IP address: +Enable KNI interface and assign an IP address: .. code-block:: console - #ifconfig vEth0_0 192.168.0.1 + # ifconfig vEth0_0 192.168.0.1 -Displaying the NIC registers: +Show KNI interface configuration and statistics: .. code-block:: console - #ethtool -d vEth0_0 + # ifconfig vEth0_0 + +The user can also check and reset the packet statistics inside the ``kni`` +application by sending the app the USR1 and USR2 signals: + +.. code-block:: console + + # Print statistics + # kill -SIGUSR1 `pidof kni` + + # Zero statistics + # kill -SIGUSR2 `pidof kni` -Dumping the network traffic: +Dump network traffic: .. code-block:: console - #tcpdump -i vEth0_0 + # tcpdump -i vEth0_0 + +The normal Linux commands can also be used to change the MAC address and +MTU size used by the physical NIC which corresponds to the KNI interface. +However, if more than one KNI interface is configured for a physical port, +these commands will only work on the first KNI interface for that port. Change the MAC address: .. code-block:: console - #ifconfig vEth0_0 hw ether 0C:01:02:03:04:08 + # ifconfig vEth0_0 hw ether 0C:01:02:03:04:08 + +Change the MTU size: + +.. code-block:: console + + # ifconfig vEth0_0 mtu 1450 + +If DPDK is compiled with ``CONFIG_RTE_KNI_KMOD_ETHTOOL=y`` and an Intel +NIC is used, the user can use ``ethtool`` on the KNI interface as if it +were a normal Linux kernel interface. + +Displaying the NIC registers: + +.. code-block:: console + + # ethtool -d vEth0_0 -When the DPDK userspace application is closed, all the KNI devices are deleted from Linux*. +When the ``kni`` application is closed, all the KNI interfaces are deleted +from the Linux kernel. Explanation ----------- @@ -227,7 +312,7 @@ to see if this lcore is reading from or writing to kernel NIC interfaces. For the case that reads from a NIC port and writes to the kernel NIC interfaces (``kni_ingress``), the packet reception is the same as in L2 Forwarding sample application (see :ref:`l2_fwd_app_rx_tx_packets`). -The packet transmission is done by sending mbufs into the kernel NIC interfaces by rte_kni_tx_burst(). +The packet transmission is done by sending mbufs into the kernel NIC interfaces by ``rte_kni_tx_burst()``. The KNI library automatically frees the mbufs after the kernel successfully copied the mbufs. For the other case that reads from kernel NIC interfaces @@ -235,16 +320,3 @@ and writes to a physical NIC port (``kni_egress``), packets are retrieved by reading mbufs from kernel NIC interfaces by ``rte_kni_rx_burst()``. The packet transmission is the same as in the L2 Forwarding sample application (see :ref:`l2_fwd_app_rx_tx_packets`). - -Callbacks for Kernel Requests -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -To execute specific PMD operations in user space requested by some Linux* commands, -callbacks must be implemented and filled in the struct rte_kni_ops structure. -Currently, setting a new MTU, change in MAC address, configuring promiscusous mode and -configuring the network interface(up/down) re supported. -Default implementation for following is available in rte_kni library. -Application may choose to not implement following callbacks: - -- ``config_mac_address`` -- ``config_promiscusity`` diff --git a/examples/kni/Makefile b/examples/kni/Makefile index 7e19d2e2a..dd90d7d73 100644 --- a/examples/kni/Makefile +++ b/examples/kni/Makefile @@ -20,6 +20,7 @@ static: build/$(APP)-static PC_FILE := $(shell pkg-config --path libdpdk) CFLAGS += -O3 $(shell pkg-config --cflags libdpdk) +CFLAGS += -DALLOW_EXPERIMENTAL_API LDFLAGS_SHARED = $(shell pkg-config --libs libdpdk) LDFLAGS_STATIC = -Wl,-Bstatic $(shell pkg-config --static --libs libdpdk) @@ -54,6 +55,7 @@ please change the definition of the RTE_TARGET environment variable) endif CFLAGS += -O3 +CFLAGS += -DALLOW_EXPERIMENTAL_API CFLAGS += $(WERROR_FLAGS) include $(RTE_SDK)/mk/rte.extapp.mk diff --git a/examples/kni/main.c b/examples/kni/main.c index 80c401c51..4fda51713 100644 --- a/examples/kni/main.c +++ b/examples/kni/main.c @@ -106,6 +106,8 @@ static struct rte_mempool * pktmbuf_pool = NULL; static uint32_t ports_mask = 0; /* Ports set in promiscuous mode off by default. */ static int promiscuous_on = 0; +/* Monitor link status continually. off by default. */ +static int monitor_links; /* Structure type for recording kni interface specific stats */ struct kni_interface_stats { @@ -325,11 +327,12 @@ main_loop(__rte_unused void *arg) static void print_usage(const char *prgname) { - RTE_LOG(INFO, APP, "\nUsage: %s [EAL options] -- -p PORTMASK -P " + RTE_LOG(INFO, APP, "\nUsage: %s [EAL options] -- -p PORTMASK -P -m " "[--config (port,lcore_rx,lcore_tx,lcore_kthread...)" "[,(port,lcore_rx,lcore_tx,lcore_kthread...)]]\n" " -p PORTMASK: hex bitmask of ports to use\n" " -P : enable promiscuous mode\n" + " -m : enable monitoring of port carrier state\n" " --config (port,lcore_rx,lcore_tx,lcore_kthread...): " "port and lcore configurations\n", prgname); @@ -510,7 +513,7 @@ parse_args(int argc, char **argv) opterr = 0; /* Parse command line */ - while ((opt = getopt_long(argc, argv, "p:P", longopts, + while ((opt = getopt_long(argc, argv, "p:Pm", longopts, &longindex)) != EOF) { switch (opt) { case 'p': @@ -519,6 +522,9 @@ parse_args(int argc, char **argv) case 'P': promiscuous_on = 1; break; + case 'm': + monitor_links = 1; + break; case 0: if (!strncmp(longopts[longindex].name, CMDLINE_OPT_CONFIG, @@ -674,6 +680,55 @@ check_all_ports_link_status(uint32_t port_mask) } } +static void +log_link_state(struct rte_kni *kni, int prev, struct rte_eth_link *link) +{ + if (kni == NULL || link == NULL) + return; + + if (prev == ETH_LINK_DOWN && link->link_status == ETH_LINK_UP) { + RTE_LOG(INFO, APP, "%s NIC Link is Up %d Mbps %s %s.\n", + rte_kni_get_name(kni), + link->link_speed, + link->link_autoneg ? "(AutoNeg)" : "(Fixed)", + link->link_duplex ? "Full Duplex" : "Half Duplex"); + } else if (prev == ETH_LINK_UP && link->link_status == ETH_LINK_DOWN) { + RTE_LOG(INFO, APP, "%s NIC Link is Down.\n", + rte_kni_get_name(kni)); + } +} + +/* + * Monitor the link status of all ports and update the + * corresponding KNI interface(s) + */ +static void * +monitor_all_ports_link_status(void *arg) +{ + uint16_t portid; + struct rte_eth_link link; + unsigned int i; + struct kni_port_params **p = kni_port_params_array; + int prev; + (void) arg; + + while (monitor_links) { + rte_delay_ms(500); + RTE_ETH_FOREACH_DEV(portid) { + if ((ports_mask & (1 << portid)) == 0) + continue; + memset(&link, 0, sizeof(link)); + rte_eth_link_get_nowait(portid, &link); + for (i = 0; i < p[portid]->nb_kni; i++) { + prev = rte_kni_update_link(p[portid]->kni[i], + link.link_status); + log_link_state(p[portid]->kni[i], prev, &link); + } + } + } + return NULL; +} + /* Callback for request of changing MTU */ static int kni_change_mtu(uint16_t port_id, unsigned int new_mtu) @@ -893,6 +948,8 @@ main(int argc, char** argv) int ret; uint16_t nb_sys_ports, port; unsigned i; + void *retval; + pthread_t kni_link_tid; /* Associate signal_hanlder function with USR signals */ signal(SIGUSR1, signal_handler); @@ -949,12 +1006,21 @@ main(int argc, char** argv) } check_all_ports_link_status(ports_mask); + ret = rte_ctrl_thread_create(&kni_link_tid, + "KNI link status check", NULL, + monitor_all_ports_link_status, NULL); + if (ret < 0) + rte_exit(EXIT_FAILURE, + "Could not create link status thread!\n"); + /* Launch per-lcore function on every lcore */ rte_eal_mp_remote_launch(main_loop, NULL, CALL_MASTER); RTE_LCORE_FOREACH_SLAVE(i) { if (rte_eal_wait_lcore(i) < 0) return -1; } + monitor_links = 0; + pthread_join(kni_link_tid, &retval); /* Release resources */ RTE_ETH_FOREACH_DEV(port) { From patchwork Wed Oct 24 22:26:30 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Gora X-Patchwork-Id: 47386 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 9FA1B568A; Thu, 25 Oct 2018 00:26:57 +0200 (CEST) Received: from mail-qk1-f195.google.com (mail-qk1-f195.google.com [209.85.222.195]) by dpdk.org (Postfix) with ESMTP id 53B2E5398 for ; Thu, 25 Oct 2018 00:26:55 +0200 (CEST) Received: by mail-qk1-f195.google.com with SMTP id o89so4510310qko.0 for ; Wed, 24 Oct 2018 15:26:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=1W+gCUuhLSwf9fXApqIyFdqnIzxHZWjNFsIVtIx0sc8=; b=HT/UJiPFq5gmkMXlyAfK/AOkoL0V3b55fSXpc2C4CuMgfwou7KNn+wMJ1wwAeoa7XX R6no63+qZeWSL6JbPDL5fnk9q5i3YTmoI6Hu4XDJYT8xwzBCS6mhWfR2kjamGTEA6Qmh 7ClTzPITFfqVf/UYYNtGxD3zzS3kdcn3UUTKsywD5LZ/ckXExYRRhPy38VKFC2hxqaTd msYs/R0RW4Y+K9NTrIAN7MxX/RTufbLW8KfK2XXHm3ephrMZNya8zGCmZzz/CN/QXmEW bUe1H5JhX29D8XXuqt/1ZlA5l7TLztH6OCi9p0hY8GUpESR79hXh8MHt67kZW6Zner1U eTvQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references:mime-version:content-transfer-encoding; bh=1W+gCUuhLSwf9fXApqIyFdqnIzxHZWjNFsIVtIx0sc8=; b=BYN9qMmLcIRFqHhnFbw5KzSW91zbAd8DYWRdclNXjVtXzC8Il/UIFYZzQ4HHTvGczX wUrFX4w0XfdpCxiqoF7x7IllGrlQzmnI5gP6Oh03/6VdqwDhOuN74KkuxXa/QQkimWrZ NyYqTlD1Uy1BFOwdls/mJu1lrnotxv+YNw/VixZyr+67bLj79mEZVgJjpgl1VueIAoVS HDBAnk9f4WVG7/aqGi+J4/LvsYYWYAOt6RvjicTAl7mWSGTITFGj9I0KrMydt3LmPQpp lOKde6Zf8Sa7MFHBGUfbwNs+Rx48fk2hFa3snxAuPwnTPIBZS9xtfe8hZDJFd9GaHa4T n2+g== X-Gm-Message-State: AGRZ1gKz5Q14jC0qoAhYw03ZYEejBQG5CawXmDCWaHmP8kHq3zK7hXKa cIhGHfUw2CLpy5prIEh3Bw/DHoPT X-Google-Smtp-Source: AJdET5eZsYNKRKSnSuxXce5aR9kO28UwoSmVgx6HK28YRCq1AaqPY3Zix40eShp04EOMGMR2C8d5Cw== X-Received: by 2002:a37:e50f:: with SMTP id e15-v6mr3256501qkg.84.1540420014295; Wed, 24 Oct 2018 15:26:54 -0700 (PDT) Received: from snappy.local.lan ([179.228.213.114]) by smtp.gmail.com with ESMTPSA id s52-v6sm5506026qts.5.2018.10.24.15.26.51 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Wed, 24 Oct 2018 15:26:53 -0700 (PDT) From: Dan Gora To: dev@dpdk.org Cc: Igor Ryzhov , Stephen Hemminger , Ferruh Yigit , Dan Gora Date: Wed, 24 Oct 2018 19:26:30 -0300 Message-Id: <20181024222631.30461-5-dg@adax.com> X-Mailer: git-send-email 2.19.0 In-Reply-To: <20181024222631.30461-1-dg@adax.com> References: <20180911232906.18352-1-dg@adax.com> <20181024222631.30461-1-dg@adax.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v7 4/5] examples/kni: add log msgs to show and clear stats X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add logging messages showing the commands necessary for the user to have the application display and zero the statistics. Signed-off-by: Dan Gora --- examples/kni/main.c | 16 +++++++++++++--- 1 file changed, 13 insertions(+), 3 deletions(-) diff --git a/examples/kni/main.c b/examples/kni/main.c index 4fda51713..0e3b2a2f7 100644 --- a/examples/kni/main.c +++ b/examples/kni/main.c @@ -171,14 +171,13 @@ signal_handler(int signum) /* When we receive a USR2 signal, reset stats */ if (signum == SIGUSR2) { memset(&kni_stats, 0, sizeof(kni_stats)); - printf("\n**Statistics have been reset**\n"); + printf("\n** Statistics have been reset **\n"); return; } /* When we receive a RTMIN or SIGINT signal, stop kni processing */ if (signum == SIGRTMIN || signum == SIGINT){ - printf("SIGRTMIN is received, and the KNI processing is " - "going to stop\n"); + printf("\nSIGRTMIN/SIGINT received. KNI processing stopping.\n"); rte_atomic32_inc(&kni_stop); return; } @@ -950,6 +949,7 @@ main(int argc, char** argv) unsigned i; void *retval; pthread_t kni_link_tid; + int pid; /* Associate signal_hanlder function with USR signals */ signal(SIGUSR1, signal_handler); @@ -1006,6 +1006,16 @@ main(int argc, char** argv) } check_all_ports_link_status(ports_mask); + pid = getpid(); + RTE_LOG(INFO, APP, "========================\n"); + RTE_LOG(INFO, APP, "KNI Running\n"); + RTE_LOG(INFO, APP, "kill -SIGUSR1 %d\n", pid); + RTE_LOG(INFO, APP, " Show KNI Statistics.\n"); + RTE_LOG(INFO, APP, "kill -SIGUSR2 %d\n", pid); + RTE_LOG(INFO, APP, " Zero KNI Statistics.\n"); + RTE_LOG(INFO, APP, "========================\n"); + fflush(stdout); + ret = rte_ctrl_thread_create(&kni_link_tid, "KNI link status check", NULL, monitor_all_ports_link_status, NULL); From patchwork Wed Oct 24 22:26:31 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Gora X-Patchwork-Id: 47387 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id A697C58C4; Thu, 25 Oct 2018 00:27:00 +0200 (CEST) Received: from mail-qt1-f194.google.com (mail-qt1-f194.google.com [209.85.160.194]) by dpdk.org (Postfix) with ESMTP id 126BA4C99 for ; Thu, 25 Oct 2018 00:26:58 +0200 (CEST) Received: by mail-qt1-f194.google.com with SMTP id p24-v6so1661234qtq.0 for ; Wed, 24 Oct 2018 15:26:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=+6LWhTYsYoXPIT940Q4sXLq6n9xxmxPAOZVAMSdxNcE=; b=JwmDWRL5/7RwjznFzzkNPP/qS13MQTZLJsGZzBQSHNsU7zo9FIJOiolBlyQwIxSIU0 6rrTtXSbrUcSN4aIJxtRqEbDzp0AYk97Cd21DiD7B+k4uDhjd4LNWHaXTS3FQupfMOvT XDJJkya23EcXrfZE9z1DdmQVlDzZdjGKGZycShXAu/RXJFHtXMzl1RC+dqciCDOKQes7 bUtrlCl7FszDez+GTBQRsna1fPiKD4UnYuTKXcKfO9yugCjQauQNCQsmU7M8oFxf6hWh ADcoi0morfGtDWC1mQBr5XOa5Fvbqzqy7UdvHrcInXllBGbS5uoQmqvP3DtkRqMWGPHH Qoyw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references:mime-version:content-transfer-encoding; bh=+6LWhTYsYoXPIT940Q4sXLq6n9xxmxPAOZVAMSdxNcE=; b=fVjPvQq7fTiEEjgEVMOu3k3lnCvIli5j6pbjnJO2lornHkSkM7h3aI/yJu99Mf5xAc 7zBZ0ygKG0vXxXxCKyPkM96flWC2t2LkCyokSBBEvDthDmUTEdxRa9HhD5QehevuG4/M 8Q/hdvk8dsgsAgaRmGP4y1f0JvbbadPd589WlR2tBW3Zp6S97WlrR9rJPvcESxOXehej lXg43+J0HAhx+H1mO60YcTHNzcp+wkZWRRlH4aj2W2Kz5RXc3ITPBVZMYJroJEv0fAfY Tcn/1rLGClvOrB6IOfQL/q9WGf1pN4WN34dsW2Dkbuty/S5/WxGMRW8WIA9G1QshV2/x veRQ== X-Gm-Message-State: AGRZ1gJxojSzl+RL778l1gFZJqXNBnnTedGEcJMD13aKdcKcJu+1EXQw +dpk0MFhxyWSuv6IeNex3Iy1Xpam X-Google-Smtp-Source: AJdET5eeTOocmNu0AT7ep2j4ZYbkR6CysO1OMWfb0WLOpPpgS0KiPLDTsjzBkpxViDuNxlQRQ57SVQ== X-Received: by 2002:a0c:f3c1:: with SMTP id f1mr4256887qvm.167.1540420017084; Wed, 24 Oct 2018 15:26:57 -0700 (PDT) Received: from snappy.local.lan ([179.228.213.114]) by smtp.gmail.com with ESMTPSA id s52-v6sm5506026qts.5.2018.10.24.15.26.54 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Wed, 24 Oct 2018 15:26:56 -0700 (PDT) From: Dan Gora To: dev@dpdk.org Cc: Igor Ryzhov , Stephen Hemminger , Ferruh Yigit , Dan Gora Date: Wed, 24 Oct 2018 19:26:31 -0300 Message-Id: <20181024222631.30461-6-dg@adax.com> X-Mailer: git-send-email 2.19.0 In-Reply-To: <20181024222631.30461-1-dg@adax.com> References: <20180911232906.18352-1-dg@adax.com> <20181024222631.30461-1-dg@adax.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v7 5/5] examples/kni: improve zeroing statistics X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" The worker threads incrementing the rx/tx_packets race with the signal handler from the main thread zeroing the entire statistics structure. This can cause the statistics to fail to be zeroed, even when there is no traffic on those interfaces. Improve zeroing the statistics by only incrementing rx/tx_packets in worker threads by a non-zero amount. This limits the race to the periods in which traffic is actually being received or transmitted. Signed-off-by: Dan Gora --- examples/kni/main.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/examples/kni/main.c b/examples/kni/main.c index 0e3b2a2f7..e37b1ad36 100644 --- a/examples/kni/main.c +++ b/examples/kni/main.c @@ -223,7 +223,8 @@ kni_ingress(struct kni_port_params *p) } /* Burst tx to kni */ num = rte_kni_tx_burst(p->kni[i], pkts_burst, nb_rx); - kni_stats[port_id].rx_packets += num; + if (num) + kni_stats[port_id].rx_packets += num; rte_kni_handle_request(p->kni[i]); if (unlikely(num < nb_rx)) { @@ -260,7 +261,8 @@ kni_egress(struct kni_port_params *p) } /* Burst tx to eth */ nb_tx = rte_eth_tx_burst(port_id, 0, pkts_burst, (uint16_t)num); - kni_stats[port_id].tx_packets += nb_tx; + if (nb_tx) + kni_stats[port_id].tx_packets += nb_tx; if (unlikely(nb_tx < num)) { /* Free mbufs not tx to NIC */ kni_burst_free_mbufs(&pkts_burst[nb_tx], num - nb_tx);