[v3] kni: rework rte_kni_update_link using ioctl
diff mbox series

Message ID 20190925093623.18419-1-iryzhov@nfware.com
State Under Review
Delegated to: Thomas Monjalon
Headers show
Series
  • [v3] kni: rework rte_kni_update_link using ioctl
Related show

Checks

Context Check Description
ci/checkpatch success coding style OK
ci/Intel-compilation success Compilation OK
ci/iol-dpdk_compile success Compile Testing PASS
ci/iol-dpdk_compile_ovs success Compile Testing PASS
ci/iol-dpdk_compile_spdk success Compile Testing PASS
ci/iol-intel-Performance success Performance Testing PASS
ci/iol-mellanox-Performance success Performance Testing PASS

Commit Message

Igor Ryzhov Sept. 25, 2019, 9:36 a.m. UTC
Current implementation doesn't allow us to update KNI carrier if the
interface is not yet UP in kernel. It means that we can't use it in the
same thread which is processing rte_kni_ops.config_network_if, which is
very convenient, because it allows us to have correct carrier status
of the interface right after we enabled it and we don't have to use any
additional thread to track link status.

Signed-off-by: Igor Ryzhov <iryzhov@nfware.com>
---
v3: remove unused variables
v2: fix checkpatch warnings

 kernel/linux/kni/compat.h                     |  4 --
 kernel/linux/kni/kni_misc.c                   | 42 +++++++++++++++++++
 kernel/linux/kni/kni_net.c                    | 15 -------
 .../linux/eal/include/rte_kni_common.h        |  6 +++
 lib/librte_kni/rte_kni.c                      | 33 +++------------
 lib/librte_kni/rte_kni.h                      |  3 +-
 6 files changed, 55 insertions(+), 48 deletions(-)

Comments

Ferruh Yigit Oct. 14, 2019, 4:10 p.m. UTC | #1
On 9/25/2019 10:36 AM, Igor Ryzhov wrote:
> Current implementation doesn't allow us to update KNI carrier if the
> interface is not yet UP in kernel. It means that we can't use it in the
> same thread which is processing rte_kni_ops.config_network_if, which is
> very convenient, because it allows us to have correct carrier status
> of the interface right after we enabled it and we don't have to use any
> additional thread to track link status.

Hi Igor,

The existing thread tracks the link status of the physical device and reflects
the changes to the kni netdev, but the "struct rte_kni_ops"
(rte_kni_ops.config_network_if) works other way around, it captures (some)
requests to kni netdev and reflects them to the underlying physical device.
Even 'rte_kni_update_link()' updated to use ioctl, the thread still looks
required and this patch doesn't really changes that part.

Also I am reluctant to extend the KNI ioctl interface when there is a generic
way to do that work.

What is the use case of updating kni netdev carrier status when the interface is
down?

> 
> Signed-off-by: Igor Ryzhov <iryzhov@nfware.com>

<...>
Ferruh Yigit Oct. 14, 2019, 4:17 p.m. UTC | #2
On 10/14/2019 5:10 PM, Ferruh Yigit wrote:
> On 9/25/2019 10:36 AM, Igor Ryzhov wrote:
>> Current implementation doesn't allow us to update KNI carrier if the
>> interface is not yet UP in kernel. It means that we can't use it in the
>> same thread which is processing rte_kni_ops.config_network_if, which is
>> very convenient, because it allows us to have correct carrier status
>> of the interface right after we enabled it and we don't have to use any
>> additional thread to track link status.
> 
> Hi Igor,
> 
> The existing thread tracks the link status of the physical device and reflects
> the changes to the kni netdev, but the "struct rte_kni_ops"
> (rte_kni_ops.config_network_if) works other way around, it captures (some)
> requests to kni netdev and reflects them to the underlying physical device.
> Even 'rte_kni_update_link()' updated to use ioctl, the thread still looks
> required and this patch doesn't really changes that part.
> 
> Also I am reluctant to extend the KNI ioctl interface when there is a generic
> way to do that work.
> 
> What is the use case of updating kni netdev carrier status when the interface is
> down?

btw, if the problem is status of the interface being 'no-carrier' by default,
this can be changed by "carrier=on" parameter of the kni kernel module:
"insmod ./build/kmod/rte_kni.ko carrier=on"

> 
>>
>> Signed-off-by: Igor Ryzhov <iryzhov@nfware.com>
> 
> <...>
>
Dan Gora Oct. 14, 2019, 7:01 p.m. UTC | #3
My original patch to add this feature was basically the same thing as
this: setting the link status via a KNI ioctl. That method was
rejected after _much_ discussion and we eventually settled on the
currently implementation.

My original patch was here: Message-Id: <20180628225548.21885-1-dg@adax.com>

If you search for KNI and dg@adax.com in the DPDK devel list you
should be able to suss out the whole discussion that lead to the
current implementation.

thanks
dan

On Mon, Oct 14, 2019 at 1:17 PM Ferruh Yigit <ferruh.yigit@intel.com> wrote:
>
> On 10/14/2019 5:10 PM, Ferruh Yigit wrote:
> > On 9/25/2019 10:36 AM, Igor Ryzhov wrote:
> >> Current implementation doesn't allow us to update KNI carrier if the
> >> interface is not yet UP in kernel. It means that we can't use it in the
> >> same thread which is processing rte_kni_ops.config_network_if, which is
> >> very convenient, because it allows us to have correct carrier status
> >> of the interface right after we enabled it and we don't have to use any
> >> additional thread to track link status.
> >
> > Hi Igor,
> >
> > The existing thread tracks the link status of the physical device and reflects
> > the changes to the kni netdev, but the "struct rte_kni_ops"
> > (rte_kni_ops.config_network_if) works other way around, it captures (some)
> > requests to kni netdev and reflects them to the underlying physical device.
> > Even 'rte_kni_update_link()' updated to use ioctl, the thread still looks
> > required and this patch doesn't really changes that part.
> >
> > Also I am reluctant to extend the KNI ioctl interface when there is a generic
> > way to do that work.
> >
> > What is the use case of updating kni netdev carrier status when the interface is
> > down?
>
> btw, if the problem is status of the interface being 'no-carrier' by default,
> this can be changed by "carrier=on" parameter of the kni kernel module:
> "insmod ./build/kmod/rte_kni.ko carrier=on"
>
> >
> >>
> >> Signed-off-by: Igor Ryzhov <iryzhov@nfware.com>
> >
> > <...>
> >
>
Dan Gora Oct. 14, 2019, 8:55 p.m. UTC | #4
Here's another link to the thread where this was discussed last year..
Igor was actually on this thread as well...

https://mails.dpdk.org/archives/dev/2018-August/110383.html

On Mon, Oct 14, 2019 at 4:01 PM Dan Gora <dg@adax.com> wrote:
>
> My original patch to add this feature was basically the same thing as
> this: setting the link status via a KNI ioctl. That method was
> rejected after _much_ discussion and we eventually settled on the
> currently implementation.
>
> My original patch was here: Message-Id: <20180628225548.21885-1-dg@adax.com>
>
> If you search for KNI and dg@adax.com in the DPDK devel list you
> should be able to suss out the whole discussion that lead to the
> current implementation.
>
> thanks
> dan
>
> On Mon, Oct 14, 2019 at 1:17 PM Ferruh Yigit <ferruh.yigit@intel.com> wrote:
> >
> > On 10/14/2019 5:10 PM, Ferruh Yigit wrote:
> > > On 9/25/2019 10:36 AM, Igor Ryzhov wrote:
> > >> Current implementation doesn't allow us to update KNI carrier if the
> > >> interface is not yet UP in kernel. It means that we can't use it in the
> > >> same thread which is processing rte_kni_ops.config_network_if, which is
> > >> very convenient, because it allows us to have correct carrier status
> > >> of the interface right after we enabled it and we don't have to use any
> > >> additional thread to track link status.
> > >
> > > Hi Igor,
> > >
> > > The existing thread tracks the link status of the physical device and reflects
> > > the changes to the kni netdev, but the "struct rte_kni_ops"
> > > (rte_kni_ops.config_network_if) works other way around, it captures (some)
> > > requests to kni netdev and reflects them to the underlying physical device.
> > > Even 'rte_kni_update_link()' updated to use ioctl, the thread still looks
> > > required and this patch doesn't really changes that part.
> > >
> > > Also I am reluctant to extend the KNI ioctl interface when there is a generic
> > > way to do that work.
> > >
> > > What is the use case of updating kni netdev carrier status when the interface is
> > > down?
> >
> > btw, if the problem is status of the interface being 'no-carrier' by default,
> > this can be changed by "carrier=on" parameter of the kni kernel module:
> > "insmod ./build/kmod/rte_kni.ko carrier=on"
Igor Ryzhov Oct. 27, 2019, 8:16 p.m. UTC | #5
Hi Ferruh, Dan,

Sure, I remember last year discussion but now I see the problem in current
implementation.

Ferruh, here is an example:

We have a thread in the application that processes KNI commands from the
kernel.
It receives config_network_if command to set interface up, calls
rte_eth_dev_start, and here is the problem.
We cannot call current rte_kni_update_link from here as the interface is
not yet up in the kernel,
as we didn't send a response for config_network_if yet. So we need to send
a response first and only
after that, we can use rte_kni_update_link. Actually, we don't even know
the exact time between we
send a response and the moment when the kernel receives it and the
interface becomes up.
We always have a dependency on the interface state in the kernel. With
ioctl approach, we don't
have such dependency - we can call rte_kni_update_link whenever we want,
even when the interface is
down in the kernel. As I explained, it's common when processing
config_network_if to set interface up.

Igor

On Mon, Oct 14, 2019 at 11:56 PM Dan Gora <dg@adax.com> wrote:

> Here's another link to the thread where this was discussed last year..
> Igor was actually on this thread as well...
>
> https://mails.dpdk.org/archives/dev/2018-August/110383.html
>
> On Mon, Oct 14, 2019 at 4:01 PM Dan Gora <dg@adax.com> wrote:
> >
> > My original patch to add this feature was basically the same thing as
> > this: setting the link status via a KNI ioctl. That method was
> > rejected after _much_ discussion and we eventually settled on the
> > currently implementation.
> >
> > My original patch was here: Message-Id: <
> 20180628225548.21885-1-dg@adax.com>
> >
> > If you search for KNI and dg@adax.com in the DPDK devel list you
> > should be able to suss out the whole discussion that lead to the
> > current implementation.
> >
> > thanks
> > dan
> >
> > On Mon, Oct 14, 2019 at 1:17 PM Ferruh Yigit <ferruh.yigit@intel.com>
> wrote:
> > >
> > > On 10/14/2019 5:10 PM, Ferruh Yigit wrote:
> > > > On 9/25/2019 10:36 AM, Igor Ryzhov wrote:
> > > >> Current implementation doesn't allow us to update KNI carrier if the
> > > >> interface is not yet UP in kernel. It means that we can't use it in
> the
> > > >> same thread which is processing rte_kni_ops.config_network_if,
> which is
> > > >> very convenient, because it allows us to have correct carrier status
> > > >> of the interface right after we enabled it and we don't have to use
> any
> > > >> additional thread to track link status.
> > > >
> > > > Hi Igor,
> > > >
> > > > The existing thread tracks the link status of the physical device
> and reflects
> > > > the changes to the kni netdev, but the "struct rte_kni_ops"
> > > > (rte_kni_ops.config_network_if) works other way around, it captures
> (some)
> > > > requests to kni netdev and reflects them to the underlying physical
> device.
> > > > Even 'rte_kni_update_link()' updated to use ioctl, the thread still
> looks
> > > > required and this patch doesn't really changes that part.
> > > >
> > > > Also I am reluctant to extend the KNI ioctl interface when there is
> a generic
> > > > way to do that work.
> > > >
> > > > What is the use case of updating kni netdev carrier status when the
> interface is
> > > > down?
> > >
> > > btw, if the problem is status of the interface being 'no-carrier' by
> default,
> > > this can be changed by "carrier=on" parameter of the kni kernel module:
> > > "insmod ./build/kmod/rte_kni.ko carrier=on"
>

Patch
diff mbox series

diff --git a/kernel/linux/kni/compat.h b/kernel/linux/kni/compat.h
index fe0ee55e7..e0a491bcd 100644
--- a/kernel/linux/kni/compat.h
+++ b/kernel/linux/kni/compat.h
@@ -61,10 +61,6 @@ 
 #define kni_sock_map_fd(s) sock_map_fd(s, 0)
 #endif
 
-#if LINUX_VERSION_CODE >= KERNEL_VERSION(3, 9, 0)
-#define HAVE_CHANGE_CARRIER_CB
-#endif
-
 #if LINUX_VERSION_CODE < KERNEL_VERSION(3, 14, 0)
 #define ether_addr_copy(dst, src) memcpy(dst, src, ETH_ALEN)
 #endif
diff --git a/kernel/linux/kni/kni_misc.c b/kernel/linux/kni/kni_misc.c
index aeb275329..9bcba68c8 100644
--- a/kernel/linux/kni/kni_misc.c
+++ b/kernel/linux/kni/kni_misc.c
@@ -462,6 +462,45 @@  kni_ioctl_release(struct net *net, uint32_t ioctl_num,
 	return ret;
 }
 
+static int
+kni_ioctl_link(struct net *net, uint32_t ioctl_num,
+		unsigned long ioctl_param)
+{
+	struct kni_net *knet = net_generic(net, kni_net_id);
+	int ret = -EINVAL;
+	struct kni_dev *dev, *n;
+	struct rte_kni_link_info link_info;
+	struct net_device *netdev;
+
+	if (_IOC_SIZE(ioctl_num) > sizeof(link_info))
+		return -EINVAL;
+
+	if (copy_from_user(&link_info, (void *)ioctl_param, sizeof(link_info)))
+		return -EFAULT;
+
+	if (strlen(link_info.name) == 0)
+		return -EINVAL;
+
+	down_read(&knet->kni_list_lock);
+	list_for_each_entry_safe(dev, n, &knet->kni_list_head, list) {
+		if (strncmp(dev->name, link_info.name, RTE_KNI_NAMESIZE) != 0)
+			continue;
+
+		netdev = dev->net_dev;
+
+		if (link_info.linkup)
+			netif_carrier_on(netdev);
+		else
+			netif_carrier_off(netdev);
+
+		ret = 0;
+		break;
+	}
+	up_read(&knet->kni_list_lock);
+
+	return ret;
+}
+
 static int
 kni_ioctl(struct inode *inode, uint32_t ioctl_num, unsigned long ioctl_param)
 {
@@ -483,6 +522,9 @@  kni_ioctl(struct inode *inode, uint32_t ioctl_num, unsigned long ioctl_param)
 	case _IOC_NR(RTE_KNI_IOCTL_RELEASE):
 		ret = kni_ioctl_release(net, ioctl_num, ioctl_param);
 		break;
+	case _IOC_NR(RTE_KNI_IOCTL_LINK):
+		ret = kni_ioctl_link(net, ioctl_num, ioctl_param);
+		break;
 	default:
 		pr_debug("IOCTL default\n");
 		break;
diff --git a/kernel/linux/kni/kni_net.c b/kernel/linux/kni/kni_net.c
index 7bd3a9f1e..cd852eea3 100644
--- a/kernel/linux/kni/kni_net.c
+++ b/kernel/linux/kni/kni_net.c
@@ -706,18 +706,6 @@  kni_net_set_mac(struct net_device *netdev, void *p)
 	return (ret == 0 ? req.result : ret);
 }
 
-#ifdef HAVE_CHANGE_CARRIER_CB
-static int
-kni_net_change_carrier(struct net_device *dev, bool new_carrier)
-{
-	if (new_carrier)
-		netif_carrier_on(dev);
-	else
-		netif_carrier_off(dev);
-	return 0;
-}
-#endif
-
 static const struct header_ops kni_net_header_ops = {
 	.create  = kni_net_header,
 	.parse   = eth_header_parse,
@@ -736,9 +724,6 @@  static const struct net_device_ops kni_net_netdev_ops = {
 	.ndo_change_mtu = kni_net_change_mtu,
 	.ndo_tx_timeout = kni_net_tx_timeout,
 	.ndo_set_mac_address = kni_net_set_mac,
-#ifdef HAVE_CHANGE_CARRIER_CB
-	.ndo_change_carrier = kni_net_change_carrier,
-#endif
 };
 
 static void kni_get_drvinfo(struct net_device *dev,
diff --git a/lib/librte_eal/linux/eal/include/rte_kni_common.h b/lib/librte_eal/linux/eal/include/rte_kni_common.h
index 70992d835..07a10dd93 100644
--- a/lib/librte_eal/linux/eal/include/rte_kni_common.h
+++ b/lib/librte_eal/linux/eal/include/rte_kni_common.h
@@ -125,10 +125,16 @@  struct rte_kni_device_info {
 	uint8_t mac_addr[6];
 };
 
+struct rte_kni_link_info {
+	char name[RTE_KNI_NAMESIZE];
+	unsigned int linkup;
+};
+
 #define KNI_DEVICE "kni"
 
 #define RTE_KNI_IOCTL_TEST    _IOWR(0, 1, int)
 #define RTE_KNI_IOCTL_CREATE  _IOWR(0, 2, struct rte_kni_device_info)
 #define RTE_KNI_IOCTL_RELEASE _IOWR(0, 3, struct rte_kni_device_info)
+#define RTE_KNI_IOCTL_LINK    _IOWR(0, 4, struct rte_kni_link_info)
 
 #endif /* _RTE_KNI_COMMON_H_ */
diff --git a/lib/librte_kni/rte_kni.c b/lib/librte_kni/rte_kni.c
index 521db27c4..f80b97e2f 100644
--- a/lib/librte_kni/rte_kni.c
+++ b/lib/librte_kni/rte_kni.c
@@ -733,41 +733,20 @@  rte_kni_unregister_handlers(struct rte_kni *kni)
 int
 rte_kni_update_link(struct rte_kni *kni, unsigned int linkup)
 {
-	char path[64];
-	char old_carrier[2];
-	const char *new_carrier;
-	int old_linkup;
-	int fd, ret;
+	struct rte_kni_link_info link_info;
 
 	if (kni == NULL)
 		return -1;
 
-	snprintf(path, sizeof(path), "/sys/devices/virtual/net/%s/carrier",
-		kni->name);
+	snprintf(link_info.name, RTE_KNI_NAMESIZE, "%s", kni->name);
+	link_info.linkup = linkup;
 
-	fd = open(path, O_RDWR);
-	if (fd == -1) {
-		RTE_LOG(ERR, KNI, "Failed to open file: %s.\n", path);
+	if (ioctl(kni_fd, RTE_KNI_IOCTL_LINK, &link_info) < 0) {
+		RTE_LOG(ERR, KNI, "Fail to update KNI link\n");
 		return -1;
 	}
 
-	ret = read(fd, old_carrier, 2);
-	if (ret < 1) {
-		close(fd);
-		return -1;
-	}
-	old_linkup = (old_carrier[0] == '1');
-
-	new_carrier = linkup ? "1" : "0";
-	ret = write(fd, new_carrier, 1);
-	if (ret < 1) {
-		RTE_LOG(ERR, KNI, "Failed to write file: %s.\n", path);
-		close(fd);
-		return -1;
-	}
-
-	close(fd);
-	return old_linkup;
+	return 0;
 }
 
 void
diff --git a/lib/librte_kni/rte_kni.h b/lib/librte_kni/rte_kni.h
index b22446fa7..7e1cce1e0 100644
--- a/lib/librte_kni/rte_kni.h
+++ b/lib/librte_kni/rte_kni.h
@@ -247,9 +247,8 @@  int rte_kni_unregister_handlers(struct rte_kni *kni);
  *  > 0 for linkup.
  *
  * @return
+ *  On success: 0
  *  On failure: -1
- *  Previous link state == linkdown: 0
- *  Previous link state == linkup: 1
  */
 __rte_experimental
 int