From patchwork Thu Jun 14 08:34:58 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adrien Mazarguil X-Patchwork-Id: 41083 Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 6F5D21E4BB; Thu, 14 Jun 2018 10:35:17 +0200 (CEST) Received: from mail-wr0-f193.google.com (mail-wr0-f193.google.com [209.85.128.193]) by dpdk.org (Postfix) with ESMTP id 898931E49A for ; Thu, 14 Jun 2018 10:35:14 +0200 (CEST) Received: by mail-wr0-f193.google.com with SMTP id l41-v6so5437638wre.7 for ; Thu, 14 Jun 2018 01:35:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=6wind-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=yNUUjGItogqUZ+HSpP8/oe62lbl8J/hjOY7zS+xO7ZE=; b=fjPfuWHkVjpLymbcQVbgnWehh3GFCjekgbibyghx/BTeaYNcDQrdFY49w9J+WThUZB c34Ahvzhz2vQbCLnUKSATmM0kiWc/jIHuyP73mtYSEsp2yotiiLzeY2Ym2+lQdfp1N9l /1rFsYljBUFuwvx2fAgaIRZCWpbU3HmduA6qNbi6vs16k908JuTEwsxbG05vVW+DvLZP wRcMt44+wJI5JIFNCJMC/sWZYPCkUYsTkVLprJMC0I7pI0OXQvtQ3i7cfnMG3y2oyOq8 xR8q0eHZmdwGD5w9CcRtNMc1LXq84c+ZWJ3HhBfJILM85QIhFLefjBf4AH3KB7pgz87x 4HXg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=yNUUjGItogqUZ+HSpP8/oe62lbl8J/hjOY7zS+xO7ZE=; b=ZIEPUEVG5Whl3zpNiNjXwtewBe4xJQ00CvWyqXCl4V1gELV2Blo2I4jBIFPOQr/VMd cZjSLD8NYj9GXcTZR86nZIB8hEuWtqj/x4hWVO/Re08O+/xz5PVhPz3IjlnmFh2XDg8r SHL/XJkmIAAkYzzJ3/itrS/zMm6pAJmF5TKwcJK/rGq9zEjKdJwN2DOhF0If7P7QKCV9 qvYDlhPJaqQAYnLPcApiJ7e8qUFpfAvGN2glnNFeXBEGpHElwFvljrTdiMvQV30Hyf89 zkBfwC99LW7a56fTkUTHhqYJNYN1MWVkqwyA8G4NP2n8uiMuMzUrpXtGZWYCp71uSuzd OfgQ== X-Gm-Message-State: APt69E2GKEkOkcODEyFN/SQb6GxTaP804Hpmlwu1FKVWIedlmXOfTrYH qJ8yTaXnweS/Ew/C7IspTmViKg== X-Google-Smtp-Source: ADUXVKIzjuXBNJRsHjBv4olGUeMxRVY6xtG3Yo2GrgMjj7gprHkLMebVK/ei6K28l1cvsKERfSNmXA== X-Received: by 2002:adf:979c:: with SMTP id s28-v6mr1410129wrb.28.1528965314245; Thu, 14 Jun 2018 01:35:14 -0700 (PDT) Received: from 6wind.com (host.78.145.23.62.rev.coltfrance.com. [62.23.145.78]) by smtp.gmail.com with ESMTPSA id a9-v6sm4338695wmh.38.2018.06.14.01.35.13 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 14 Jun 2018 01:35:13 -0700 (PDT) Date: Thu, 14 Jun 2018 10:34:58 +0200 From: Adrien Mazarguil To: Shahaf Shuler Cc: dev@dpdk.org Message-ID: <20180614083047.10812-6-adrien.mazarguil@6wind.com> References: <20180525161814.13873-1-adrien.mazarguil@6wind.com> <20180614083047.10812-1-adrien.mazarguil@6wind.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20180614083047.10812-1-adrien.mazarguil@6wind.com> X-Mailer: git-send-email 2.11.0 Subject: [dpdk-dev] [PATCH v2 5/7] net/mlx5: add port representor awareness X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" The current PCI probing method is not aware of Verbs port representors, which appear as standard Verbs devices bound to the same PCI address and cannot be distinguished. Problem is that more often than not, the wrong Verbs device is used, resulting in unexpected traffic. This patch adds necessary heuristics to bind affected driver instances to the intended (i.e. non-representor) device. (Patch based on prior work from Yuanhan Liu) Signed-off-by: Adrien Mazarguil Reviewed-by: Xueming Li --- v2 changes: - Fixed digit detection in mlx5_cmp_ibv_name() so that "foo1" and "foo10" are compared on the integer conversion of "1" against "10" instead of "" and "0". --- drivers/net/mlx5/mlx5.c | 66 ++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 61 insertions(+), 5 deletions(-) diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index c9815d721..498f80c89 100644 --- a/drivers/net/mlx5/mlx5.c +++ b/drivers/net/mlx5/mlx5.c @@ -3,6 +3,7 @@ * Copyright 2015 Mellanox Technologies, Ltd */ +#include #include #include #include @@ -1170,6 +1171,34 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev, } /** + * Comparison callback to sort Verbs device names. + * + * This is meant to be used with qsort(). + * + * @param a[in] + * Pointer to pointer to first Verbs device. + * @param b[in] + * Pointer to pointer to second Verbs device. + * + * @return + * 0 if both names are equal, less than 0 if the first argument is less + * than the second, greater than 0 otherwise. + */ +static int +mlx5_cmp_ibv_name(const void *a, const void *b) +{ + const char *name_a = (*(const struct ibv_device *const *)a)->name; + const char *name_b = (*(const struct ibv_device *const *)b)->name; + size_t i = 0; + + while (name_a[i] && name_a[i] == name_b[i]) + ++i; + while (i && isdigit(name_a[i - 1]) && isdigit(name_b[i - 1])) + --i; + return atoi(name_a + i) - atoi(name_b + i); +} + +/** * DPDK callback to register a PCI device. * * This function creates an Ethernet device for each port of a given @@ -1189,6 +1218,7 @@ mlx5_pci_probe(struct rte_pci_driver *pci_drv __rte_unused, { struct ibv_device **ibv_list; struct rte_eth_dev **eth_list = NULL; + int n = 0; int vf; int ret; @@ -1210,6 +1240,9 @@ mlx5_pci_probe(struct rte_pci_driver *pci_drv __rte_unused, DRV_LOG(ERR, "cannot list devices, is ib_uverbs loaded?"); return -rte_errno; } + + struct ibv_device *ibv_match[ret + 1]; + while (ret-- > 0) { struct rte_pci_addr pci_addr; @@ -1221,14 +1254,37 @@ mlx5_pci_probe(struct rte_pci_driver *pci_drv __rte_unused, pci_dev->addr.devid != pci_addr.devid || pci_dev->addr.function != pci_addr.function) continue; - DRV_LOG(INFO, "PCI information matches, using device \"%s\"", + DRV_LOG(INFO, "PCI information matches for device \"%s\"", ibv_list[ret]->name); - break; + ibv_match[n++] = ibv_list[ret]; + } + ibv_match[n] = NULL; + if (n > 1) { + /* + * The existence of several matching entries means port + * representors have been instantiated. No existing Verbs + * call nor /sys entries can tell them apart at this point. + * + * While definitely hackish, assume their names are numbered + * based on order of creation with master device first, + * followed by first port representor, followed by the + * second one and so on. + */ + DRV_LOG(WARNING, + "probing device with port representors involves" + " heuristics with uncertain outcome"); + qsort(ibv_match, n, sizeof(*ibv_match), mlx5_cmp_ibv_name); + DRV_LOG(WARNING, "assuming \"%s\" is the master device", + ibv_match[0]->name); + for (ret = 1; ret < n; ++ret) + DRV_LOG(WARNING, + "assuming \"%s\" is port representor #%d", + ibv_match[ret]->name, ret - 1); } - if (ret >= 0) - eth_list = mlx5_dev_spawn(&pci_dev->device, ibv_list[ret], vf); + if (n) + eth_list = mlx5_dev_spawn(&pci_dev->device, ibv_match[0], vf); mlx5_glue->free_device_list(ibv_list); - if (!ret) { + if (!n) { DRV_LOG(WARNING, "no Verbs device matches PCI device " PCI_PRI_FMT "," " are kernel drivers loaded?",