From patchwork Fri Jan 8 15:14:59 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xueming Li X-Patchwork-Id: 86219 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (xvm-189-124.dc0.ghst.net [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id E3C50A0524; Fri, 8 Jan 2021 16:15:10 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 5F7AF140E7A; Fri, 8 Jan 2021 16:15:10 +0100 (CET) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by mails.dpdk.org (Postfix) with ESMTP id 085DD140E76 for ; Fri, 8 Jan 2021 16:15:08 +0100 (CET) Received: from Internal Mail-Server by MTLPINE1 (envelope-from xuemingl@nvidia.com) with SMTP; 8 Jan 2021 17:15:07 +0200 Received: from nvidia.com ([172.27.8.145]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 108FF3ZI000760; Fri, 8 Jan 2021 17:15:05 +0200 From: Xueming Li To: Viacheslav Ovsiienko Cc: dev@dpdk.org, Matan Azrad , Shahaf Shuler , Thomas Monjalon , xuemingl@nvidia.com, Asaf Penso Date: Fri, 8 Jan 2021 23:14:59 +0800 Message-Id: <20210108151500.30308-2-xuemingl@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <1608304614-13908-2-git-send-email-xuemingl@nvidia.com> References: <1608304614-13908-2-git-send-email-xuemingl@nvidia.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v1 1/2] common/mlx5: support device global syntax X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This patch support new device global syntax: bus=,k=v,,,/class=,k=v,,,/driver=,k=v,,,, To reuse class name of global syntax, this patch also changes internal class name introduced by commit [1] to algin with RTE class name. [1] 8a41f4deccc3: common/mlx5: introduce layer for multiple class drivers Signed-off-by: Xueming Li --- drivers/common/mlx5/mlx5_common_pci.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/drivers/common/mlx5/mlx5_common_pci.c b/drivers/common/mlx5/mlx5_common_pci.c index 5208972bb6..b1eda7b3c8 100644 --- a/drivers/common/mlx5/mlx5_common_pci.c +++ b/drivers/common/mlx5/mlx5_common_pci.c @@ -4,6 +4,7 @@ #include #include +#include #include "mlx5_common_utils.h" #include "mlx5_common_pci.h" @@ -26,7 +27,7 @@ static const struct { unsigned int driver_class; } mlx5_classes[] = { { .name = "vdpa", .driver_class = MLX5_CLASS_VDPA }, - { .name = "net", .driver_class = MLX5_CLASS_NET }, + { .name = "eth", .driver_class = MLX5_CLASS_NET }, { .name = "regex", .driver_class = MLX5_CLASS_REGEX }, }; @@ -115,6 +116,9 @@ parse_class_options(const struct rte_devargs *devargs) if (devargs == NULL) return 0; + if (devargs->cls) + /* support new global syntax */ + return class_name_to_value(devargs->cls->name); kvlist = rte_kvargs_parse(devargs->args, NULL); if (kvlist == NULL) return 0; From patchwork Fri Jan 8 15:15:00 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xueming Li X-Patchwork-Id: 86221 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (xvm-189-124.dc0.ghst.net [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id EA900A0524; Fri, 8 Jan 2021 16:15:24 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id AA12A141020; Fri, 8 Jan 2021 16:15:14 +0100 (CET) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by mails.dpdk.org (Postfix) with ESMTP id CA0E3141015 for ; Fri, 8 Jan 2021 16:15:13 +0100 (CET) Received: from Internal Mail-Server by MTLPINE1 (envelope-from xuemingl@nvidia.com) with SMTP; 8 Jan 2021 17:15:09 +0200 Received: from nvidia.com ([172.27.8.145]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 108FF3ZJ000760; Fri, 8 Jan 2021 17:15:08 +0200 From: Xueming Li To: Viacheslav Ovsiienko Cc: dev@dpdk.org, Matan Azrad , Shahaf Shuler , Thomas Monjalon , xuemingl@nvidia.com, Asaf Penso Date: Fri, 8 Jan 2021 23:15:00 +0800 Message-Id: <20210108151500.30308-3-xuemingl@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <1608304614-13908-2-git-send-email-xuemingl@nvidia.com> References: <1608304614-13908-2-git-send-email-xuemingl@nvidia.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v1 2/2] net/mlx5: support new device global syntax X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This patch support new device global syntax like: bus=pci,addr=BB:DD.F/class=eth/driver=mlx5,devargs,.. Ignore "driver" key as part of new global device syntax in devargs. The representor devarg is supposed to come from either class section or driver section. Signed-off-by: Xueming Li --- drivers/net/mlx5/linux/mlx5_os.c | 18 ++++++++++++++++-- drivers/net/mlx5/mlx5.c | 6 +++++- 2 files changed, 21 insertions(+), 3 deletions(-) diff --git a/drivers/net/mlx5/linux/mlx5_os.c b/drivers/net/mlx5/linux/mlx5_os.c index 6812a1f215..f1ed3505b1 100644 --- a/drivers/net/mlx5/linux/mlx5_os.c +++ b/drivers/net/mlx5/linux/mlx5_os.c @@ -699,13 +699,27 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev, if (switch_info->representor && dpdk_dev->devargs) { struct rte_eth_devargs eth_da; - err = rte_eth_devargs_parse(dpdk_dev->devargs->args, ð_da); + /* Representer should come from class argument or driver */ + if (dpdk_dev->devargs->cls_str) + err = rte_eth_devargs_parse(dpdk_dev->devargs->cls_str, + ð_da); if (err) { rte_errno = -err; DRV_LOG(ERR, "failed to process device arguments: %s", - strerror(rte_errno)); + dpdk_dev->devargs->cls_str); return NULL; } + if (eth_da.type == RTE_ETH_REPRESENTOR_NONE) { + /* Support legacy device argument */ + err = rte_eth_devargs_parse(dpdk_dev->devargs->args, + ð_da); + if (err) { + rte_errno = -err; + DRV_LOG(ERR, "failed to process device arguments: %s", + dpdk_dev->devargs->args); + return NULL; + } + } for (i = 0; i < eth_da.nb_representor_ports; ++i) if (eth_da.representor_ports[i] == (uint16_t)switch_info->port_name) diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index 023ef50a77..f2b6cf9fd6 100644 --- a/drivers/net/mlx5/mlx5.c +++ b/drivers/net/mlx5/mlx5.c @@ -41,6 +41,9 @@ #include "mlx5_flow_os.h" #include "rte_pmd_mlx5.h" +/* Driver type key for new device global syntax. */ +#define MLX5_DRIVER_KEY "driver" + /* Device parameter to enable RX completion queue compression. */ #define MLX5_RXQ_CQE_COMP_EN "rxq_cqe_comp_en" @@ -1600,7 +1603,7 @@ mlx5_args_check(const char *key, const char *val, void *opaque) signed long tmp; /* No-op, port representors are processed in mlx5_dev_spawn(). */ - if (!strcmp(MLX5_REPRESENTOR, key)) + if (!strcmp(MLX5_DRIVER_KEY, key) || !strcmp(MLX5_REPRESENTOR, key)) return 0; errno = 0; tmp = strtol(val, NULL, 0); @@ -1754,6 +1757,7 @@ int mlx5_args(struct mlx5_dev_config *config, struct rte_devargs *devargs) { const char **params = (const char *[]){ + MLX5_DRIVER_KEY, MLX5_RXQ_CQE_COMP_EN, MLX5_RXQ_CQE_PAD_EN, MLX5_RXQ_PKT_PAD_EN, From patchwork Tue Jan 19 07:28:09 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xueming Li X-Patchwork-Id: 86865 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 2CC45A0A03; Tue, 19 Jan 2021 08:29:56 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id DAA56140CF2; Tue, 19 Jan 2021 08:29:55 +0100 (CET) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by mails.dpdk.org (Postfix) with ESMTP id D9A50140CCF for ; Tue, 19 Jan 2021 08:29:53 +0100 (CET) Received: from Internal Mail-Server by MTLPINE1 (envelope-from xuemingl@nvidia.com) with SMTP; 19 Jan 2021 09:29:52 +0200 Received: from nvidia.com (pegasus05.mtr.labs.mlnx [10.210.16.100]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 10J7Tpqm014531; Tue, 19 Jan 2021 09:29:51 +0200 From: Xueming Li To: Viacheslav Ovsiienko Cc: dev@dpdk.org, xuemingl@nvidia.com, Asaf Penso Date: Tue, 19 Jan 2021 07:28:09 +0000 Message-Id: <1611041295-12797-4-git-send-email-xuemingl@nvidia.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1611041295-12797-1-git-send-email-xuemingl@nvidia.com> References: <1611041295-12797-1-git-send-email-xuemingl@nvidia.com> In-Reply-To: <1608304614-13908-2-git-send-email-xuemingl@nvidia.com> References: <1608304614-13908-2-git-send-email-xuemingl@nvidia.com> Subject: [dpdk-dev] [PATCH v4 3/8] net/mlx5: revert setting bonding representor to first PF X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" With kernel bonding, representors on second PF are being probed by devargs: ,representor=pf1vf No need to save primary PF port ID and lookup when probing sibling ports, revert patch [1] [1]: commit e6818853c022 ("net/mlx5: set representor to first PF in bonding mode") Signed-off-by: Xueming Li Acked-by: Viacheslav Ovsiienko --- drivers/net/mlx5/linux/mlx5_os.c | 20 ++------------------ drivers/net/mlx5/mlx5.c | 1 - drivers/net/mlx5/mlx5.h | 1 - 3 files changed, 2 insertions(+), 20 deletions(-) diff --git a/drivers/net/mlx5/linux/mlx5_os.c b/drivers/net/mlx5/linux/mlx5_os.c index b2776c080a..7b320e8b72 100644 --- a/drivers/net/mlx5/linux/mlx5_os.c +++ b/drivers/net/mlx5/linux/mlx5_os.c @@ -816,13 +816,7 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev, rte_errno = ENOMEM; return NULL; } - priv = eth_dev->data->dev_private; - if (priv->sh->bond_dev != UINT16_MAX) - /* For bonding port, use primary PCI device. */ - eth_dev->device = - rte_eth_devices[priv->sh->bond_dev].device; - else - eth_dev->device = dpdk_dev; + eth_dev->device = dpdk_dev; eth_dev->dev_ops = &mlx5_dev_sec_ops; eth_dev->rx_descriptor_status = mlx5_rx_descriptor_status; eth_dev->tx_descriptor_status = mlx5_tx_descriptor_status; @@ -1439,17 +1433,7 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev, eth_dev->data->dev_private = priv; priv->dev_data = eth_dev->data; eth_dev->data->mac_addrs = priv->mac; - if (spawn->pf_bond < 0) { - eth_dev->device = dpdk_dev; - } else { - /* Use primary bond PCI as device. */ - if (sh->bond_dev == UINT16_MAX) { - sh->bond_dev = eth_dev->data->port_id; - eth_dev->device = dpdk_dev; - } else { - eth_dev->device = rte_eth_devices[sh->bond_dev].device; - } - } + eth_dev->device = dpdk_dev; eth_dev->data->dev_flags |= RTE_ETH_DEV_AUTOFILL_QUEUE_XSTATS; /* Configure the first MAC address by default. */ if (mlx5_get_mac(eth_dev, &mac.addr_bytes)) { diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index e245276fce..5e8cd6a3df 100644 --- a/drivers/net/mlx5/mlx5.c +++ b/drivers/net/mlx5/mlx5.c @@ -914,7 +914,6 @@ mlx5_alloc_shared_dev_ctx(const struct mlx5_dev_spawn_data *spawn, goto error; } sh->refcnt = 1; - sh->bond_dev = UINT16_MAX; sh->max_port = spawn->max_port; strncpy(sh->ibdev_name, mlx5_os_get_ctx_device_name(sh->ctx), sizeof(sh->ibdev_name) - 1); diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index 3836a9696c..e06e0ff3bb 100644 --- a/drivers/net/mlx5/mlx5.h +++ b/drivers/net/mlx5/mlx5.h @@ -668,7 +668,6 @@ struct mlx5_flex_parser_profiles { struct mlx5_dev_ctx_shared { LIST_ENTRY(mlx5_dev_ctx_shared) next; uint32_t refcnt; - uint16_t bond_dev; /* Bond primary device id. */ uint32_t devx:1; /* Opened with DV. */ uint32_t flow_hit_aso_en:1; /* Flow Hit ASO is supported. */ uint32_t max_port; /* Maximal IB device port index. */ From patchwork Tue Jan 19 07:28:12 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xueming Li X-Patchwork-Id: 86867 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 1260AA0A03; Tue, 19 Jan 2021 08:30:09 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 5224B140D52; Tue, 19 Jan 2021 08:29:58 +0100 (CET) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by mails.dpdk.org (Postfix) with ESMTP id E6DA0140CF4 for ; Tue, 19 Jan 2021 08:29:53 +0100 (CET) Received: from Internal Mail-Server by MTLPINE1 (envelope-from xuemingl@nvidia.com) with SMTP; 19 Jan 2021 09:29:52 +0200 Received: from nvidia.com (pegasus05.mtr.labs.mlnx [10.210.16.100]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 10J7Tpqp014531; Tue, 19 Jan 2021 09:29:52 +0200 From: Xueming Li To: Viacheslav Ovsiienko Cc: dev@dpdk.org, xuemingl@nvidia.com, Asaf Penso , Anatoly Burakov Date: Tue, 19 Jan 2021 07:28:12 +0000 Message-Id: <1611041295-12797-7-git-send-email-xuemingl@nvidia.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1611041295-12797-1-git-send-email-xuemingl@nvidia.com> References: <1611041295-12797-1-git-send-email-xuemingl@nvidia.com> In-Reply-To: <1608304614-13908-2-git-send-email-xuemingl@nvidia.com> References: <1608304614-13908-2-git-send-email-xuemingl@nvidia.com> Subject: [dpdk-dev] [PATCH v4 6/8] net/mlx5: save bonding member ports information X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Since kernel bonding netdev doesn't provide statistics counter that reflects all member ports, PMD has to manually summarize counters from each member ports. As a preparation, this patch collects bonding member port information and saves to shared context data. Signed-off-by: Xueming Li Acked-by: Viacheslav Ovsiienko --- drivers/net/mlx5/linux/mlx5_ethdev_os.c | 4 +- drivers/net/mlx5/linux/mlx5_os.c | 91 ++++++++++++++++--------- drivers/net/mlx5/mlx5.c | 2 + drivers/net/mlx5/mlx5.h | 21 +++++- drivers/net/mlx5/mlx5_ethdev.c | 5 +- 5 files changed, 86 insertions(+), 37 deletions(-) diff --git a/drivers/net/mlx5/linux/mlx5_ethdev_os.c b/drivers/net/mlx5/linux/mlx5_ethdev_os.c index ac311de46d..84610a7bc0 100644 --- a/drivers/net/mlx5/linux/mlx5_ethdev_os.c +++ b/drivers/net/mlx5/linux/mlx5_ethdev_os.c @@ -150,8 +150,8 @@ mlx5_get_ifname(const struct rte_eth_dev *dev, char (*ifname)[MLX5_NAMESIZE]) MLX5_ASSERT(priv); MLX5_ASSERT(priv->sh); - if (priv->bond_ifindex > 0) { - memcpy(ifname, priv->bond_name, MLX5_NAMESIZE); + if (priv->master && priv->sh->bond.ifindex > 0) { + memcpy(ifname, priv->sh->bond.ifname, MLX5_NAMESIZE); return 0; } ifindex = mlx5_ifindex(dev); diff --git a/drivers/net/mlx5/linux/mlx5_os.c b/drivers/net/mlx5/linux/mlx5_os.c index 521a0a5789..47a7c3dff0 100644 --- a/drivers/net/mlx5/linux/mlx5_os.c +++ b/drivers/net/mlx5/linux/mlx5_os.c @@ -1417,19 +1417,6 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev, */ MLX5_ASSERT(spawn->ifindex); priv->if_index = spawn->ifindex; - if (priv->pf_bond >= 0 && priv->master) { - /* Get bond interface info */ - err = mlx5_sysfs_bond_info(priv->if_index, - &priv->bond_ifindex, - priv->bond_name); - if (err) - DRV_LOG(ERR, "unable to get bond info: %s", - strerror(rte_errno)); - else - DRV_LOG(INFO, "PF device %u, bond device %u(%s)", - priv->if_index, priv->bond_ifindex, - priv->bond_name); - } eth_dev->data->dev_private = priv; priv->dev_data = eth_dev->data; eth_dev->data->mac_addrs = priv->mac; @@ -1697,6 +1684,8 @@ mlx5_dev_spawn_data_cmp(const void *a, const void *b) * Netlink RDMA group socket handle. * @param[in] owner * Rerepsentor owner PF index. + * @param[out] bond_info + * Pointer to bonding information. * * @return * negative value if no bonding device found, otherwise @@ -1705,19 +1694,22 @@ mlx5_dev_spawn_data_cmp(const void *a, const void *b) static int mlx5_device_bond_pci_match(const struct ibv_device *ibv_dev, const struct rte_pci_addr *pci_dev, - int nl_rdma, uint16_t owner) + int nl_rdma, uint16_t owner, + struct mlx5_bond_info *bond_info) { char ifname[IF_NAMESIZE + 1]; unsigned int ifindex; unsigned int np, i; - FILE *file = NULL; + FILE *bond_file = NULL, *file; int pf = -1; + int ret; /* * Try to get master device name. If something goes * wrong suppose the lack of kernel support and no * bonding devices. */ + memset(bond_info, 0, sizeof(*bond_info)); if (nl_rdma < 0) return -1; if (!strstr(ibv_dev->name, "bond")) @@ -1741,15 +1733,15 @@ mlx5_device_bond_pci_match(const struct ibv_device *ibv_dev, /* Try to read bonding slave names from sysfs. */ MKSTR(slaves, "/sys/class/net/%s/master/bonding/slaves", ifname); - file = fopen(slaves, "r"); - if (file) + bond_file = fopen(slaves, "r"); + if (bond_file) break; } - if (!file) + if (!bond_file) return -1; /* Use safe format to check maximal buffer length. */ MLX5_ASSERT(atol(RTE_STR(IF_NAMESIZE)) == IF_NAMESIZE); - while (fscanf(file, "%" RTE_STR(IF_NAMESIZE) "s", ifname) == 1) { + while (fscanf(bond_file, "%" RTE_STR(IF_NAMESIZE) "s", ifname) == 1) { char tmp_str[IF_NAMESIZE + 32]; struct rte_pci_addr pci_addr; struct mlx5_switch_info info; @@ -1762,13 +1754,7 @@ mlx5_device_bond_pci_match(const struct ibv_device *ibv_dev, " for netdev \"%s\"", ifname); continue; } - if (pci_dev->domain != pci_addr.domain || - pci_dev->bus != pci_addr.bus || - pci_dev->devid != pci_addr.devid || - pci_dev->function + owner != pci_addr.function) - continue; /* Slave interface PCI address match found. */ - fclose(file); snprintf(tmp_str, sizeof(tmp_str), "/sys/class/net/%s/phys_port_name", ifname); file = fopen(tmp_str, "rb"); @@ -1777,13 +1763,52 @@ mlx5_device_bond_pci_match(const struct ibv_device *ibv_dev, info.name_type = MLX5_PHYS_PORT_NAME_TYPE_NOTSET; if (fscanf(file, "%32s", tmp_str) == 1) mlx5_translate_port_name(tmp_str, &info); - if (info.name_type == MLX5_PHYS_PORT_NAME_TYPE_LEGACY || - info.name_type == MLX5_PHYS_PORT_NAME_TYPE_UPLINK) + fclose(file); + /* Only process PF ports. */ + if (info.name_type != MLX5_PHYS_PORT_NAME_TYPE_LEGACY && + info.name_type != MLX5_PHYS_PORT_NAME_TYPE_UPLINK) + continue; + /* Check max bonding member. */ + if (info.port_name >= MLX5_BOND_MAX_PORTS) { + DRV_LOG(WARNING, "bonding index out of range, " + "please increase MLX5_BOND_MAX_PORTS: %s", + tmp_str); + break; + } + /* Match PCI address. */ + if (pci_dev->domain == pci_addr.domain && + pci_dev->bus == pci_addr.bus && + pci_dev->devid == pci_addr.devid && + pci_dev->function + owner == pci_addr.function) pf = info.port_name; - break; - } - if (file) + /* Get ifindex. */ + snprintf(tmp_str, sizeof(tmp_str), + "/sys/class/net/%s/ifindex", ifname); + file = fopen(tmp_str, "rb"); + if (!file) + break; + ret = fscanf(file, "%u", &ifindex); fclose(file); + if (ret != 1) + break; + /* Save bonding info. */ + strncpy(bond_info->ports[info.port_name].ifname, ifname, + sizeof(bond_info->ports[0].ifname)); + bond_info->ports[info.port_name].pci_addr = pci_addr; + bond_info->ports[info.port_name].ifindex = ifindex; + bond_info->n_port++; + } + if (pf >= 0) { + /* Get bond interface info */ + ret = mlx5_sysfs_bond_info(ifindex, &bond_info->ifindex, + bond_info->ifname); + if (ret) + DRV_LOG(ERR, "unable to get bond info: %s", + strerror(rte_errno)); + else + DRV_LOG(INFO, "PF device %u, bond device %u(%s)", + ifindex, bond_info->ifindex, bond_info->ifname); + } return pf; } @@ -1838,6 +1863,7 @@ mlx5_os_pci_probe_pf(struct rte_pci_device *pci_dev, unsigned int dev_config_vf; struct rte_eth_devargs eth_da = *req_eth_da; struct rte_pci_addr owner_pci = pci_dev->addr; /* Owner PF. */ + struct mlx5_bond_info bond_info; int ret = -1; if (rte_eal_process_type() == RTE_PROC_PRIMARY) @@ -1869,7 +1895,8 @@ mlx5_os_pci_probe_pf(struct rte_pci_device *pci_dev, DRV_LOG(DEBUG, "checking device \"%s\"", ibv_list[ret]->name); bd = mlx5_device_bond_pci_match - (ibv_list[ret], &owner_pci, nl_rdma, owner_id); + (ibv_list[ret], &owner_pci, nl_rdma, owner_id, + &bond_info); if (bd >= 0) { /* * Bonding device detected. Only one match is allowed, @@ -1978,6 +2005,7 @@ mlx5_os_pci_probe_pf(struct rte_pci_device *pci_dev, MLX5_ASSERT(nd == 1); MLX5_ASSERT(np); for (i = 1; i <= np; ++i) { + list[ns].bond_info = &bond_info; list[ns].max_port = np; list[ns].phys_port = i; list[ns].phys_dev = ibv_match[0]; @@ -2068,6 +2096,7 @@ mlx5_os_pci_probe_pf(struct rte_pci_device *pci_dev, */ for (i = 0; i != nd; ++i) { memset(&list[ns].info, 0, sizeof(list[ns].info)); + list[ns].bond_info = NULL; list[ns].max_port = 1; list[ns].phys_port = 1; list[ns].phys_dev = ibv_match[i]; diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index d613ffd655..e170db948d 100644 --- a/drivers/net/mlx5/mlx5.c +++ b/drivers/net/mlx5/mlx5.c @@ -927,6 +927,8 @@ mlx5_alloc_shared_dev_ctx(const struct mlx5_dev_spawn_data *spawn, rte_errno = ENOMEM; goto exit; } + if (spawn->bond_info) + sh->bond = *spawn->bond_info; err = mlx5_os_open_device(spawn, config, sh); if (!sh->ctx) goto error; diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index e7afa438ce..c15af1d794 100644 --- a/drivers/net/mlx5/mlx5.h +++ b/drivers/net/mlx5/mlx5.h @@ -115,6 +115,7 @@ struct mlx5_dev_spawn_data { void *phys_dev; /**< Associated physical device. */ struct rte_eth_dev *eth_dev; /**< Associated Ethernet device. */ struct rte_pci_device *pci_dev; /**< Backend PCI device. */ + struct mlx5_bond_info *bond_info; }; /** Key string for IPC. */ @@ -661,6 +662,21 @@ struct mlx5_flex_parser_profiles { void *obj; /* Flex parser node object. */ }; +/* Max member ports per bonding device. */ +#define MLX5_BOND_MAX_PORTS 2 + +/* Bonding device information. */ +struct mlx5_bond_info { + int n_port; /* Number of bond member ports. */ + uint32_t ifindex; + char ifname[MLX5_NAMESIZE + 1]; + struct { + char ifname[MLX5_NAMESIZE + 1]; + uint32_t ifindex; + struct rte_pci_addr pci_addr; + } ports[MLX5_BOND_MAX_PORTS]; +}; + /* * Shared Infiniband device context for Master/Representors * which belong to same IB device with multiple IB ports. @@ -671,6 +687,7 @@ struct mlx5_dev_ctx_shared { uint32_t devx:1; /* Opened with DV. */ uint32_t flow_hit_aso_en:1; /* Flow Hit ASO is supported. */ uint32_t max_port; /* Maximal IB device port index. */ + struct mlx5_bond_info bond; /* Bonding information. */ void *ctx; /* Verbs/DV/DevX context. */ void *pd; /* Protection Domain. */ uint32_t pdn; /* Protection Domain number. */ @@ -916,10 +933,8 @@ struct mlx5_priv { uint32_t vport_meta_tag; /* Used for vport index match ove VF LAG. */ uint32_t vport_meta_mask; /* Used for vport index field match mask. */ int32_t representor_id; /* RTE_ETH_REPR(), -1 if not a representor. */ - int32_t pf_bond; /* >=0 means PF index in bonding configuration. */ + int32_t pf_bond; /* >=0, representor owner PF index in bonding. */ unsigned int if_index; /* Associated kernel network device index. */ - uint32_t bond_ifindex; /**< Bond interface index. */ - char bond_name[MLX5_NAMESIZE]; /**< Bond interface name. */ /* RX/TX queues. */ unsigned int rxqs_n; /* RX queues array size. */ unsigned int txqs_n; /* TX queues array size. */ diff --git a/drivers/net/mlx5/mlx5_ethdev.c b/drivers/net/mlx5/mlx5_ethdev.c index 5341eb16c9..29389fc98f 100644 --- a/drivers/net/mlx5/mlx5_ethdev.c +++ b/drivers/net/mlx5/mlx5_ethdev.c @@ -42,7 +42,10 @@ mlx5_ifindex(const struct rte_eth_dev *dev) MLX5_ASSERT(priv); MLX5_ASSERT(priv->if_index); - ifindex = priv->bond_ifindex > 0 ? priv->bond_ifindex : priv->if_index; + if (priv->master && priv->sh->bond.ifindex > 0) + ifindex = priv->sh->bond.ifindex; + else + ifindex = priv->if_index; if (!ifindex) rte_errno = ENXIO; return ifindex; From patchwork Tue Jan 19 07:28:13 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xueming Li X-Patchwork-Id: 86866 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 9EF9FA0A03; Tue, 19 Jan 2021 08:30:01 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 0C2F8140D48; Tue, 19 Jan 2021 08:29:57 +0100 (CET) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by mails.dpdk.org (Postfix) with ESMTP id DDD19140CF2 for ; Tue, 19 Jan 2021 08:29:53 +0100 (CET) Received: from Internal Mail-Server by MTLPINE1 (envelope-from xuemingl@nvidia.com) with SMTP; 19 Jan 2021 09:29:53 +0200 Received: from nvidia.com (pegasus05.mtr.labs.mlnx [10.210.16.100]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 10J7Tpqq014531; Tue, 19 Jan 2021 09:29:52 +0200 From: Xueming Li To: Viacheslav Ovsiienko Cc: dev@dpdk.org, xuemingl@nvidia.com, Asaf Penso Date: Tue, 19 Jan 2021 07:28:13 +0000 Message-Id: <1611041295-12797-8-git-send-email-xuemingl@nvidia.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1611041295-12797-1-git-send-email-xuemingl@nvidia.com> References: <1611041295-12797-1-git-send-email-xuemingl@nvidia.com> In-Reply-To: <1608304614-13908-2-git-send-email-xuemingl@nvidia.com> References: <1608304614-13908-2-git-send-email-xuemingl@nvidia.com> Subject: [dpdk-dev] [PATCH v4 7/8] net/mlx5: fix setting VF default MAC through representor X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" With kernel bonding, there was an error when setting VF MAC address through representor. The Netlink api requires ifindex of owner PF, not bonding device ifindex. Uses owner PF ifindex to modify VF default MAC in case of bonding device. Fixes: c21e5facf7d2 ("net/mlx5: use bond index for netdev operations") Signed-off-by: Xueming Li Acked-by: Viacheslav Ovsiienko --- drivers/net/mlx5/mlx5_mac.c | 24 +++++++++++++++--------- 1 file changed, 15 insertions(+), 9 deletions(-) diff --git a/drivers/net/mlx5/mlx5_mac.c b/drivers/net/mlx5/mlx5_mac.c index b5b810b508..5a3aec89c1 100644 --- a/drivers/net/mlx5/mlx5_mac.c +++ b/drivers/net/mlx5/mlx5_mac.c @@ -154,6 +154,7 @@ mlx5_mac_addr_set(struct rte_eth_dev *dev, struct rte_ether_addr *mac_addr) { uint16_t port_id; struct mlx5_priv *priv = dev->data->dev_private; + struct mlx5_priv *pf_priv; /* * Configuring the VF instead of its representor, @@ -162,19 +163,24 @@ mlx5_mac_addr_set(struct rte_eth_dev *dev, struct rte_ether_addr *mac_addr) if (priv->representor && !mlx5_is_hpf(dev)) { DRV_LOG(DEBUG, "VF represented by port %u setting primary MAC address", dev->data->port_id); + if (priv->pf_bond >= 0) { + /* Bonding, get owner PF ifindex from shared data. */ + return mlx5_os_vf_mac_addr_modify + (priv, + priv->sh->bond.ports[priv->pf_bond].ifindex, + mac_addr, + rte_eth_representor_id_parse( + priv->representor_id, + NULL, NULL, NULL)); + } RTE_ETH_FOREACH_DEV_SIBLING(port_id, dev->data->port_id) { - priv = rte_eth_devices[port_id].data->dev_private; - if (priv->master == 1) { - priv = dev->data->dev_private; + pf_priv = rte_eth_devices[port_id].data->dev_private; + if (pf_priv->master == 1) return mlx5_os_vf_mac_addr_modify - (priv, - mlx5_ifindex(&rte_eth_devices[port_id]), - mac_addr, + (priv, pf_priv->if_index, mac_addr, rte_eth_representor_id_parse( priv->representor_id, - NULL, NULL, NULL) - ); - } + NULL, NULL, NULL)); } rte_errno = -ENOTSUP; return rte_errno;