From patchwork Thu Apr 26 16:26:16 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adrien Mazarguil X-Patchwork-Id: 39052 X-Patchwork-Delegate: shahafs@mellanox.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 41627AA9E; Thu, 26 Apr 2018 18:26:35 +0200 (CEST) Received: from mail-wm0-f68.google.com (mail-wm0-f68.google.com [74.125.82.68]) by dpdk.org (Postfix) with ESMTP id 8C196A495 for ; Thu, 26 Apr 2018 18:26:31 +0200 (CEST) Received: by mail-wm0-f68.google.com with SMTP id a8so14043354wmg.5 for ; Thu, 26 Apr 2018 09:26:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=6wind-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=N66OGSiKNyo1Cp9fZnZJj6U6qYWlFvICev5Me2q2D5I=; b=KhLdWiRr8nfofzzhuKaSj4r24PXbB6oAp0zlDWPSKpRtQAnMDMUqwk7PTCx7bMCsZW 7cY+jTmIZtrO3IOglx6alFiqNmO94H3NMSrLGjDAj8jeV0T/eE7gD2GaeYpGJFmy7WYM Bfxog1wGcAnzj//Zf/i/MPZjnxeBvOCEhb4rSihvoXgso7XpdaVUWZtPlDEL5Q91uCev OuLzpW30JEs8gZF3hyQttHUziGhc8yBbAhYdyNftZGOFOeGAmM6MtlLIjlMVxLlIAXA8 +7u8V4LxYLPIBvfMKHpzTou1M/1Y17WkmVueIryMWHI3aiAShBtpN0WSSnCnOPMKTNxx oXOw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=N66OGSiKNyo1Cp9fZnZJj6U6qYWlFvICev5Me2q2D5I=; b=gAjdS5aXr+IrTguYXBkUn+ocCO0DU0fUiZv4UKqbOspEEi6OWugp3jytgsxQK1KtFT 8vxgRApLNvCP9acZuOoilHIqFUi0HLdZYJD80ixjHUBYNsXSk5X/BIhLqPEDwDYVWQ7S FHUfABNc9qt2P8a8AVyDDmnTsKBpqqRgiYWlC0+6B+whQ2uRIYgLRC6+I9jQS7C6kV+J 3olfz0yxlUjQuXfVwsqn38IO7lC1diDMNh/hKRliZcdEkCD4q+EJGvTz/+1s/KacqP8x WazeplA5gpQHSJMpQKnJPOZtKp9i3p458gZOjUrGobVMtF7coIuMxEfbM8GrgiT1RoBH KcyA== X-Gm-Message-State: ALQs6tA4rkNsdXJ+u2Pe7YgZN9rgj3l6HBUdL8D0+YN91tNcT9f01xYl 1TinOcKnQ6yFChgaEed2jb2IHw== X-Google-Smtp-Source: AB8JxZptWhsEFyRaH3pJR1FOD00s32oM/enjV/UCvkEmW1m16Ir4HYeMeYEOj04NJwb+1F/8OXOHtA== X-Received: by 10.28.69.26 with SMTP id s26mr19691454wma.19.1524759991204; Thu, 26 Apr 2018 09:26:31 -0700 (PDT) Received: from 6wind.com (host.78.145.23.62.rev.coltfrance.com. [62.23.145.78]) by smtp.gmail.com with ESMTPSA id 55-v6sm40752450wrw.52.2018.04.26.09.26.30 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 26 Apr 2018 09:26:30 -0700 (PDT) Date: Thu, 26 Apr 2018 18:26:16 +0200 From: Adrien Mazarguil To: Shahaf Shuler Cc: Ferruh Yigit , dev@dpdk.org, stable@dpdk.org Message-ID: <20180426162142.6617-3-adrien.mazarguil@6wind.com> References: <20180425153739.9157-1-adrien.mazarguil@6wind.com> <20180426162142.6617-1-adrien.mazarguil@6wind.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20180426162142.6617-1-adrien.mazarguil@6wind.com> X-Mailer: git-send-email 2.11.0 Subject: [dpdk-dev] [PATCH v2 3/3] net/mlx4: fix inner RSS support for broken kernels X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Linux 4.15 and 4.16 may report inner RSS as a supported capability of the device, however it can't be used due to missing code in the kernel. This triggers an error when creating the default hash QP and prevents this PMD from starting up without a prior call to rte_flow_isolate(). Fixes: 55e8991e3199 ("net/mlx4: restore inner VXLAN RSS support") Cc: stable@dpdk.org Signed-off-by: Adrien Mazarguil --- drivers/net/mlx4/mlx4.c | 107 ++++++++++++++++++++++++++++++++++++++----- 1 file changed, 95 insertions(+), 12 deletions(-) diff --git a/drivers/net/mlx4/mlx4.c b/drivers/net/mlx4/mlx4.c index 4e472fa1d..c1cc95786 100644 --- a/drivers/net/mlx4/mlx4.c +++ b/drivers/net/mlx4/mlx4.c @@ -387,6 +387,99 @@ mlx4_args(struct rte_devargs *devargs, struct mlx4_conf *conf) return ret; } +/** + * Interpret RSS capabilities reported by device. + * + * This function returns the set of usable Verbs RSS hash fields, kernel + * quirks taken into account. + * + * @param ctx + * Verbs context. + * @param pd + * Verbs protection domain. + * @param device_attr_ex + * Extended device attributes to interpret. + * + * @return + * Usable RSS hash fields mask in Verbs format. + */ +static uint64_t +mlx4_hw_rss_sup(struct ibv_context *ctx, struct ibv_pd *pd, + struct ibv_device_attr_ex *device_attr_ex) +{ + uint64_t hw_rss_sup = device_attr_ex->rss_caps.rx_hash_fields_mask; + struct ibv_cq *cq = NULL; + struct ibv_wq *wq = NULL; + struct ibv_rwq_ind_table *ind = NULL; + struct ibv_qp *qp = NULL; + + if (!hw_rss_sup) { + WARN("no RSS capabilities reported; disabling support for UDP" + " RSS and inner VXLAN RSS"); + return IBV_RX_HASH_SRC_IPV4 | IBV_RX_HASH_DST_IPV4 | + IBV_RX_HASH_SRC_IPV6 | IBV_RX_HASH_DST_IPV6 | + IBV_RX_HASH_SRC_PORT_TCP | IBV_RX_HASH_DST_PORT_TCP; + } + if (!(hw_rss_sup & IBV_RX_HASH_INNER)) + return hw_rss_sup; + /* + * Although reported as supported, missing code in some Linux + * versions (v4.15, v4.16) prevents the creation of hash QPs with + * inner capability. + * + * There is no choice but to attempt to instantiate a temporary RSS + * context in order to confirm its support. + */ + cq = mlx4_glue->create_cq(ctx, 1, NULL, NULL, 0); + wq = cq ? mlx4_glue->create_wq + (ctx, + &(struct ibv_wq_init_attr){ + .wq_type = IBV_WQT_RQ, + .max_wr = 1, + .max_sge = 1, + .pd = pd, + .cq = cq, + }) : NULL; + ind = wq ? mlx4_glue->create_rwq_ind_table + (ctx, + &(struct ibv_rwq_ind_table_init_attr){ + .log_ind_tbl_size = 0, + .ind_tbl = &wq, + .comp_mask = 0, + }) : NULL; + qp = ind ? mlx4_glue->create_qp_ex + (ctx, + &(struct ibv_qp_init_attr_ex){ + .comp_mask = + (IBV_QP_INIT_ATTR_PD | + IBV_QP_INIT_ATTR_RX_HASH | + IBV_QP_INIT_ATTR_IND_TABLE), + .qp_type = IBV_QPT_RAW_PACKET, + .pd = pd, + .rwq_ind_tbl = ind, + .rx_hash_conf = { + .rx_hash_function = IBV_RX_HASH_FUNC_TOEPLITZ, + .rx_hash_key_len = MLX4_RSS_HASH_KEY_SIZE, + .rx_hash_key = mlx4_rss_hash_key_default, + .rx_hash_fields_mask = hw_rss_sup, + }, + }) : NULL; + if (!qp) { + WARN("disabling unusable inner RSS capability due to kernel" + " quirk"); + hw_rss_sup &= ~IBV_RX_HASH_INNER; + } else { + claim_zero(ibv_destroy_qp(qp)); + } + if (ind) + claim_zero(ibv_destroy_rwq_ind_table(ind)); + if (wq) + claim_zero(ibv_destroy_wq(wq)); + if (cq) + claim_zero(ibv_destroy_cq(cq)); + return hw_rss_sup; +} + static struct rte_pci_driver mlx4_driver; /** @@ -565,18 +658,8 @@ mlx4_pci_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) PCI_DEVICE_ID_MELLANOX_CONNECTX3PRO); DEBUG("L2 tunnel checksum offloads are %ssupported", priv->hw_csum_l2tun ? "" : "not "); - priv->hw_rss_sup = device_attr_ex.rss_caps.rx_hash_fields_mask; - if (!priv->hw_rss_sup) { - WARN("no RSS capabilities reported; disabling support" - " for UDP RSS and inner VXLAN RSS"); - priv->hw_rss_sup = - IBV_RX_HASH_SRC_IPV4 | - IBV_RX_HASH_DST_IPV4 | - IBV_RX_HASH_SRC_IPV6 | - IBV_RX_HASH_DST_IPV6 | - IBV_RX_HASH_SRC_PORT_TCP | - IBV_RX_HASH_DST_PORT_TCP; - } + priv->hw_rss_sup = mlx4_hw_rss_sup(priv->ctx, priv->pd, + &device_attr_ex); DEBUG("supported RSS hash fields mask: %016" PRIx64, priv->hw_rss_sup); priv->hw_fcs_strip = !!(device_attr_ex.raw_packet_caps &