From patchwork Mon Oct 9 18:46:59 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yongseok Koh X-Patchwork-Id: 30011 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 1C2E91B287; Mon, 9 Oct 2017 20:47:33 +0200 (CEST) Received: from EUR03-AM5-obe.outbound.protection.outlook.com (mail-eopbgr30066.outbound.protection.outlook.com [40.107.3.66]) by dpdk.org (Postfix) with ESMTP id 96A1A1B267; Mon, 9 Oct 2017 20:47:26 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=zSDH78o7a/F+0Pxqo7XCGJiUAFEEizrhKJIzmrDEZ7s=; b=uMdZxtV0UQQaqa0oecGIziQIqFFpzPYuUPBKWS0UBIrlFofF4ArbebhS/QiQhQmTFYKvg8ZGlyPIAGx2YSzYV7z+N+QmbhydLXRiej3svija/A+fndIKR+6GV8fRiNtnitrdimpocaVJ5yVpFerqUupX1KxeUxNKp9ySg4GIRH0= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=yskoh@mellanox.com; Received: from mellanox.com (209.116.155.178) by VI1PR0501MB2046.eurprd05.prod.outlook.com (2603:10a6:800:36::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.77.7; Mon, 9 Oct 2017 18:47:23 +0000 From: Yongseok Koh To: adrien.mazarguil@6wind.com, nelio.laranjeiro@6wind.com Cc: dev@dpdk.org, Yongseok Koh , stable@dpdk.org Date: Mon, 9 Oct 2017 11:46:59 -0700 Message-Id: <6c4d67063ddbd00cf14525007867e3acbb667d2c.1507574644.git.yskoh@mellanox.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: References: <20171005230032.7548-1-yskoh@mellanox.com> MIME-Version: 1.0 X-Originating-IP: [209.116.155.178] X-ClientProxiedBy: MWHPR2201CA0009.namprd22.prod.outlook.com (2603:10b6:301:28::22) To VI1PR0501MB2046.eurprd05.prod.outlook.com (2603:10a6:800:36::20) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: bb0242d5-8aee-4d0b-d26f-08d50f462ebc X-MS-Office365-Filtering-HT: Tenant X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001)(2017030254152)(48565401081)(2017052603199)(201703131423075)(201703031133081)(201702281549075); SRVR:VI1PR0501MB2046; X-Microsoft-Exchange-Diagnostics: 1; VI1PR0501MB2046; 3:IOKOk+vMTygNOeMU2madUhfMjuffhIf/o2byj8eMOrHO8P+P9YKVpQi9BEYYw4kFRXTo4WdntJDXJU7Q0NoE/lSOZfhzE92Mu+UMDuveErhz+dyqU2r6IGUd9OG/0GlrWBkgEWC2+lT1l9u6tyWYCnxQdxp/BfyuZgNeKyZEhFtg9hMYTPbSL2gVhQ573m7z++tODDB896SxL58Fo8KJb1Igko/foG/w9n98bGYMXAH0GsVzqUiXLX0+Y/ywv6Jj; 25:yhgoJjR7iAC6vM8Ya2idvfUDsKLEOgqW14T5WfMvyjohZtHPeMMDbOGRb9ttwQG0N+WOdLfO6oTAd9duJi5fo16nv6b/hslS4yYseuV1c+HtDi4iBm97Vsoh5xr8gV7XW2fpw3L5S9/GIYa9YCnHrpJJEYH1qZZETBztLABsipbZx8udgqAyT7Jz52yLXL4HWAG8tgJcWv/Js6TLaFO1XjJ66hfpUeoJaWmdoL/acZjMF9tKNruacPNs80pSRVUWJ65biNKo8Pk0qoQja83VgQf4y1c5sbss76s6RCLiF3l4KI5MvHuGd6fqOIdyQl6rFLe1ZJ9uhE1a0iJNgbytSA==; 31:wE3ImSQnh4Xh1iudgQnFLDEGwTgFZMyH4EBIJpr1Onrlm8j+EOUaPy1gxV2PfuhqbXevdNGbMJ2JefvPxpA1jW5VT/YDBjSVaxxWH0lrH4kuYymGA5u3xpjOGhgfcSNDIXJXq27b4GfkPKqNrI4J4RZtrMqkyhgaiMc6rCK+geitmgnMtvyjffGpMg0j+BUqQLSz9Q6aZYKIH4GA8PBoJWYDtq0qBY39z7BqwBle0Js= X-MS-TrafficTypeDiagnostic: VI1PR0501MB2046: X-LD-Processed: a652971c-7d2e-4d9b-a6a4-d149256f461b,ExtAddr X-Microsoft-Exchange-Diagnostics: 1; VI1PR0501MB2046; 20:DgXiFz8Chzp4vUt8bnIze9VCIo+SJeeKtl46Iw784B9gQohCE3GZqqD8D3YsHRFBlZt4J/abTiVfgADBDnIVabFTKayNxqApIx+GbEZuoAreljXlNRfgHEkis4plDh51xz8OAYIyMw9taJ4UavinOYXlN7TXI1H6zgCJ/Huk2mnEAzqjqOdNlBGzYftCywFXah1BBTNcRqlcIBWoGN4qZFvTu5PCNnEEZ+eaKfNZSP82FIU3wHdifN9KhabENjq5fUdNxNcEUh+Zy+zfL96C9HnnYSEMaFzMOmOWmMuhPjgzW0R6x/m5dwSxPHvgAeT4H+brcIDUQ8Jqt589tyDuLUsELSO5u0anNt+6GSzYv94k3guQ87+0OGNBy1b4jo6I1hRLkePaGiBktbdEQyFAIcZGUVLplpEKnFa/tl1N3KIGT2RHHUDlulNFju+VaSB45CbUoHVKFRs6dSGO5BtYnsoBup83tEKdxEZEKHc7J/P1xlB/PaO4pKG9sxII8ACu; 4:VoKN6AH0qQ7VY6opZIHnJWkGKlUKT39MNDpxleqfQPjj9L2YjhS90uLsEUhYcGNRIgtb2ofp9SCi3Ei3KesMPbn+oNXIPKFae8Ojtgv8rsDoaSQAmVwQksYxHXkd6cPMm5rV1IqUJy/2RDUzjo8uHJ68g6JOW0IH5lgH36u6k+S+8FzwDxuhEv2sZbqAWYK9st8Uk/0I4nol8TFHenyYhnVwsIxbVrx9wb9y/eLuuDxo71swPt27gyxgWsmntAvz X-Exchange-Antispam-Report-Test: UriScan:; X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(2401047)(5005006)(8121501046)(3002001)(93006095)(93001095)(100000703101)(100105400095)(10201501046)(6055026)(6041248)(20161123564025)(20161123562025)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123558100)(20161123560025)(20161123555025)(6072148)(201708071742011)(100000704101)(100105200095)(100000705101)(100105500095); SRVR:VI1PR0501MB2046; BCL:0; PCL:0; RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095); SRVR:VI1PR0501MB2046; X-Forefront-PRVS: 045584D28C X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(6009001)(376002)(39860400002)(346002)(199003)(189002)(50466002)(21086003)(66066001)(118296001)(4326008)(2906002)(55016002)(2950100002)(478600001)(575784001)(7736002)(53936002)(189998001)(6666003)(81156014)(69596002)(47776003)(86362001)(305945005)(16526018)(5660300001)(5003940100001)(8676002)(106356001)(316002)(36756003)(8936002)(25786009)(50226002)(50986999)(97736004)(16586007)(76176999)(101416001)(33646002)(3846002)(68736007)(6116002)(48376002)(81166006)(105586002)(19627235001); DIR:OUT; SFP:1101; SCL:1; SRVR:VI1PR0501MB2046; H:mellanox.com; FPR:; SPF:None; PTR:InfoNoRecords; A:1; MX:1; LANG:en; Received-SPF: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; VI1PR0501MB2046; 23:JFb4O/nkOqXXIU3YnuxMYOoI1UQj/ds257H4dZa?= DiqZrAgZV4LLYrQnmG3QDH1CsRb+2fiMzMbzlQmoZ3fPdol74hq5dwoSfO4Crb0BmpdKgf6F/azvGSjsK2OiIpImS3v62LgAN2/mfSA4TES1fXqfxVTLDRnsrtV48+H5SkmAHGRcIw2H2YAbp3UfkAcLs0/y/56da3GAeab/qqi3rTQbpAtvM81EZM9RRO+BbkS/wKjhJ3ROCOz4AxgM19l6oOXUJriYZEHbmCIrHFRrXk5rzNhx6hSrRxo06Z2xji+Hc4gL5QR4IcQN7HZzWCC6WS7wgbiFoNwzYB/LqELzkyxIfk0S5PWMOGhacRoCSRzz29nPlhIH5QDm915ZZRzPv+jqrfX/Yr46xErzTPQVv4iJ8MC2veVXI2ZqgUflbiKduWxtQhh42siZqaS8kvhqq8GDL2xKZRsWCePW1UYNvfrcyhHq3wsfuQlH4XONJVVtkse9LzT2lTY5v0wwEYTFSWFSEuea/ejWPS1cgYhyuwnpOILoO9TDYPQphTgjaZD7+SWBtT/sTEBdXHuWFGmQk8jDVvcaAFDlY7rPEb4VCvVJIUOmpAxHU+2EYEXGLNvo9L1IuJmrnn+/e4rbsYgk418rdr+t3d94IJmLj3YGHopTuY5lSqnRr7Ae1+B4JsbLbV6I1K8oWhGtCGNlaUyBpJpKxzwbgJ9Tw/wpT9WhEdXtV1oBcBXYe9AMuzCRFlBhIDVA+c9OSWA3nVSZ2PuczktDBJxaZBoj77OE9dYdFGhrKftuuzvwvxrLs0NjSPyOctwSCbP2ASlcAYbApCqunGPgFrTeETueW41uXF1z3UGIANJpq5q+NKRV3fdIgYVusAyRJmcRWETDqpGRDBABq80ttZ38QJ2MSSDWLZo1ZXhXE20/1ilw4a0ishQRtmT/YZce2VsOrubmJjNapx3J39+OWHWZWAvNE5lp4H1zEI0XKp6fj+8K2I9dtpgZGBnDaDMb47WFDxfU035rwFDeEpHSyj3ZCjqHCJ2zTZHfJU8tQNSC1399xD4eHLHS8kmz/PgmsVRNrQ+O+OWIQOR7a9beNDRYm2ZAldLUmBoFoxBUmsEiCaMIDyNThVuSd5WBsJY7N6S/y9IHLobITiSK4 X-Microsoft-Exchange-Diagnostics: 1; VI1PR0501MB2046; 6:NpOXfuynFCNlOn5di8tK5BygzBldmRTL3F9FWAAQkxWQQQhPn9DpOYfW2ucf4vFMzrSCI80pwRV+WOq9qX0P6u7XAZAV/2kyuql3w5brIhayVdjOzJqPyQQc1dqCRWNASWV5izLb1hkGThm6Tfc7CxkrQh6aZQSZua9tq2PQm6jBx53izzoW0dpLJqVTG3m/dmTMjfdJi1P0AjjskIbi9IaOumztRnk5uPR/ZCFF192tq43/w4ZMMlG3BU5phMbkHy2DGU4Eh4RkqoKApFtopW6aR7dRMwNiEFdtdsrZ0WhIF+zphEH4NbsGvV0WzhrV3Fyp3sOxOIpMFG35y1l6vQ==; 5:Z/tY6qB0HxeCZQY3hboD6Pl7lc05AoolZUSYLhA/V9KCzsRMQlVtkHTKLst75neK4AKpZ3ZdLYSpPmZ5jB4IQ63W5qRzEVwFkL+BzTbOX1cej1K0u1dDmRabHhPsGMWieXECbbBuICNqmdukhDXsjAYlvZHIdGNzoesYf3BwpL8=; 24:0V+AuqO+Fq6EQKSSfVh48MocuR+xGbEQhWawPWXjVlRHwE2XzKtPl7FEtbmg88rbdBohy7Dqilr8dmnX3hDsNcCuRyUqNIICt41Gl80kvDg=; 7:DFT5ipLnGMz2aGN87LOyFcqkSG9WdCchXGcJn4RG5TKb16zFgcq34pvI2hKczJqaWjmeKJJuqtKq+XiBu8oDPuyDCa35lzWuCqzRBy7Qr5ptn58S96g2ZabWZGH8H7Jji5xAMgdxAkvuD/EuUl8vcV8jnGE5oi8GF0XhMETpXfQDhwau4iYFEkyv4jpAI5j0qK6hzLMve4OGNX/QplTuIFqB2P1c/CrI/h40KghC81Q= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Oct 2017 18:47:23.6077 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0501MB2046 Subject: [dpdk-dev] [PATCH v2 6/7] net/mlx5: fix configuration of Rx CQE compression X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" With the upstream rdma-core, to enable Rx CQE compression, mlx5dv_create_cq() in Direct Verbs has to be used instead of regular Verbs call (ibv_create_cq()). And if the size of CQE is 128 bytes, compression is supported only by certain devices. Thus, it has to be decided by checking the capabilitiy bits. Fixes: bba710e6b99b ("net/mlx5: support upstream rdma-core") Cc: stable@dpdk.org Signed-off-by: Yongseok Koh --- drivers/net/mlx5/Makefile | 5 +++++ drivers/net/mlx5/mlx5.c | 16 +++++++++++++++- drivers/net/mlx5/mlx5_rxq.c | 20 +++++++++++++++----- 3 files changed, 35 insertions(+), 6 deletions(-) diff --git a/drivers/net/mlx5/Makefile b/drivers/net/mlx5/Makefile index e64702cb4..bd223e815 100644 --- a/drivers/net/mlx5/Makefile +++ b/drivers/net/mlx5/Makefile @@ -124,6 +124,11 @@ mlx5_autoconf.h.new: $(RTE_SDK)/buildtools/auto-config-h.sh enum MLX5DV_CONTEXT_FLAGS_MPW_ALLOWED \ $(AUTOCONF_OUTPUT) $Q sh -- '$<' '$@' \ + HAVE_IBV_MLX5_MOD_CQE_128B_COMP \ + infiniband/mlx5dv.h \ + enum MLX5DV_CONTEXT_FLAGS_CQE_128B_COMP \ + $(AUTOCONF_OUTPUT) + $Q sh -- '$<' '$@' \ HAVE_ETHTOOL_LINK_MODE_25G \ /usr/include/linux/ethtool.h \ enum ETHTOOL_LINK_MODE_25000baseCR_Full_BIT \ diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index e1aa9b914..c0f7b1b76 100644 --- a/drivers/net/mlx5/mlx5.c +++ b/drivers/net/mlx5/mlx5.c @@ -101,6 +101,10 @@ #define MLX5DV_CONTEXT_FLAGS_ENHANCED_MPW (1 << 3) #endif +#ifndef HAVE_IBV_MLX5_MOD_CQE_128B_COMP +#define MLX5DV_CONTEXT_FLAGS_CQE_128B_COMP (1 << 4) +#endif + struct mlx5_args { int cqe_comp; int txq_inline; @@ -539,6 +543,7 @@ mlx5_pci_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) struct ibv_device_attr_ex device_attr; unsigned int sriov; unsigned int mps; + unsigned int cqe_comp; unsigned int tunnel_en = 0; int idx; int i; @@ -642,6 +647,11 @@ mlx5_pci_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) INFO("MPW is disabled\n"); mps = MLX5_MPW_DISABLED; } + if (RTE_CACHE_LINE_SIZE == 128 && + !(attrs_out.flags & MLX5DV_CONTEXT_FLAGS_CQE_128B_COMP)) + cqe_comp = 0; + else + cqe_comp = 1; if (ibv_query_device_ex(attr_ctx, NULL, &device_attr)) goto error; INFO("%u port(s) detected", device_attr.orig_attr.phys_port_cnt); @@ -758,7 +768,7 @@ mlx5_pci_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) priv->pd = pd; priv->mtu = ETHER_MTU; priv->mps = mps; /* Enable MPW by default if supported. */ - priv->cqe_comp = 1; /* Enable compression by default. */ + priv->cqe_comp = cqe_comp; priv->tunnel_en = tunnel_en; /* Enable vector by default if supported. */ priv->tx_vec_en = 1; @@ -847,6 +857,10 @@ mlx5_pci_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) priv->txq_inline = MLX5_WQE_SIZE_MAX - MLX5_WQE_SIZE; } + if (priv->cqe_comp && !cqe_comp) { + WARN("Rx CQE compression isn't supported"); + priv->cqe_comp = 0; + } /* Configure the first MAC address by default. */ if (priv_get_mac(priv, &mac.addr_bytes)) { ERROR("cannot get MAC address, is mlx5_en loaded?" diff --git a/drivers/net/mlx5/mlx5_rxq.c b/drivers/net/mlx5/mlx5_rxq.c index e7ec1dae3..e1867cb60 100644 --- a/drivers/net/mlx5/mlx5_rxq.c +++ b/drivers/net/mlx5/mlx5_rxq.c @@ -558,7 +558,10 @@ mlx5_priv_rxq_ibv_new(struct priv *priv, uint16_t idx) container_of(rxq_data, struct mlx5_rxq_ctrl, rxq); struct ibv_wq_attr mod; union { - struct ibv_cq_init_attr_ex cq; + struct { + struct ibv_cq_init_attr_ex ibv; + struct mlx5dv_cq_init_attr mlx5; + } cq; struct ibv_wq_init_attr wq; struct ibv_cq_ex cq_attr; } attr; @@ -597,12 +600,18 @@ mlx5_priv_rxq_ibv_new(struct priv *priv, uint16_t idx) goto error; } } - attr.cq = (struct ibv_cq_init_attr_ex){ + attr.cq.ibv = (struct ibv_cq_init_attr_ex){ + .cqe = cqe_n, + .channel = tmpl->channel, + .comp_mask = 0, + }; + attr.cq.mlx5 = (struct mlx5dv_cq_init_attr){ .comp_mask = 0, }; if (priv->cqe_comp) { - attr.cq.comp_mask |= IBV_CQ_INIT_ATTR_MASK_FLAGS; - attr.cq.flags |= MLX5DV_CQ_INIT_ATTR_MASK_COMPRESSED_CQE; + attr.cq.mlx5.comp_mask |= + MLX5DV_CQ_INIT_ATTR_MASK_COMPRESSED_CQE; + attr.cq.mlx5.cqe_comp_res_format = MLX5DV_CQE_RES_FORMAT_HASH; /* * For vectorized Rx, it must not be doubled in order to * make cq_ci and rq_ci aligned. @@ -610,7 +619,8 @@ mlx5_priv_rxq_ibv_new(struct priv *priv, uint16_t idx) if (rxq_check_vec_support(rxq_data) < 0) cqe_n *= 2; } - tmpl->cq = ibv_create_cq(priv->ctx, cqe_n, NULL, tmpl->channel, 0); + tmpl->cq = ibv_cq_ex_to_cq(mlx5dv_create_cq(priv->ctx, &attr.cq.ibv, + &attr.cq.mlx5)); if (tmpl->cq == NULL) { ERROR("%p: CQ creation failure", (void *)rxq_ctrl); goto error;