From patchwork Thu Oct 6 11:00:59 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dariusz Sosnowski X-Patchwork-Id: 117450 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 52B23A00C2; Thu, 6 Oct 2022 13:02:18 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 7C33442C09; Thu, 6 Oct 2022 13:02:09 +0200 (CEST) Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2047.outbound.protection.outlook.com [40.107.220.47]) by mails.dpdk.org (Postfix) with ESMTP id F1F0942B70 for ; Thu, 6 Oct 2022 13:02:07 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=H8Y605A9OkEeeCg/kzBtuLeF97jUWudkzjMnejx++n4qoTwiixzjsfTfgzot2KwCNTaM24t+U6eAYHo8/x/PRuEFiU57+/mxMEzbs/q45spNrcQvYXnEiB9pDTycWjgo69dXdFCny1lIva8DLEkv3vr/H9cr3pFQNvOvVmuc6M07mcpQSPaFWvQmj6pTon/OlR24MB4gP4FbN80NGcwdgUOV8dhSUMC90qjejPcN5pGrfsmS6e9bgXdXhD/6ZWwv8Hnpgjznze7Vti4dx8qDXDjM7ZKTvaVhehcCzAr5IzC6q6nEi5WlYKyfXC1LR/vFhfuInOsQQwY7PPG0u3F74g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=89I2iS9IgrjmNUYpYX8WUXBc4kdf4p8OUOLnHkMS5t8=; b=dDYoL2TsVtQfxaM7QG0nfcKXs6XIHzNcl+MsaleEF3PCPFhw6VZ579k4gkDWHgEaYmcFa9oGG0DaD4YPB9LiPFLaJsVy44uu+HBb7MzFfB00UPsohJuKVtKFesPmAPoYxoJhTvnq7LbQDJ55+PKtltaJ4irR76F58pZwffQY4w0IFzhUQS2UNlyErUn7Bkm0rCwRpb0f4uSly3x1LMJ2RCJ+s4iOSsfVoBMxhNeZuLiEhrSIOEAnZLcO9fl0uzgHB1xKvw4ubepOX7rZTO1hK3B+lqpjp9tAArrW5vKsNu4+AaRBEUwqBU5ihizDHcWUY3VoWAd9QEa8lDnOm57b0w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=temperror (sender ip is 216.228.117.160) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=temperror action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=89I2iS9IgrjmNUYpYX8WUXBc4kdf4p8OUOLnHkMS5t8=; b=U3Nm2neU1oYmfXWQ2b1XCh9s31Ggj4ibbZYWJl9cFIOxfV6DZ4hQR1mOhwEdBOBhg4dse1C70dCvV4zlbBp0A2xHLzh1DtQl+Rwi0XkNYIUgz7yY9XXJ6uGSnth1zYkopaDtcxMoD2P5jXD6bN58OG0hVCqbZcwzjwkp+8ELn6hdDCUyssT09I0KTKMxVqykB7piCn0i7XaO9yTwuBRGqHGDCMrhtUMUzNdBLRqhvbwTS+jHrwRyQJjypv2PLvnENyT+xgU6e5F/BDG78cwvuc15RDq7E8mtlQBqk8NzF1moz8M+3GOBc/fIO4ULQh2eDNgoh3gghdfOf1Q6gJU4aQ== Received: from BN0PR04CA0193.namprd04.prod.outlook.com (2603:10b6:408:e9::18) by BL3PR12MB6427.namprd12.prod.outlook.com (2603:10b6:208:3b6::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5676.32; Thu, 6 Oct 2022 11:02:05 +0000 Received: from BN8NAM11FT020.eop-nam11.prod.protection.outlook.com (2603:10b6:408:e9:cafe::fc) by BN0PR04CA0193.outlook.office365.com (2603:10b6:408:e9::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5676.23 via Frontend Transport; Thu, 6 Oct 2022 11:02:05 +0000 X-MS-Exchange-Authentication-Results: spf=temperror (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=temperror action=none header.from=nvidia.com; Received-SPF: TempError (protection.outlook.com: error in processing during lookup of nvidia.com: DNS Timeout) Received: from mail.nvidia.com (216.228.117.160) by BN8NAM11FT020.mail.protection.outlook.com (10.13.176.223) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5709.10 via Frontend Transport; Thu, 6 Oct 2022 11:02:03 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.26; Thu, 6 Oct 2022 04:01:52 -0700 Received: from nvidia.com (10.126.230.35) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.29; Thu, 6 Oct 2022 04:01:51 -0700 From: Dariusz Sosnowski To: Matan Azrad , Viacheslav Ovsiienko CC: Subject: [PATCH v2 2/8] common/mlx5: add hairpin SQ buffer type capabilities Date: Thu, 6 Oct 2022 11:00:59 +0000 Message-ID: <20221006110105.2986966-3-dsosnowski@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221006110105.2986966-1-dsosnowski@nvidia.com> References: <20221006110105.2986966-1-dsosnowski@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [10.126.230.35] X-ClientProxiedBy: rnnvmail202.nvidia.com (10.129.68.7) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN8NAM11FT020:EE_|BL3PR12MB6427:EE_ X-MS-Office365-Filtering-Correlation-Id: 104b228e-b882-40cc-2d24-08daa78a33aa X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 90+ximccAqoVezRcIGtV039Cdfiy3OGZONGbddoVSXshURUvFSPlZAze6I3Nj0FulGrTbY1DLVy5KO2XOqVSaron59vILwKp+tnwwuDlhExm58udO7CBUdYsEU23eMXm/+gF2dY8Z8ksejpYDi/hCf7UQdjHLN3FHHJe+CB1zEFautlvYpLc9dURYwSmPS8COJFAz6bJcIaShc1n6cBiOevCHKNMYSgvW9zh+XTsFKyBoQrQxXY4QTAIv7wI5C/2BWUy8yFG0BpyQTiY1NdF9ixykMWKyye3AT9jYKYIZj2dAcApemTEeZ/AjXZ36s+47N263N/LPICA7mMp2MYfGeg8leUAMi+RHtVQz1qSxoQBHnq6tbCF+uWhwPB2v62Kg1MS0ayMoJN9Cm+HtK5WkQonw3F59NymFMsYNMcLL92ps9uHeuGi3EgcENiclJPY1WzkXp+CfZ/SLsFMiIZRffc+h1nAeeGVNsk8SyyTLsjZEYtqAConMYbcyL3j8pFGCrcvmafnGLHj6iIsYrPC7Xcfsf/V1EagV4QknCDEXY4SIEOWk+dQznQiEbSxGdv2HlP1Jwl7F6MJvg0Hi8OqDtvyt3U7ciZnfJnYhXkwVffQJ+tAvkjD+IrQpOWekg5sOlggDVba9IeIvFYKO39vHeEMGLmt5P86GHRKPZ97kKoN+60Nwqu8K6G2wy7B4pRdb7NZPayi1jB34iCPhJVzNYf1rDTYWbtZAmMpUzJg33M6d8Xr+EzchKqi4RLW3F+fSgbeuJamr5wvV/OouCzXag== X-Forefront-Antispam-Report: CIP:216.228.117.160; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc6edge1.nvidia.com; CAT:NONE; SFS:(13230022)(4636009)(39860400002)(396003)(376002)(136003)(346002)(451199015)(36840700001)(46966006)(40470700004)(16526019)(426003)(336012)(7636003)(47076005)(356005)(1076003)(83380400001)(186003)(86362001)(63350400001)(82740400003)(82310400005)(36860700001)(70206006)(41300700001)(8936002)(5660300002)(110136005)(4326008)(316002)(6636002)(70586007)(7696005)(40480700001)(8676002)(55016003)(6286002)(2906002)(26005)(40460700003)(2616005)(478600001)(63370400001)(36756003); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Oct 2022 11:02:03.8431 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 104b228e-b882-40cc-2d24-08daa78a33aa X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.117.160]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT020.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL3PR12MB6427 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org This patch extends HCA_CAP and SQ Context structs available in PRM. This fields allow checking if NIC supports storing hairpin SQ's WQ buffer in host memory and configuring such memory placement. HCA capabilities are extended with the following fields: - hairpin_sq_wq_in_host_mem - If set, then NIC supports using host memory as a backing storage for hairpin SQ's WQ buffer. - hairpin_sq_wqe_bb_size - Indicates the required size of SQ WQE basic block. SQ Context is extended with hairpin_wq_buffer_type which informs NIC where SQ's WQ buffer will be stored. This field can take the following values: - MLX5_SQC_HAIRPIN_WQ_BUFFER_TYPE_INTERNAL_BUFFER - WQ buffer will be stored in unlocked device memory. - MLX5_SQC_HAIRPIN_WQ_BUFFER_TYPE_HOST_MEMORY - WQ buffer will be stored in host memory. Buffer is provided by PMD. Signed-off-by: Dariusz Sosnowski Acked-by: Viacheslav Ovsiienko --- drivers/common/mlx5/mlx5_devx_cmds.c | 5 +++++ drivers/common/mlx5/mlx5_devx_cmds.h | 3 +++ drivers/common/mlx5/mlx5_prm.h | 15 +++++++++++++-- 3 files changed, 21 insertions(+), 2 deletions(-) diff --git a/drivers/common/mlx5/mlx5_devx_cmds.c b/drivers/common/mlx5/mlx5_devx_cmds.c index fb33023138..a1e8179568 100644 --- a/drivers/common/mlx5/mlx5_devx_cmds.c +++ b/drivers/common/mlx5/mlx5_devx_cmds.c @@ -989,6 +989,10 @@ mlx5_devx_cmd_query_hca_attr(void *ctx, } attr->log_min_stride_wqe_sz = MLX5_GET(cmd_hca_cap_2, hcattr, log_min_stride_wqe_sz); + attr->hairpin_sq_wqe_bb_size = MLX5_GET(cmd_hca_cap_2, hcattr, + hairpin_sq_wqe_bb_size); + attr->hairpin_sq_wq_in_host_mem = MLX5_GET(cmd_hca_cap_2, hcattr, + hairpin_sq_wq_in_host_mem); } if (attr->log_min_stride_wqe_sz == 0) attr->log_min_stride_wqe_sz = MLX5_MPRQ_LOG_MIN_STRIDE_WQE_SIZE; @@ -1706,6 +1710,7 @@ mlx5_devx_cmd_create_sq(void *ctx, MLX5_SET(sqc, sq_ctx, hairpin, sq_attr->hairpin); MLX5_SET(sqc, sq_ctx, non_wire, sq_attr->non_wire); MLX5_SET(sqc, sq_ctx, static_sq_wq, sq_attr->static_sq_wq); + MLX5_SET(sqc, sq_ctx, hairpin_wq_buffer_type, sq_attr->hairpin_wq_buffer_type); MLX5_SET(sqc, sq_ctx, user_index, sq_attr->user_index); MLX5_SET(sqc, sq_ctx, cqn, sq_attr->cqn); MLX5_SET(sqc, sq_ctx, packet_pacing_rate_limit_index, diff --git a/drivers/common/mlx5/mlx5_devx_cmds.h b/drivers/common/mlx5/mlx5_devx_cmds.h index af6053a788..9ac2d75df4 100644 --- a/drivers/common/mlx5/mlx5_devx_cmds.h +++ b/drivers/common/mlx5/mlx5_devx_cmds.h @@ -191,6 +191,8 @@ struct mlx5_hca_attr { uint32_t log_max_hairpin_queues:5; uint32_t log_max_hairpin_wq_data_sz:5; uint32_t log_max_hairpin_num_packets:5; + uint32_t hairpin_sq_wqe_bb_size:4; + uint32_t hairpin_sq_wq_in_host_mem:1; uint32_t vhca_id:16; uint32_t relaxed_ordering_write:1; uint32_t relaxed_ordering_read:1; @@ -407,6 +409,7 @@ struct mlx5_devx_create_sq_attr { uint32_t non_wire:1; uint32_t static_sq_wq:1; uint32_t ts_format:2; + uint32_t hairpin_wq_buffer_type:3; uint32_t user_index:24; uint32_t cqn:24; uint32_t packet_pacing_rate_limit_index:16; diff --git a/drivers/common/mlx5/mlx5_prm.h b/drivers/common/mlx5/mlx5_prm.h index 4346279c81..04d35ca845 100644 --- a/drivers/common/mlx5/mlx5_prm.h +++ b/drivers/common/mlx5/mlx5_prm.h @@ -2020,7 +2020,11 @@ struct mlx5_ifc_cmd_hca_cap_2_bits { u8 reserved_at_d8[0x3]; u8 log_max_conn_track_offload[0x5]; u8 reserved_at_e0[0x20]; /* End of DW7. */ - u8 reserved_at_100[0x700]; + u8 reserved_at_100[0x60]; + u8 reserved_at_160[0x3]; + u8 hairpin_sq_wqe_bb_size[0x5]; + u8 hairpin_sq_wq_in_host_mem[0x1]; + u8 reserved_at_169[0x697]; }; struct mlx5_ifc_esw_cap_bits { @@ -2673,6 +2677,11 @@ enum { MLX5_SQC_STATE_ERR = 0x3, }; +enum { + MLX5_SQC_HAIRPIN_WQ_BUFFER_TYPE_INTERNAL_BUFFER = 0x0, + MLX5_SQC_HAIRPIN_WQ_BUFFER_TYPE_HOST_MEMORY = 0x1, +}; + struct mlx5_ifc_sqc_bits { u8 rlky[0x1]; u8 cd_master[0x1]; @@ -2686,7 +2695,9 @@ struct mlx5_ifc_sqc_bits { u8 hairpin[0x1]; u8 non_wire[0x1]; u8 static_sq_wq[0x1]; - u8 reserved_at_11[0x9]; + u8 reserved_at_11[0x4]; + u8 hairpin_wq_buffer_type[0x3]; + u8 reserved_at_18[0x2]; u8 ts_format[0x02]; u8 reserved_at_1c[0x4]; u8 reserved_at_20[0x8];