From patchwork Thu Oct 6 11:01:03 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dariusz Sosnowski X-Patchwork-Id: 117454 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 064CFA00C2; Thu, 6 Oct 2022 13:02:47 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 1DB4242C1B; Thu, 6 Oct 2022 13:02:22 +0200 (CEST) Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2054.outbound.protection.outlook.com [40.107.244.54]) by mails.dpdk.org (Postfix) with ESMTP id 1AD5442C17 for ; Thu, 6 Oct 2022 13:02:20 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=COghkzNDyNZNuFvGEtSpNJuSTAftK93j1GJHyIEPblOGchnEHx5uOTs8/UfVyOONcVLbz4ikt3Z7N30RRSZjDZknZyyqdLwFng1qTtQ4a9DRRUNgEu884sfx8ZgFEzNgDzSLwK1YIX/CdoKyPxTxdOWagl6Sck8CSoLbKPlrzKDsaiOZbFc0083MMH7qgSjfEy7iCvRnEFqwNz7qTEWt49NuBEughKDXIGKwpwF2fjU7VPZ5GrggUbouCKtvWMHUNpo224IqpTVIhag+8SidYtY7ws1onqxZQmMwMSm3Cp9wV6Rnh6BwsQQ7DLIXwDDHMifd0w7nN7vql8dWD5Co4A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=+u4rBjgUF8FJiGunBf5Y+6C7BtvT3WRbs+eXXqOhLd8=; b=aj5y9t4zQ9REqbkYtG/KSm7CCpzNGdCrnObWweoBWRIgwS7m9BOrZyTYKOg4CRnyEg3gh3hQyrYlFyXlpztJQ7i2SJ7OSrB7G4M0GVTNR1JRxIyc61MYI/+Mmay6ZC4qQkl8FWCaqmfqbSPILJb20yPUW9MGJ1ceEwncArYIe+nxj4uZcPfQZW/lGFNZxHThyePfqQrDzGVfTv7a26XNCH2fdY7Ae2X6lgOTBbavkfUVIM08/JQ3p7bo43lDalV6SpWDx6ZDJA+kHXwstLTdWBtTEcxpFmnlsg+X/WZhc5o067NFTw6mGIe1OYWdp54UaDihEWBb5d10fqhseLFOww== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=+u4rBjgUF8FJiGunBf5Y+6C7BtvT3WRbs+eXXqOhLd8=; b=X826KAxuRHJOosXWKF81uljceDDvzTUvcjLLPTdFVfSI2aC48tpFr7uplyabfwb/QLZjx2OmtgEM59pyZYNmwDEnkMpj5Lp1GsyZhWoGiGe+mv6Y12eMl+YUmO++Fcc5SXSe0OMUqRdO7mmMkNEBheOWy26BVkmAacU3HJ0FmE2B6A+3UOoxPi/AZMmwVDF45325Q8pNDAcOJHZ13wRUeg0LVHKa9VUO0Kl+6O0F3FnMd+xMxbgylCEJfAaMU99Ll9MKuh3MfydqAr9rZElQkIdL885uQiQ7/1DWerw3AWGHWpvOUl8rSe9o4YqETv4iqFgfGtCSmbSpFdcwU6mQ2w== Received: from BN0PR04CA0193.namprd04.prod.outlook.com (2603:10b6:408:e9::18) by SN7PR12MB6689.namprd12.prod.outlook.com (2603:10b6:806:273::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5676.20; Thu, 6 Oct 2022 11:02:18 +0000 Received: from BN8NAM11FT020.eop-nam11.prod.protection.outlook.com (2603:10b6:408:e9:cafe::31) by BN0PR04CA0193.outlook.office365.com (2603:10b6:408:e9::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5676.23 via Frontend Transport; Thu, 6 Oct 2022 11:02:18 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by BN8NAM11FT020.mail.protection.outlook.com (10.13.176.223) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5709.10 via Frontend Transport; Thu, 6 Oct 2022 11:02:17 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.26; Thu, 6 Oct 2022 04:02:04 -0700 Received: from nvidia.com (10.126.230.35) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.29; Thu, 6 Oct 2022 04:02:02 -0700 From: Dariusz Sosnowski To: Matan Azrad , Viacheslav Ovsiienko CC: Subject: [PATCH v2 6/8] doc: add notes for hairpin to mlx5 documentation Date: Thu, 6 Oct 2022 11:01:03 +0000 Message-ID: <20221006110105.2986966-7-dsosnowski@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221006110105.2986966-1-dsosnowski@nvidia.com> References: <20221006110105.2986966-1-dsosnowski@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [10.126.230.35] X-ClientProxiedBy: rnnvmail202.nvidia.com (10.129.68.7) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN8NAM11FT020:EE_|SN7PR12MB6689:EE_ X-MS-Office365-Filtering-Correlation-Id: a924439b-a36e-455b-a0d1-08daa78a3c10 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: /+LUd9RefP+E5jOmjHgXVHwbhyFuWV1+PeqVdxFyQse9kYh1qcj7vQrOypYU6SMbAwELnQ11AxOnR2dUjezz9MEicy9yvGYwJZVbXLoeBzernq+2o/3y7wyFR7ZhGdx0fmPb6O8bYRY+YStRnzQZpYomm4zeipDZf6u1/UDetWBKAgoahqh3q3DnNRg4B4IrwsuKGfflVPkc7LkT4MVklLUlNq9fxWu7ZVuvsepuNw1c+r3VltuWFQPAnb7rX8FoXiw/oa/UrhkmX46nGl82KrgEXC9ueUdLvRi9yN/vEtcmQhQuVG9QUuHqT4A1Ymjasx8EdtTyQahHY5vBLee2BtERGOCoooo3TrnRKVOWUkDkw7UwHAo4e5ISmwjzO8TvcmMypOijtgDxbD4dsqizNPOpKK1aZC2yM0iXSHLvTzbqHi7VRLXJA/GnOPOVswGk/e1Q8jUDaKdPap6CXKxfA+4NizVzfa7jlfVY3+xJHj4Yc2TXcIUgfiB5A9ViAPpc38pi70EzYa4zav2PLNbzB0QxZ0m8Q+K344WkDIeQQWV16ka5lWjV4anRo2xILdpfpp6NAYKzEKhE8CBPq1OnnZ0fi4rXdFYl39WTgyPyxlzlZLSKmi8kpiN/he27VVxF2+/ny1qvLM4xAKtwAE4kZTcTuBs2431Ts7uVIYQiR/lai2UiDhkk0KjOC4CJGDxkqUSfVsbrDdjULCuBH7EzXz9/ADOEQ7O58N4ES8cljv/N2wr0cgLakZKlUWEgtk3oh5j1sHI4NLsgBib0QN3j5A== X-Forefront-Antispam-Report: CIP:216.228.117.160; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc6edge1.nvidia.com; CAT:NONE; SFS:(13230022)(4636009)(346002)(136003)(39860400002)(376002)(396003)(451199015)(36840700001)(46966006)(40470700004)(16526019)(186003)(40460700003)(2616005)(86362001)(1076003)(7636003)(6636002)(7696005)(110136005)(5660300002)(41300700001)(316002)(356005)(6286002)(478600001)(36756003)(70206006)(4326008)(70586007)(26005)(8676002)(82310400005)(55016003)(40480700001)(36860700001)(8936002)(47076005)(82740400003)(426003)(83380400001)(336012)(2906002); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Oct 2022 11:02:17.9363 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: a924439b-a36e-455b-a0d1-08daa78a3c10 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.117.160]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT020.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN7PR12MB6689 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org This patch extends mlx5 PMD documentation with more information regarding hairpin support. The following is added to mlx5 PMD documentation: - description of the default behavior of hairpin queues, - description of use_locked_device_memory effect on hairpin queue configuration, - description of use_rte_memory effect on hairpin queue configuration, - DPDK and OFED requirements for new memory options for hairpin. Signed-off-by: Dariusz Sosnowski Acked-by: Viacheslav Ovsiienko --- doc/guides/nics/mlx5.rst | 37 +++++++++++++++++++++++++++++++++++++ 1 file changed, 37 insertions(+) diff --git a/doc/guides/nics/mlx5.rst b/doc/guides/nics/mlx5.rst index 3d4ee31f8d..997cb19ba2 100644 --- a/doc/guides/nics/mlx5.rst +++ b/doc/guides/nics/mlx5.rst @@ -1517,6 +1517,43 @@ behavior as librte_net_mlx4:: > port config all rss all > port start all +Notes for hairpin +----------------- + +NVIDIA Connect-X and BlueField devices support specifying memory +placement for hairpin Rx and Tx queues. This feature requires OFED 5.8. + +By default, data buffers and packet descriptors for hairpin queues are placed +in device memory which is shared with other resources (e.g. flow rules). + +Starting with DPDK 22.11 and OFED 5.8 applications are allowed to: + +#. Place data buffers and Rx packet descriptors in dedicated device memory. + Application can request that configuration through ``use_locked_device_memory`` + configuration option. + + Placing data buffers and Rx packet descriptors in dedicated device memory + can decrease latency on hairpinned traffic, since traffic processing + for the hairpin queue will not be memory starved. + + However, reserving device memory for hairpin Rx queues may decrease throughput + under heavy load, since less resources will be available on device. + + This option is supported only for Rx hairpin queues. + +#. Place Tx packet descriptors in host memory. + Application can request that configuration through ``use_rte_memory`` + configuration option. + + Placing Tx packet descritors in host memory can increase traffic throughput. + This results in more resources available on the device for other purposes, + which reduces memory contention on device. + Side effect of this option is visible increase in latency, since each packet + incurs additional PCI transactions. + + This option is supported only for Tx hairpin queues. + + Usage example -------------