From patchwork Tue Jun 1 06:42:34 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shun Hao X-Patchwork-Id: 93686 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 4376AA0524; Tue, 1 Jun 2021 08:43:09 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 107F140041; Tue, 1 Jun 2021 08:43:09 +0200 (CEST) Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2046.outbound.protection.outlook.com [40.107.244.46]) by mails.dpdk.org (Postfix) with ESMTP id 3327D40040 for ; Tue, 1 Jun 2021 08:43:07 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=PzGCChaN9XUfpeBeD3hAm6QpGmN/AMxoQvyuIZdnJuG1HYbD43YTJGP6CesZZxI/hmIQwt4CcDZIcpogGtM4YO21ZZh0wik1r08cH/mecVmL3WbqZbVXQ/emfd7Ge2xR+Y+D0mTYGel6zDJpKmdw/aVNrv97fi7ARIipYiJJw/NkJ+JwTAOx88/ZEPB+jppxbEwbtAy1RNEV0V0eChONzwEOdybXyd4yNovu6r68qmOKILcyyCN5Qf5OPuuF5nuJWsh7kPeYDQdbX3QiangOVzMTzycw1qBsAz7IcHe5Yp/WS3zQAdwbfBgV2/Gf5nNYu7h/Qw7Er4/MaaE7ckvyXA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=O7yu3xQK9xbQtYCARmdw9qrDafa6iYuGLhqMzl5kv/U=; b=L3VOzkLF13WmF92msilwxYyDl9j4tqk8OLaONuq9o6lRNBvP/7RfxmwoMYLSUUrnh8NZgsmruMNdTxU7jMJl8Q9rO0Jsgre9rBpD7vLwTLygGr5dB+c8L9oxuL7etCJMd3K6xlVOfE+0Zt3j0wNUvLznm743swZnEWuMeKJwTentuyLLnV+zwVIVJuOHc5Sdfms7XDlbXJ+971Sq2dWaEj3u+IZPSNxhs8nUfOwCr9RQOxbwA8fRS+yl8pJ9W755dAW9gGR1qDjsz3VptG9ksZrF5vVfNRITqnGeS5eNrwQYMVapS/4gYnBOcFxI/HXZ2L30Pzpo9wus2FHGZQoy8A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=O7yu3xQK9xbQtYCARmdw9qrDafa6iYuGLhqMzl5kv/U=; b=SgKYYPlxUQuHDd1bBOOVserg/jMOTJsHKvHj083I+fl7jMGjb7JddKvq0yb2kc/jdd6R8UQTz5kGPiwAgy4yArEA/yu/tfVccxBMNEkdxMddGJJNgP1Annp+fydqfa2m51irIA1ZDaUPiEVmltucZhHrtIQW0vHBQDnG5Phbz8TzEbmPNyIhgDuDJVhj7p1izXzoOH2ODNAGqKSBhM2HicUFj8wV/Jt+rDhsUnWp3jYKo21Vfw/4JeBG2QSpGZo8L7QyJDuk4G7WT80fJtm7Y/KSZCSM4vzEwQHmeUt5Vpp1LyPVih4OuCRXLQyjHxbBd5s5glJTQ+6Ia5hqOG8Geg== Received: from DM6PR14CA0070.namprd14.prod.outlook.com (2603:10b6:5:18f::47) by SN1PR12MB2399.namprd12.prod.outlook.com (2603:10b6:802:2b::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4173.29; Tue, 1 Jun 2021 06:43:05 +0000 Received: from DM6NAM11FT020.eop-nam11.prod.protection.outlook.com (2603:10b6:5:18f:cafe::84) by DM6PR14CA0070.outlook.office365.com (2603:10b6:5:18f::47) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4173.20 via Frontend Transport; Tue, 1 Jun 2021 06:43:05 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; dpdk.org; dkim=none (message not signed) header.d=none;dpdk.org; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by DM6NAM11FT020.mail.protection.outlook.com (10.13.172.224) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4150.30 via Frontend Transport; Tue, 1 Jun 2021 06:43:04 +0000 Received: from nvidia.com (172.20.145.6) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Tue, 1 Jun 2021 06:42:59 +0000 From: Shun Hao To: , , , , Shahaf Shuler CC: , Date: Tue, 1 Jun 2021 09:42:34 +0300 Message-ID: <20210601064234.30113-1-shunh@nvidia.com> X-Mailer: git-send-email 2.20.0 MIME-Version: 1.0 X-Originating-IP: [172.20.145.6] X-ClientProxiedBy: HQMAIL111.nvidia.com (172.20.187.18) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: d78d513a-9186-4225-b7c3-08d924c8821a X-MS-TrafficTypeDiagnostic: SN1PR12MB2399: X-LD-Processed: 43083d15-7273-40c1-b7db-39efd9ccc17a,ExtAddr X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:4714; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: lxJteSFLgFu2HjqcOk2y7T8kcf4q66bjNLneWs5CuzftmICwofEt9dhwKwcbXfNGG44g2AL75XQ+UWnHxhgAnBAHV9Vh0pAckv7Ie759cuJux3iO8KB2+Hms8LFG0Qsz80NlPpNnqoa9dIKeKRUe/FnZl0OBvo3VSq3DJRQPKMSs8Qq/cv5XBBhyIU2+gsuqCeKX0Ll7ib/RLpSHBYspOQZanLRrtzLMrxKvT91v2N7uxduX/Okrugn0aMa/VU/VFaDqDRWrUI+Bg/owlrxwDnDcWPg8xbQOEQmAQMMZoAlgYcJSQBEgZ0N6TNhRNCJhB69GWurN4QTiQOFtEQLUwXygGHWdyJ6zMrU6+/WWd5MbgdXcQ1Fa/ehPw12t5YVzXhudRGYgq+UiM8YZP1VPVt9s2ngctZRZDNgWIldL5M8WqYHGw+hZN8/OXm54RC6KPgTbKhZhcWgwHH04Of3PifBLWhW0anr5kzkTVeQRRrV+06pWb+cfeWVb8KKFJD/te0cVD+1OwhFgpT4b75vpqMFTKiu0H+zKLw0etuC6MwVQWePMCdUfBHI2NHSC8VFWqCVc8178PVDfxQxf1U2XDvv/qZkraMMiJZp4nvflE6h4Gf5e5Q5XOC436+KtqH4NqMR17qcbt+okZULE5s8z/Q== X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(136003)(396003)(39860400002)(376002)(346002)(46966006)(36840700001)(8676002)(7636003)(7696005)(4326008)(36756003)(86362001)(8936002)(2906002)(82740400003)(2616005)(6666004)(336012)(356005)(426003)(6286002)(26005)(36906005)(82310400003)(55016002)(6636002)(1076003)(70206006)(70586007)(16526019)(186003)(107886003)(110136005)(478600001)(5660300002)(54906003)(83380400001)(47076005)(36860700001)(316002); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Jun 2021 06:43:04.3364 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: d78d513a-9186-4225-b7c3-08d924c8821a X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT020.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN1PR12MB2399 Subject: [dpdk-dev] [RFC] net/mlx5: add meter hierarchy support X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Currently, using multiple meters in one flow is not supported. Users must give every meter a different group, and use multiple flows to jump from one group to another, so as to apply multiple meters on the traffic path. It's not good for performance cause the jumps between every two meters are waste, so there's requirement that to have a better way for meter hierarchy. To support it, now meter action is supported in meter policy, which means after one meter, in its policy, the packet can go to next meter directly, without any extra jump action. This makes all meters in the hierarchy be chained together directly. Users just need to use the first meter in their flow, and all meters in the hierarchy will be working. There's no API changes, just add support of meter action in policy creation and related validation. Validation during policy creation: 1. The target meter must exist and must be termination policy meter. 2. No more fate actions accepted besides the meter action. 3. All meters in the hierarchy should not be a loop. Policy creation: 1. If the final fate action of the meter chain is not RSS, just create the policy table and policy flow, which has the destination of the direct next meter and its policy table. 2. If the final fate action of the meter chain is RSS, hold on for the policy table creation, until the first RTE flow uses this meter. This is just similar as the creation and usage of a normal RSS policy meter. Limitation: 1. The max meter number in one hierarchy is 8. 2. Currently only meters with termination policy are allowed in meter hierarchy. Signed-off-by: Shun Hao --- doc/guides/nics/mlx5.rst | 18 +++++++++++++++++ drivers/net/mlx5/mlx5.h | 5 +++++ drivers/net/mlx5/mlx5_flow.h | 1 + drivers/net/mlx5/mlx5_flow_dv.c | 35 +++++++++++++++++++++++++++++++++ 4 files changed, 59 insertions(+) diff --git a/doc/guides/nics/mlx5.rst b/doc/guides/nics/mlx5.rst index 83299646dd..7296860dd1 100644 --- a/doc/guides/nics/mlx5.rst +++ b/doc/guides/nics/mlx5.rst @@ -1909,3 +1909,21 @@ all flows with assistance of external tools. .. code-block:: console mlx_steering_dump.py -f -flowptr + +How to use meter hierarchy +----------------- + +This section demonstrates how to use meter hierarchy. A meter M and a meter N +can be chained and used together in one flow. + +1. Use the later meter N to create a new policy X: + + add port meter policy 0 X g_actions meter mtr_id N / end ... + +2. Use the new policy X to create the former meter M: + + create port meter 0 M 1 X yes 0xffff 1 0 + +3. Apply meter M in flow: + + flow create 0 ingress transfer pattern eth / end actions meter mtr_id M / end diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index 32b2817bf2..96741b9948 100644 --- a/drivers/net/mlx5/mlx5.h +++ b/drivers/net/mlx5/mlx5.h @@ -682,6 +682,8 @@ struct mlx5_meter_policy_action_container { /* Flow meter policy parameter structure. */ struct mlx5_flow_meter_policy { + uint32_t next_mtr_id; + /* The next meter id if in meter hierarchy. */ uint32_t is_rss:1; /* Is RSS policy table. */ uint32_t ingress:1; @@ -692,6 +694,8 @@ struct mlx5_flow_meter_policy { /* Rule applies to transfer domain. */ uint32_t is_queue:1; /* Is queue action in policy table. */ + uint32_t is_meter:1; + /* Is meter action in policy table. */ rte_spinlock_t sl; uint32_t ref_cnt; /* Use count. */ @@ -710,6 +714,7 @@ struct mlx5_flow_meter_policy { #define MLX5_MTR_SUB_POLICY_NUM_SHIFT 3 #define MLX5_MTR_SUB_POLICY_NUM_MASK 0x7 #define MLX5_MTRS_DEFAULT_RULE_PRIORITY 0xFFFF +#define MLX5_MTR_CHAIN_MAX_NUM 8 /* Flow meter default policy parameter structure. * Policy index 0 is reserved by default policy table. diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h index 2f2aa962f9..3190d9c32d 100644 --- a/drivers/net/mlx5/mlx5_flow.h +++ b/drivers/net/mlx5/mlx5_flow.h @@ -454,6 +454,7 @@ enum mlx5_flow_fate_type { MLX5_FLOW_FATE_DROP, MLX5_FLOW_FATE_DEFAULT_MISS, MLX5_FLOW_FATE_SHARED_RSS, + MLX5_FLOW_FATE_MTR, MLX5_FLOW_FATE_MAX, }; diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c index c50649a107..7a20e4ee3b 100644 --- a/drivers/net/mlx5/mlx5_flow_dv.c +++ b/drivers/net/mlx5/mlx5_flow_dv.c @@ -14977,6 +14977,9 @@ __flow_dv_create_domain_policy_acts(struct rte_eth_dev *dev, action_flags |= MLX5_FLOW_ACTION_JUMP; break; } + case RTE_FLOW_ACTION_TYPE_METER: + /* Get next meter's action. */ + break; default: return -rte_mtr_error_set(error, ENOTSUP, RTE_MTR_ERROR_TYPE_METER_POLICY, @@ -15578,6 +15581,9 @@ __flow_dv_create_policy_acts_rules(struct rte_eth_dev *dev, hrxq->action; acts[i].actions_n++; break; + case MLX5_FLOW_FATE_MTR: + /* Set dv_actions with next meter's action. */ + break; default: /*Queue action do nothing*/ break; @@ -16420,11 +16426,14 @@ flow_dv_validate_mtr_policy_acts(struct rte_eth_dev *dev, struct mlx5_dev_config *dev_conf = &priv->config; const struct rte_flow_action *act; uint64_t action_flags = 0; + uint64_t fate_flags = 0; int actions_n; int i, ret; struct rte_flow_error flow_err; uint8_t domain_color[RTE_COLORS] = {0}; uint8_t def_domain = MLX5_MTR_ALL_DOMAIN_BIT; + struct mlx5_flow_meter_info *fm; + const struct rte_flow_action_meter *mtr; if (!priv->config.dv_esw_en) def_domain &= ~MLX5_MTR_DOMAIN_TRANSFER_BIT; @@ -16602,12 +16611,38 @@ flow_dv_validate_mtr_policy_acts(struct rte_eth_dev *dev, ++actions_n; action_flags |= MLX5_FLOW_ACTION_JUMP; break; + case RTE_FLOW_ACTION_TYPE_METER: + mtr = act->conf; + fm = mlx5_flow_meter_find(priv, mtr->mtr_id, + NULL); + if (!fm) + return -rte_mtr_error_set(error, + EINVAL, + RTE_MTR_ERROR_TYPE_MTR_ID, NULL, + "Meter not found"); + if (fm->def_policy) + return -rte_mtr_error_set(error, + EINVAL, + RTE_MTR_ERROR_TYPE_MTR_ID, NULL, + "Default policy meter " + "not supported."); + ++actions_n; + action_flags |= + MLX5_FLOW_ACTION_METER_WITH_TERMINATED_POLICY; + break; default: return -rte_mtr_error_set(error, ENOTSUP, RTE_MTR_ERROR_TYPE_METER_POLICY, NULL, "Doesn't support optional action"); } + fate_flags = action_flags & + ~(MLX5_FLOW_ACTION_MARK | MLX5_FLOW_ACTION_SET_TAG); + if (fate_flags && (fate_flags & (fate_flags - 1))) + return -rte_mtr_error_set(error, EINVAL, + RTE_MTR_ERROR_TYPE_METER_POLICY, + NULL, "Multiple fate actions " + "not supported."); } /* Yellow is not supported, just skip. */ if (i == RTE_COLOR_YELLOW)