From patchwork Fri Jul 2 09:14:43 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shun Hao X-Patchwork-Id: 95208 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 472BCA0A0C; Fri, 2 Jul 2021 11:15:14 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 2EB0241368; Fri, 2 Jul 2021 11:15:14 +0200 (CEST) Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12on2050.outbound.protection.outlook.com [40.107.237.50]) by mails.dpdk.org (Postfix) with ESMTP id 565634014E for ; Fri, 2 Jul 2021 11:15:13 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Ugq7mOk4OrmvuanSdVbOZdcm3uaC0vJz82G/vhwlb2U/vFkud7czAZDtfb/O04ZOHd5Yax5VGKrM2ciJzd+E0AxkTQbuXr9p4UNTwXk3FMFNtFXpLjuKGVQ0m7jrD9frBP4mpl1CFkXzfHn2goNOYLjrnQwUVIt2AJfEdUyqRi8xpdyq5yfqdEKfSc+bU/igyWzrAQ5luWVR05y3T3uWV7m/oiyhN1qjhNeTEu0ToNxAGM64adrXZ0dzYSG692jRi7Q8PHkDm9NEdm4sKDcgRYPQ1ILl0+inUnQp7PmICTzUeVU2uj4BBDEgTH2C6JEx1tzYPk5tg6ROBq6QInDjcQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=0nwto0kKj3JA6q6Xmuu8eSVLgXKIh26HXta5iyfsZyA=; b=ESICtzJcHqR+DSNYPv6goJeOU4N5hEBTsPGZe3Z5tnbECzYCImGNytJzPJcZb6dueN35qjTqHxPE+voMzbOa/Jc0GZ49ySS7UG5NPTaDsfLz8hYEzfgioO/+JNQGMTHc/tR3SY9KXDhiu+u2Zn4t6A7wznkG7nf96RtH4lsD/2XJ4JCIPKHORhPybzSB4UrDqybBcdwKtNvuKb9yI5IWP0sJZ9Ox3QZEneQpOiSJ78Y5EzeVHRjaU/qZhA+ZF7XKiWUhGgdZYGi+NdVjRVGYwK7BGZiQMPGgr2+jsGcumw9xs5krQ2yvXLlj4wO7vyPJyAw3/UudOuEQH7BMYjWpMQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=monjalon.net smtp.mailfrom=nvidia.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=0nwto0kKj3JA6q6Xmuu8eSVLgXKIh26HXta5iyfsZyA=; b=T+Ub+hegNhf9pLh52aDU515fDYZR9TX8Hl1pxgssf0dzKh0r9U4MtzA0KpL+3EGCIDcoifgr3wZSPNYoIcEg8MfAH5bcH7/G21wQWp1bvDa/2bbA6yXFNFHLjR7B45YNntUHNgafOxadmDJOtcp5CUILPfJ7dXkynVO53wO4UaPQ0VgNvNUwSuRppSTudghEYpcRbgi8UZIxzGZ6EIMJ6I77F/DJEo+7Ul+7Z+t1bMC25VEGa7YZJlBo0QPkT8eYRxvaTakZTrru8tAOFj6SNoR7bYzAq0J3seGENJGK7ig6RMSsxJNs9fhq4gkzInReqXw+FM3R9vOmNdCu0JE0gA== Received: from DM5PR21CA0004.namprd21.prod.outlook.com (2603:10b6:3:ac::14) by MN2PR12MB2958.namprd12.prod.outlook.com (2603:10b6:208:aa::29) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4264.19; Fri, 2 Jul 2021 09:15:11 +0000 Received: from DM6NAM11FT034.eop-nam11.prod.protection.outlook.com (2603:10b6:3:ac:cafe::5f) by DM5PR21CA0004.outlook.office365.com (2603:10b6:3:ac::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4308.9 via Frontend Transport; Fri, 2 Jul 2021 09:15:11 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; monjalon.net; dkim=none (message not signed) header.d=none;monjalon.net; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by DM6NAM11FT034.mail.protection.outlook.com (10.13.173.47) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4287.22 via Frontend Transport; Fri, 2 Jul 2021 09:15:11 +0000 Received: from nvidia.com (172.20.187.6) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Fri, 2 Jul 2021 09:15:08 +0000 From: Shun Hao To: , , , CC: , , , Date: Fri, 2 Jul 2021 12:14:43 +0300 Message-ID: <20210702091446.24635-2-shunh@nvidia.com> X-Mailer: git-send-email 2.20.0 In-Reply-To: <20210702091446.24635-1-shunh@nvidia.com> References: <20210702091446.24635-1-shunh@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [172.20.187.6] X-ClientProxiedBy: HQMAIL111.nvidia.com (172.20.187.18) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 493b4073-b897-4a1c-260f-08d93d39e51f X-MS-TrafficTypeDiagnostic: MN2PR12MB2958: X-LD-Processed: 43083d15-7273-40c1-b7db-39efd9ccc17a,ExtAddr X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:33; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 2ZPq+TO3pVKdODTwAQ6Ez/quAPWq6GD1zXRwcj3Eet/rA+faBrYxYyxCUNn8cO7n0TXZSBOTkVHuc+AXssOwBJonHGTmO1ZsuAQZBz8KKdHJT2/AFPFuQCbNo7Eh6oSWy7fvdQJYfBEVBoUN8kj9N7RCPqrvnbis4+ultjdRG9ZfUDk14FCTOVhoZffzz0NTUb8rYSabtkegKs5K0bSt1L0zjBoWFNeUMkhH4KDmribnxADmA47pXMKiaAqIreEoAVEczHIS3bXYdWs9J6lum4kLQs0XpGcibxjLFhA9Ita3nwQrur2oBDSe3vKZXPOUXjlTrSd0c/vLzoDl2Xm3zcX/+WGWXpphyeOw6JN9aoRbzQdgOhWBUV6q5DndBUYNRPY2ADHdro7flf1ktWxfUkq5C37M82SSUGdPKKwjoKlnA7DxCyFhRMUdfPhbRgfO6zeBB9jhTmfjkvoDJwV7JaupETY/nmXeTXFTkVRAvDjD8kxPPgwAk0RYn0MJrKoBTOEqe6kGp9C4b1zYpwIxPPw3EQPrf2mqkikX46l9KXMiC8r8fKvkSsJCGSD5rQPAuBOI0JSHyYfuFhTnw3jgK6uNOvLh5v1L92OXHUnwVBQ6X68aWNDnFrz52SU4F7hvlBLIDBvZSbWGmc03CuroFtRCNAuNg/g7WbtOwNCaskUS5GMp855qdIG5uHsUS9SuHnCb0Wul7nV4NjQTzZj8TA== X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(396003)(136003)(346002)(376002)(39860400002)(46966006)(36840700001)(36906005)(316002)(6636002)(336012)(8936002)(54906003)(110136005)(426003)(70206006)(82740400003)(36756003)(107886003)(478600001)(6286002)(2906002)(26005)(4326008)(70586007)(86362001)(83380400001)(6666004)(47076005)(356005)(16526019)(7636003)(1076003)(55016002)(186003)(82310400003)(5660300002)(8676002)(7696005)(36860700001)(2616005); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Jul 2021 09:15:11.4101 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 493b4073-b897-4a1c-260f-08d93d39e51f X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT034.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB2958 Subject: [dpdk-dev] [PATCH v1 1/4] net/mlx5: use meter profile lookup table X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Currently a list is used to save all meter profile ids, which is not efficient when looking up profile from huge mount of profiles. This changes to use a l3 table instead to save meter profile ids, so as to improve the lookup performance. Signed-off-by: Shun Hao Acked-by: Matan Azrad --- drivers/net/mlx5/linux/mlx5_os.c | 6 +++- drivers/net/mlx5/mlx5.h | 2 +- drivers/net/mlx5/mlx5_flow_meter.c | 53 ++++++++++++++++++++---------- drivers/net/mlx5/windows/mlx5_os.c | 6 +++- 4 files changed, 47 insertions(+), 20 deletions(-) diff --git a/drivers/net/mlx5/linux/mlx5_os.c b/drivers/net/mlx5/linux/mlx5_os.c index 92b3009786..b3f9e392ab 100644 --- a/drivers/net/mlx5/linux/mlx5_os.c +++ b/drivers/net/mlx5/linux/mlx5_os.c @@ -1570,7 +1570,9 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev, priv->ctrl_flows = 0; rte_spinlock_init(&priv->flow_list_lock); TAILQ_INIT(&priv->flow_meters); - TAILQ_INIT(&priv->flow_meter_profiles); + priv->mtr_profile_tbl = mlx5_l3t_create(MLX5_L3T_TYPE_PTR); + if (!priv->mtr_profile_tbl) + goto error; /* Hint libmlx5 to use PMD allocator for data plane resources */ mlx5_glue->dv_set_context_attr(sh->ctx, MLX5DV_CTX_ATTR_BUF_ALLOCATORS, @@ -1713,6 +1715,8 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev, mlx5_vlan_vmwa_exit(priv->vmwa_context); if (eth_dev && priv->drop_queue.hrxq) mlx5_drop_action_destroy(eth_dev); + if (priv->mtr_profile_tbl) + mlx5_l3t_destroy(priv->mtr_profile_tbl); if (own_domain_id) claim_zero(rte_eth_switch_domain_free(priv->domain_id)); mlx5_cache_list_destroy(&priv->hrxqs); diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index 1b2dc8f815..0226327bc3 100644 --- a/drivers/net/mlx5/mlx5.h +++ b/drivers/net/mlx5/mlx5.h @@ -1373,8 +1373,8 @@ struct mlx5_priv { /* Hash table of Rx metadata register copy table. */ uint8_t mtr_sfx_reg; /* Meter prefix-suffix flow match REG_C. */ uint8_t mtr_color_reg; /* Meter color match REG_C. */ - struct mlx5_mtr_profiles flow_meter_profiles; /* MTR profile list. */ struct mlx5_legacy_flow_meters flow_meters; /* MTR list. */ + struct mlx5_l3t_tbl *mtr_profile_tbl; /* Meter index lookup table. */ struct mlx5_l3t_tbl *mtr_idx_tbl; /* Meter index lookup table. */ uint8_t skip_default_rss_reta; /* Skip configuration of default reta. */ uint8_t fdb_def_rule; /* Whether fdb jump to table 1 is configured. */ diff --git a/drivers/net/mlx5/mlx5_flow_meter.c b/drivers/net/mlx5/mlx5_flow_meter.c index d7ce5cd2f6..87c5782956 100644 --- a/drivers/net/mlx5/mlx5_flow_meter.c +++ b/drivers/net/mlx5/mlx5_flow_meter.c @@ -91,13 +91,20 @@ mlx5_flow_meter_action_create(struct mlx5_priv *priv, static struct mlx5_flow_meter_profile * mlx5_flow_meter_profile_find(struct mlx5_priv *priv, uint32_t meter_profile_id) { - struct mlx5_mtr_profiles *fmps = &priv->flow_meter_profiles; struct mlx5_flow_meter_profile *fmp; + union mlx5_l3t_data data; + int32_t ret; - TAILQ_FOREACH(fmp, fmps, next) - if (meter_profile_id == fmp->id) - return fmp; - return NULL; + if (mlx5_l3t_get_entry(priv->mtr_profile_tbl, + meter_profile_id, &data) || !data.ptr) + return NULL; + fmp = data.ptr; + /* Remove reference taken by the mlx5_l3t_get_entry. */ + ret = mlx5_l3t_clear_entry(priv->mtr_profile_tbl, + meter_profile_id); + if (!ret || ret == -1) + return NULL; + return fmp; } /** @@ -399,8 +406,8 @@ mlx5_flow_meter_profile_add(struct rte_eth_dev *dev, struct rte_mtr_error *error) { struct mlx5_priv *priv = dev->data->dev_private; - struct mlx5_mtr_profiles *fmps = &priv->flow_meter_profiles; struct mlx5_flow_meter_profile *fmp; + union mlx5_l3t_data data; int ret; if (!priv->mtr_en) @@ -427,8 +434,13 @@ mlx5_flow_meter_profile_add(struct rte_eth_dev *dev, ret = mlx5_flow_meter_param_fill(fmp, priv, error); if (ret) goto error; - /* Add to list. */ - TAILQ_INSERT_TAIL(fmps, fmp, next); + data.ptr = fmp; + ret = mlx5_l3t_set_entry(priv->mtr_profile_tbl, + meter_profile_id, &data); + if (ret) + return -rte_mtr_error_set(error, ENOTSUP, + RTE_MTR_ERROR_TYPE_UNSPECIFIED, + NULL, "Meter profile insert fail."); return 0; error: mlx5_free(fmp); @@ -472,8 +484,10 @@ mlx5_flow_meter_profile_delete(struct rte_eth_dev *dev, return -rte_mtr_error_set(error, EBUSY, RTE_MTR_ERROR_TYPE_METER_PROFILE_ID, NULL, "Meter profile is in use."); - /* Remove from list. */ - TAILQ_REMOVE(&priv->flow_meter_profiles, fmp, next); + if (mlx5_l3t_clear_entry(priv->mtr_profile_tbl, meter_profile_id)) + return -rte_mtr_error_set(error, EBUSY, + RTE_MTR_ERROR_TYPE_METER_PROFILE_ID, + NULL, "Meter profile remove fail."); mlx5_free(fmp); return 0; } @@ -1859,7 +1873,6 @@ mlx5_flow_meter_flush(struct rte_eth_dev *dev, struct rte_mtr_error *error) { struct mlx5_priv *priv = dev->data->dev_private; struct mlx5_legacy_flow_meters *fms = &priv->flow_meters; - struct mlx5_mtr_profiles *fmps = &priv->flow_meter_profiles; struct mlx5_flow_meter_profile *fmp; struct mlx5_legacy_flow_meter *legacy_fm; struct mlx5_flow_meter_info *fm; @@ -1921,12 +1934,18 @@ mlx5_flow_meter_flush(struct rte_eth_dev *dev, struct rte_mtr_error *error) mlx5_l3t_destroy(priv->sh->mtrmng->policy_idx_tbl); priv->sh->mtrmng->policy_idx_tbl = NULL; } - TAILQ_FOREACH_SAFE(fmp, fmps, next, tmp) { - /* Check unused. */ - MLX5_ASSERT(!fmp->ref_cnt); - /* Remove from list. */ - TAILQ_REMOVE(&priv->flow_meter_profiles, fmp, next); - mlx5_free(fmp); + if (priv->mtr_profile_tbl) { + MLX5_L3T_FOREACH(priv->mtr_profile_tbl, i, entry) { + fmp = entry; + if (mlx5_flow_meter_profile_delete(dev, fmp->id, + error)) + return -rte_mtr_error_set(error, EINVAL, + RTE_MTR_ERROR_TYPE_METER_POLICY_ID, + NULL, "Fail to destroy " + "meter profile."); + } + mlx5_l3t_destroy(priv->mtr_profile_tbl); + priv->mtr_profile_tbl = NULL; } /* Delete default policy table. */ mlx5_flow_destroy_def_policy(dev); diff --git a/drivers/net/mlx5/windows/mlx5_os.c b/drivers/net/mlx5/windows/mlx5_os.c index 3fe3f55f49..e30b682822 100644 --- a/drivers/net/mlx5/windows/mlx5_os.c +++ b/drivers/net/mlx5/windows/mlx5_os.c @@ -566,7 +566,9 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev, priv->flows = 0; priv->ctrl_flows = 0; TAILQ_INIT(&priv->flow_meters); - TAILQ_INIT(&priv->flow_meter_profiles); + priv->mtr_profile_tbl = mlx5_l3t_create(MLX5_L3T_TYPE_PTR); + if (!priv->mtr_profile_tbl) + goto error; /* Bring Ethernet device up. */ DRV_LOG(DEBUG, "port %u forcing Ethernet interface up.", eth_dev->data->port_id); @@ -644,6 +646,8 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev, return eth_dev; error: if (priv) { + if (priv->mtr_profile_tbl) + mlx5_l3t_destroy(priv->mtr_profile_tbl); if (own_domain_id) claim_zero(rte_eth_switch_domain_free(priv->domain_id)); mlx5_free(priv);