From patchwork Sun Oct 29 16:31:39 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gregory Etelson X-Patchwork-Id: 133580 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 267CA43238; Sun, 29 Oct 2023 17:33:20 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 8237540E64; Sun, 29 Oct 2023 17:32:59 +0100 (CET) Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2045.outbound.protection.outlook.com [40.107.244.45]) by mails.dpdk.org (Postfix) with ESMTP id 561B540E2D for ; Sun, 29 Oct 2023 17:32:58 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=od06B5DkrKwa0mUlFsetpFZGSnQYxYweTHEpEeJTIJqmO6NiSgEsZMC4CTRO59k+EsM3YhK210Uxt/6Q4qeK0HsnMqktAOv3hEi7VmLD64iVvpDCO935Q4UNtxhrdgPihO1dmTEJruZULMSTQiRIcW0iHpuU+/Vzpp6kUgKmCbn4SooH4dYK9nE19u51fz9bvbEd7sld7uIK5zj8n1KdsCyOnXKfdU4hdMVbWdV3rK4XLHk8tD+ANF2s/EkSFFJMp+3bKbTjV+EJCS2WBex5ltpJeoK7sw8isv6ACuNKJ6jAJUAR3NwLwDdTcSrbvvnZvBscMojt1bV/ymEy4PLcpA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=/tNcCj/KjH0lRCQy9PeWJjCX54nF6IvlQI83YPpjqIs=; b=SrP8VeR1ZNMqnUSFn1fv/snRB1EhDpUoOKJLwuHFoBCNe5wdm9fHLydKewBRSKNWMUW3blDjay9lDxPzlMa5KT2SE3FNqfnISGoN1KsSjDyT3Rp1SGZrOdhuys1qkVp7EasQX+xJl3oP+TQw0sg74P5DNB/K2htldAXv63GpLObybh9GUKHKnFXtunShND2SRdFaTxHuRjYkj4FZMk+6feG9+cCITs8bejJ9TCmiSgJSvt+keHJyb68HEo2m9roF5M71y31QB5tXirFUcMYArDYc5PICdTPWSsq7q2uCH+5p66Hbxbv+JlF3F+ZdyZ/c4BICJsFae/sYq3tjanVUGg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=/tNcCj/KjH0lRCQy9PeWJjCX54nF6IvlQI83YPpjqIs=; b=Ggc3cEygvn0v41D0Ywy5rR/+AYDSOu4yRW19mx3WLfbsBY6pSWFsO0FnxxQOOtH1R7vyBrambh7ybZnjg8o+kL/ekRiXq81poGUKO+ql1vXit2iIIG6CbIMhVOYGRRoAbjGj/goXb/GUaQ7dDAChuQlNU3mck/4MqumV9sEfxNrIL8jb1ARF2eJTVxLIsaX+wSMBC7m8lB08aWH4T5pZU4XzqHtSprjZvQbKjFcGs2HO94/nfETJj1Bv1p3/+C/GEPcp7LJZoPH/iQoKzomzNbKyWdzP1jwbNivlU3qxG0O+DRBMCOh2JqZWYM8Mm/goTx0igsDKMwvnLUFS1YWbEQ== Received: from MN2PR16CA0028.namprd16.prod.outlook.com (2603:10b6:208:134::41) by IA1PR12MB8190.namprd12.prod.outlook.com (2603:10b6:208:3f2::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6933.25; Sun, 29 Oct 2023 16:32:55 +0000 Received: from BL02EPF0001A0FC.namprd03.prod.outlook.com (2603:10b6:208:134:cafe::20) by MN2PR16CA0028.outlook.office365.com (2603:10b6:208:134::41) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6838.30 via Frontend Transport; Sun, 29 Oct 2023 16:32:55 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by BL02EPF0001A0FC.mail.protection.outlook.com (10.167.242.103) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6933.15 via Frontend Transport; Sun, 29 Oct 2023 16:32:55 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Sun, 29 Oct 2023 09:32:38 -0700 Received: from nvidia.com (10.126.231.35) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Sun, 29 Oct 2023 09:32:36 -0700 From: Gregory Etelson To: CC: , , , "Ori Kam" , Matan Azrad , Viacheslav Ovsiienko , Suanming Mou Subject: [PATCH 07/30] net/mlx5: initialize HWS flow tags registers in shared dev context Date: Sun, 29 Oct 2023 18:31:39 +0200 Message-ID: <20231029163202.216450-7-getelson@nvidia.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20231029163202.216450-1-getelson@nvidia.com> References: <20231029163202.216450-1-getelson@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [10.126.231.35] X-ClientProxiedBy: rnnvmail203.nvidia.com (10.129.68.9) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL02EPF0001A0FC:EE_|IA1PR12MB8190:EE_ X-MS-Office365-Filtering-Correlation-Id: e1db4714-d9cf-4915-187c-08dbd89cb475 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: JlN6O/b1tvcRM4uJbErRKIQZBxIEvO1NCuff1wWGR7d55Z8YMU0WuX1WPGCeBZ0NAZQFIWZ6Crldv97Wqrx2hASKDYT1QE5QIobap3BEY+i9IB13lZNO9ykreL7EP3LNXD8ywqn4wyMbSMdwVIYnEfUbqghTGnnnuCTUbekDKr51GyurUw7jY5ufLulW67EkkLgXCzUkajP/t8voN8kfQfIA0l/uBvVVP4tscNJGBYkC0zDOECoVOh0n+xbrDg6d9KUdTLxCNglgCT2ifyK7ZG3k3MH980bZHxxJeE3ZMjwI+nENgy6txgTrt2cQtpdXIYSrQ514c5mw+ZACULbwOMFWx8qa89x7SNthvYQjy+TX+vr1jfgUI3SrVi7tGtGp5b+LW/Deb3WzQmt9XG+LvUzkFd/Qskixwr9V87YX31q5O4O4lgpk7aADqhGb0tI2xhJSLkeodMiRWd+AAOz77eAsoDPu/ajJGa/IJgP9kvYu+5HaWwjy87wV/+nuk75Eonpz6KJ3I20pSbMCcy26EgmtHwZzxTvCSEr0qV4Ca37dVp774KJOk6Ry/OJNKODxk3rtZqOUNVpCoXoNCIU9kIERHBKc8w6dBGVIIQdz6bSBcmHTMf3teHMbqze7ob+nuy+45pcAhLOeiLFISk9U0JoAqwGAW7nwQysFAcS9jthXx2x0A4S41bU5hOWMxz7oNCh54+E0tywDgYxwI2XGAaktnhZW4BDIqwbQA87AtkOwDptU2ZS/AtR6XTJaHtyfnzz1DKPGRtCNJ4fGfKBxCHf/4bx6Sy0y8Do5Lny7+OA= X-Forefront-Antispam-Report: CIP:216.228.117.161; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc6edge2.nvidia.com; CAT:NONE; SFS:(13230031)(4636009)(136003)(376002)(346002)(396003)(39860400002)(230173577357003)(230922051799003)(230273577357003)(451199024)(186009)(1800799009)(64100799003)(82310400011)(40470700004)(46966006)(36840700001)(55016003)(40480700001)(47076005)(7696005)(478600001)(36756003)(70586007)(336012)(426003)(16526019)(6286002)(26005)(70206006)(356005)(2616005)(36860700001)(83380400001)(82740400003)(1076003)(7636003)(107886003)(54906003)(6916009)(316002)(8936002)(8676002)(4326008)(40460700003)(5660300002)(86362001)(2906002)(41300700001); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Oct 2023 16:32:55.5096 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: e1db4714-d9cf-4915-187c-08dbd89cb475 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.117.161]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BL02EPF0001A0FC.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR12MB8190 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Move HWS flow tags registers initialization to shared dev context. Signed-off-by: Gregory Etelson Acked-by: Ori Kam --- drivers/net/mlx5/linux/mlx5_os.c | 35 ++------------- drivers/net/mlx5/mlx5.c | 75 ++++++++++++++++++++++++++++++++ drivers/net/mlx5/mlx5.h | 6 +++ drivers/net/mlx5/mlx5_flow.h | 3 -- drivers/net/mlx5/mlx5_flow_hw.c | 34 --------------- 5 files changed, 84 insertions(+), 69 deletions(-) diff --git a/drivers/net/mlx5/linux/mlx5_os.c b/drivers/net/mlx5/linux/mlx5_os.c index ed273e14cf..ec067ef52c 100644 --- a/drivers/net/mlx5/linux/mlx5_os.c +++ b/drivers/net/mlx5/linux/mlx5_os.c @@ -1304,38 +1304,12 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev, struct mlx5_hca_attr *hca_attr = &sh->cdev->config.hca_attr; sh->steering_format_version = hca_attr->steering_format_version; -#if defined(HAVE_MLX5DV_DR) && \ - (defined(HAVE_MLX5_DR_CREATE_ACTION_FLOW_METER) || \ - defined(HAVE_MLX5_DR_CREATE_ACTION_ASO)) +#if defined(HAVE_MLX5_DR_CREATE_ACTION_ASO_EXT) if (hca_attr->qos.sup && hca_attr->qos.flow_meter_old && sh->config.dv_flow_en) { - uint8_t reg_c_mask = hca_attr->qos.flow_meter_reg_c_ids; - /* - * Meter needs two REG_C's for color match and pre-sfx - * flow match. Here get the REG_C for color match. - * REG_C_0 and REG_C_1 is reserved for metadata feature. - */ - reg_c_mask &= 0xfc; - if (rte_popcount32(reg_c_mask) < 1) { - priv->mtr_en = 0; - DRV_LOG(WARNING, "No available register for" - " meter."); - } else { - /* - * The meter color register is used by the - * flow-hit feature as well. - * The flow-hit feature must use REG_C_3 - * Prefer REG_C_3 if it is available. - */ - if (reg_c_mask & (1 << (REG_C_3 - REG_C_0))) - sh->registers.aso_reg = REG_C_3; - else - sh->registers.aso_reg = - ffs(reg_c_mask) - 1 + REG_C_0; + if (sh->registers.aso_reg != REG_NON) { priv->mtr_en = 1; priv->mtr_reg_share = hca_attr->qos.flow_meter; - DRV_LOG(DEBUG, "The REG_C meter uses is %d", - sh->registers.aso_reg); } } if (hca_attr->qos.sup && hca_attr->qos.flow_meter_aso_sup) { @@ -1358,7 +1332,7 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev, sh->tunnel_header_0_1 = 1; if (hca_attr->flow.tunnel_header_2_3) sh->tunnel_header_2_3 = 1; -#endif +#endif /* HAVE_MLX5_DR_CREATE_ACTION_ASO_EXT */ #ifdef HAVE_MLX5_DR_CREATE_ACTION_ASO if (hca_attr->flow_hit_aso && sh->registers.aso_reg == REG_C_3) { sh->flow_hit_aso_en = 1; @@ -1617,9 +1591,6 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev, err = ENOTSUP; goto error; } - /* Only HWS requires this information. */ - if (sh->refcnt == 1) - flow_hw_init_tags_set(eth_dev); if (priv->sh->config.dv_esw_en && flow_hw_create_vport_action(eth_dev)) { DRV_LOG(ERR, "port %u failed to create vport action", diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index c13ce2c13c..840c566162 100644 --- a/drivers/net/mlx5/mlx5.c +++ b/drivers/net/mlx5/mlx5.c @@ -1599,6 +1599,80 @@ mlx5_rt_timestamp_config(struct mlx5_dev_ctx_shared *sh, } } +static void +mlx5_init_hws_flow_tags_registers(struct mlx5_dev_ctx_shared *sh) +{ + struct mlx5_dev_registers *reg = &sh->registers; + uint32_t meta_mode = sh->config.dv_xmeta_en; + uint8_t masks = (uint8_t)sh->cdev->config.hca_attr.set_reg_c; + uint8_t unset = 0; + uint32_t i, j; + + /* + * The CAPA is global for common device but only used in net. + * It is shared per eswitch domain. + */ + if (reg->aso_reg != REG_NON) + unset |= 1 << mlx5_regc_index(reg->aso_reg); + unset |= 1 << mlx5_regc_index(REG_C_6); + if (sh->config.dv_esw_en) + unset |= 1 << mlx5_regc_index(REG_C_0); + if (meta_mode == MLX5_XMETA_MODE_META32_HWS) + unset |= 1 << mlx5_regc_index(REG_C_1); + masks &= ~unset; + for (i = 0, j = 0; i < MLX5_FLOW_HW_TAGS_MAX; i++) { + if (!!((1 << i) & masks)) + reg->hw_avl_tags[j++] = mlx5_regc_value(i); + } +} + +static void +mlx5_init_aso_register(struct mlx5_dev_ctx_shared *sh) +{ +#if defined(HAVE_MLX5_DR_CREATE_ACTION_ASO_EXT) + const struct mlx5_hca_attr *hca_attr = &sh->cdev->config.hca_attr; + const struct mlx5_hca_qos_attr *qos = &hca_attr->qos; + uint8_t reg_c_mask = qos->flow_meter_reg_c_ids & 0xfc; + + if (!(qos->sup && qos->flow_meter_old && sh->config.dv_flow_en)) + return; + /* + * Meter needs two REG_C's for color match and pre-sfx + * flow match. Here get the REG_C for color match. + * REG_C_0 and REG_C_1 is reserved for metadata feature. + */ + if (__builtin_popcount(reg_c_mask) > 0) { + /* + * The meter color register is used by the + * flow-hit feature as well. + * The flow-hit feature must use REG_C_3 + * Prefer REG_C_3 if it is available. + */ + if (reg_c_mask & (1 << mlx5_regc_index(REG_C_3))) + sh->registers.aso_reg = REG_C_3; + else + sh->registers.aso_reg = + mlx5_regc_value(ffs(reg_c_mask) - 1); + } +#else + RTE_SET_USED(sh); +#endif +} + +static void +mlx5_init_shared_dev_registers(struct mlx5_dev_ctx_shared *sh) +{ + if (sh->cdev->config.devx) + mlx5_init_aso_register(sh); + if (sh->registers.aso_reg != REG_NON) { + DRV_LOG(DEBUG, "ASO register: REG_C%d", + mlx5_regc_index(sh->registers.aso_reg)); + } else { + DRV_LOG(DEBUG, "ASO register: NONE"); + } + mlx5_init_hws_flow_tags_registers(sh); +} + /** * Allocate shared device context. If there is multiport device the * master and representors will share this context, if there is single @@ -1720,6 +1794,7 @@ mlx5_alloc_shared_dev_ctx(const struct mlx5_dev_spawn_data *spawn, /* Add context to the global device list. */ LIST_INSERT_HEAD(&mlx5_dev_ctx_list, sh, next); rte_spinlock_init(&sh->geneve_tlv_opt_sl); + mlx5_init_shared_dev_registers(sh); exit: pthread_mutex_unlock(&mlx5_dev_ctx_list_mutex); return sh; diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index 99a2ad88ed..a0dcd788b4 100644 --- a/drivers/net/mlx5/mlx5.h +++ b/drivers/net/mlx5/mlx5.h @@ -1380,6 +1380,12 @@ struct mlx5_dev_registers { enum modify_reg hw_avl_tags[MLX5_FLOW_HW_TAGS_MAX]; }; +#if defined(HAVE_MLX5DV_DR) && \ + (defined(HAVE_MLX5_DR_CREATE_ACTION_FLOW_METER) || \ + defined(HAVE_MLX5_DR_CREATE_ACTION_ASO)) +#define HAVE_MLX5_DR_CREATE_ACTION_ASO_EXT +#endif + /* * Shared Infiniband device context for Master/Representors * which belong to same IB device with multiple IB ports. diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h index 250d9eb1fc..aea8b38f39 100644 --- a/drivers/net/mlx5/mlx5_flow.h +++ b/drivers/net/mlx5/mlx5_flow.h @@ -1727,9 +1727,6 @@ flow_hw_get_reg_id_from_ctx(void *dr_ctx, void flow_hw_set_port_info(struct rte_eth_dev *dev); void flow_hw_clear_port_info(struct rte_eth_dev *dev); - -void flow_hw_init_tags_set(struct rte_eth_dev *dev); - int flow_hw_create_vport_action(struct rte_eth_dev *dev); void flow_hw_destroy_vport_action(struct rte_eth_dev *dev); diff --git a/drivers/net/mlx5/mlx5_flow_hw.c b/drivers/net/mlx5/mlx5_flow_hw.c index c48c2eec39..b0ef14c14e 100644 --- a/drivers/net/mlx5/mlx5_flow_hw.c +++ b/drivers/net/mlx5/mlx5_flow_hw.c @@ -8448,40 +8448,6 @@ flow_hw_clear_port_info(struct rte_eth_dev *dev) info->is_wire = 0; } -/* - * Initialize the information of available tag registers and an intersection - * of all the probed devices' REG_C_Xs. - * PS. No port concept in steering part, right now it cannot be per port level. - * - * @param[in] dev - * Pointer to the rte_eth_dev structure. - */ -void flow_hw_init_tags_set(struct rte_eth_dev *dev) -{ - struct mlx5_dev_ctx_shared *sh = MLX5_SH(dev); - struct mlx5_dev_registers *reg = &sh->registers; - uint32_t meta_mode = sh->config.dv_xmeta_en; - uint8_t masks = (uint8_t)sh->cdev->config.hca_attr.set_reg_c; - uint8_t unset = 0; - uint32_t i, j; - - /* - * The CAPA is global for common device but only used in net. - * It is shared per eswitch domain. - */ - unset |= 1 << mlx5_regc_index(reg->aso_reg); - unset |= 1 << mlx5_regc_index(REG_C_6); - if (sh->config.dv_esw_en) - unset |= 1 << mlx5_regc_index(REG_C_0); - if (meta_mode == MLX5_XMETA_MODE_META32_HWS) - unset |= 1 << mlx5_regc_index(REG_C_1); - masks &= ~unset; - for (i = 0, j = 0; i < MLX5_FLOW_HW_TAGS_MAX; i++) { - if (!!((1 << i) & masks)) - reg->hw_avl_tags[j++] = mlx5_regc_value(i); - } -} - static int flow_hw_conntrack_destroy(struct rte_eth_dev *dev __rte_unused, uint32_t idx,