From patchwork Wed May 5 06:40:51 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bing Zhao X-Patchwork-Id: 92803 Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 52270A0524; Wed, 5 May 2021 08:42:05 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 0E208410F8; Wed, 5 May 2021 08:42:05 +0200 (CEST) Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on2082.outbound.protection.outlook.com [40.107.236.82]) by mails.dpdk.org (Postfix) with ESMTP id 40589410F8 for ; Wed, 5 May 2021 08:42:04 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=jhJYpASSNAmVQUJQGqabYkX+TTteZOxM9trwTLY29rA2L3uB12lKBWoe7xIvGlRRrdPM4TMuLpPsJmA0uYPzqJat3WNdxAF5WZ4jQ3gD079CTSM7QuB4HtFlHcv/17wkWpV/TvXLG4nJ7NUIvkTI1NmunCeEve6FHVbfhpBp3ItkmEk0HzpH0vj7G4B177z51sEXSF9CytSLxCotGSJdbORMZBXJZy8C+gMt0yuTe2eQ8P4xmuIScesvwyhC2+Cki2tHEcsQ3diqeTP3srYjkwjT/EiYuBf/iT5RXeXqy2ahQ/LaSmbkiRKL4uJAGISH3sTEVadGLP78l8re3238Fg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=lFtVPxmiGdnzMFnQcOUr84MkOp6rQV+JzmP2o471acI=; b=F8rU0caKWPAu1ap9f6xwLWN1bMh9G9WrAF6CRgtNYM1Wfr+iEfrXZ9XddBN4LXHjzmog4+YHaNZVmHTXoRT2TBduMhtjjqQDHQFQ7v5eoFNG3fPJtAp77vZJzoD0DkTojsvSIt0XdnvOH3lAKWMBm6+3h01CKB/IDCOR9htiaSPv56WGH1EXW9zb1khqJhDpLqhrJPwTJAXO4TbBlDKdz6IFsoprYx0nS3N+Evv3hJdCpsjAwgBPgzGk2W0ZfqyDgdZpWxuPuaTCQvcgIB/3TFVn9UxOOuDvnWWl//DQ7ZVvSAMwWbePKXyiIhzRMnp4yf9vrDVc+GG6XIZDe4rQ+w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=lFtVPxmiGdnzMFnQcOUr84MkOp6rQV+JzmP2o471acI=; b=dj/+LQGrNkrnm8m5uMoutQLOCqrULWu0TvhMGHcVQbh7dGK9kJ/rom1fVyF9NgUvrlZaw23uqF2NbeB6+ARtAyRfo7lqVJ1P1SK6ohbPNpM7kx4w6OfCO2LPMjRYLp0bx6b1y7ze22gMOWkig6qTym1iu2Y71IdyMtDkZfB5Qtci4drECAZZ9Sv8u2ZeEKCOPJXTo1IS8pIOHLnVQGLRUYsn+fJ5R0gsmNIwnHc7FbCjmLQjGAkT4YXlaabXYjTIrRVHaxjYS0lLDHklnjsQrXjp2hG3IIxA5nq0+euJP/jkLQpUsOom28FFVJCmtzvcGbKPNOe66NR/nkULfs5lcQ== Received: from DM3PR12CA0130.namprd12.prod.outlook.com (2603:10b6:0:51::26) by DM5PR12MB1307.namprd12.prod.outlook.com (2603:10b6:3:79::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4087.35; Wed, 5 May 2021 06:42:03 +0000 Received: from DM6NAM11FT016.eop-nam11.prod.protection.outlook.com (2603:10b6:0:51:cafe::f6) by DM3PR12CA0130.outlook.office365.com (2603:10b6:0:51::26) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4108.25 via Frontend Transport; Wed, 5 May 2021 06:42:03 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; dpdk.org; dkim=none (message not signed) header.d=none;dpdk.org; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by DM6NAM11FT016.mail.protection.outlook.com (10.13.173.139) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4087.32 via Frontend Transport; Wed, 5 May 2021 06:42:02 +0000 Received: from nvidia.com (172.20.145.6) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 5 May 2021 06:41:55 +0000 From: Bing Zhao To: , , CC: , , Date: Wed, 5 May 2021 09:40:51 +0300 Message-ID: <20210505064104.30248-5-bingz@nvidia.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210505064104.30248-1-bingz@nvidia.com> References: <20210427153811.11554-1-bingz@nvidia.com> <20210505064104.30248-1-bingz@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [172.20.145.6] X-ClientProxiedBy: HQMAIL105.nvidia.com (172.20.187.12) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: cb66e084-e2ce-451e-e29b-08d90f90e44c X-MS-TrafficTypeDiagnostic: DM5PR12MB1307: X-LD-Processed: 43083d15-7273-40c1-b7db-39efd9ccc17a,ExtAddr X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:4941; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: CTG2VYS+QYnaBIxL2YmpkEx+6aWesgBs2jpmTUf/2+bhRNsK/voZ8uB62Qo7odIBdhAQiAmpw1VSoJoQsGug24i6lcyGN9iLoOw7UdYLfbuSIYyaPr6up8lm8UoCr9ktytmfB1iLI5E9qq/Dd6l7X/Oohsql7jyZeZ92s5fbEdb7AHBYLOJeB5x3fWuD3XdxiGvS9j8oDg1DjRD5MLUjcAhokU+4ax/ijZPAPVj5DD+mR33jITRFjuO7Xr/QQQurZeZbGJ+pQwE9O/lJG5iDk6qAWBuTqV7hWYLAOMtYrZwYmKFnwrI8AwPoTfjVxrEEeMVzc9QmqYXIhcJLKXlFDd65l7jNADnvQTwQkZ+E4NgxkwMdH66ijxf6jBlyWf2F9OYX+FWkfhpKpxly+1uzAl7X0m7cI6pSFDqtyCxbZF/Tsw2lTHQTzxHv1rnMuHL85YNnre3Ho4MeBGAlM+cMEImrj4YcLnLTE/mdYW4YLL5cwMeXroNIMUB2XMvmnIS9OGQQkMeEgHTrVBOrJcaxztI+e4ZTFrTv4YvPBT9qSUG3zXvmMjE6vrBHtRvg4/vhciXi8pioTYRLzOp6HvkkwTc9e0RD3tSdrwXdmfHNgNucjgodYqyCAs0cEhmWOvaLs2Gbre4aRTwPBh8ohONmw6cR0BxOxy5+1q2FhvNBuLI= X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(136003)(396003)(39860400002)(376002)(346002)(36840700001)(46966006)(316002)(7696005)(54906003)(82740400003)(8676002)(86362001)(8936002)(83380400001)(336012)(356005)(478600001)(2906002)(6666004)(110136005)(82310400003)(36756003)(36906005)(26005)(426003)(47076005)(55016002)(4326008)(5660300002)(16526019)(1076003)(70586007)(107886003)(7636003)(70206006)(36860700001)(2616005)(186003)(6286002); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 05 May 2021 06:42:02.8397 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: cb66e084-e2ce-451e-e29b-08d90f90e44c X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT016.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR12MB1307 Subject: [dpdk-dev] [PATCH v3 04/17] net/mlx5: initialization of CT management X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" The definitions of ASO connection tracking objects management structures are added. Considering performance, the bulk allocation of ASO CT objects should be used. The maximal value per bulk and the granularity could be fetched from HCA capabilities 2. Right now, a fixed number of 64 is used for each bulk for a better management purpose. The ASO QP for CT is initialized, the SQ will be used for both modify and query command. Signed-off-by: Bing Zhao --- drivers/net/mlx5/linux/mlx5_os.c | 13 +++++++++ drivers/net/mlx5/mlx5.c | 36 +++++++++++++++++++++++ drivers/net/mlx5/mlx5.h | 50 ++++++++++++++++++++++++++++++++ drivers/net/mlx5/mlx5_flow_aso.c | 50 ++++++++++++++++++++++++++++++++ 4 files changed, 149 insertions(+) diff --git a/drivers/net/mlx5/linux/mlx5_os.c b/drivers/net/mlx5/linux/mlx5_os.c index 479ee7d8d1..5ac787106d 100644 --- a/drivers/net/mlx5/linux/mlx5_os.c +++ b/drivers/net/mlx5/linux/mlx5_os.c @@ -1323,6 +1323,19 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev, DRV_LOG(DEBUG, "Flow Hit ASO is supported."); } #endif /* HAVE_MLX5_DR_CREATE_ACTION_ASO */ +#if defined(HAVE_MLX5_DR_CREATE_ACTION_ASO) && \ + defined(HAVE_MLX5_DR_ACTION_ASO_CT) + if (config->hca_attr.ct_offload && + priv->mtr_color_reg == REG_C_3) { + err = mlx5_flow_aso_ct_mng_init(sh); + if (err) { + err = -err; + goto error; + } + DRV_LOG(DEBUG, "CT ASO is supported."); + sh->ct_aso_en = 1; + } +#endif /* HAVE_MLX5_DR_CREATE_ACTION_ASO && HAVE_MLX5_DR_ACTION_ASO_CT */ #if defined(HAVE_MLX5DV_DR) && defined(HAVE_MLX5_DR_CREATE_ACTION_FLOW_SAMPLE) if (config->hca_attr.log_max_ft_sampler_num > 0 && config->dv_flow_en) { diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index 8cd6f1eaee..86dbe6d573 100644 --- a/drivers/net/mlx5/mlx5.c +++ b/drivers/net/mlx5/mlx5.c @@ -670,6 +670,42 @@ mlx5_age_event_prepare(struct mlx5_dev_ctx_shared *sh) } } +/* + * Initialize the ASO connection tracking structure. + * + * @param[in] sh + * Pointer to mlx5_dev_ctx_shared object. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +int +mlx5_flow_aso_ct_mng_init(struct mlx5_dev_ctx_shared *sh) +{ + int err; + + if (sh->ct_mng) + return 0; + sh->ct_mng = mlx5_malloc(MLX5_MEM_ZERO, sizeof(*sh->ct_mng), + RTE_CACHE_LINE_SIZE, SOCKET_ID_ANY); + if (!sh->ct_mng) { + DRV_LOG(ERR, "ASO CT management allocation failed."); + rte_errno = ENOMEM; + return -rte_errno; + } + err = mlx5_aso_queue_init(sh, ASO_OPC_MOD_CONNECTION_TRACKING); + if (err) { + mlx5_free(sh->ct_mng); + /* rte_errno should be extracted from the failure. */ + rte_errno = EINVAL; + return -rte_errno; + } + rte_spinlock_init(&sh->ct_mng->ct_sl); + rte_rwlock_init(&sh->ct_mng->resize_rwl); + LIST_INIT(&sh->ct_mng->free_cts); + return 0; +} + /** * Initialize the flow resources' indexed mempool. * diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index c62977613a..1a5c78fa3a 100644 --- a/drivers/net/mlx5/mlx5.h +++ b/drivers/net/mlx5/mlx5.h @@ -983,6 +983,52 @@ struct mlx5_bond_info { } ports[MLX5_BOND_MAX_PORTS]; }; +/* Number of connection tracking objects per pool: must be a power of 2. */ +#define MLX5_ASO_CT_ACTIONS_PER_POOL 64 + +/* ASO Conntrack state. */ +enum mlx5_aso_ct_state { + ASO_CONNTRACK_FREE, /* Inactive, in the free list. */ + ASO_CONNTRACK_WAIT, /* WQE sent in the SQ. */ + ASO_CONNTRACK_READY, /* CQE received w/o error. */ + ASO_CONNTRACK_QUERY, /* WQE for query sent. */ + ASO_CONNTRACK_MAX, /* Guard. */ +}; + +/* Generic ASO connection tracking structure. */ +struct mlx5_aso_ct_action { + LIST_ENTRY(mlx5_aso_ct_action) next; /* Pointer to the next ASO CT. */ + void *dr_action_orig; /* General action object for original dir. */ + void *dr_action_rply; /* General action object for reply dir. */ + uint32_t refcnt; /* Action used count in device flows. */ + uint16_t offset; /* Offset of ASO CT in DevX objects bulk. */ + uint16_t peer; /* The only peer port index could also use this CT. */ + enum mlx5_aso_ct_state state; /* ASO CT state. */ + bool is_original; /* The direction of the DR action to be used. */ +}; + +/* ASO connection tracking software pool definition. */ +struct mlx5_aso_ct_pool { + uint16_t index; /* Pool index in pools array. */ + struct mlx5_devx_obj *devx_obj; + /* The first devx object in the bulk, used for freeing (not yet). */ + struct mlx5_aso_ct_action actions[MLX5_ASO_CT_ACTIONS_PER_POOL]; + /* CT action structures bulk. */ +}; + +LIST_HEAD(aso_ct_list, mlx5_aso_ct_action); + +/* Pools management structure for ASO connection tracking pools. */ +struct mlx5_aso_ct_pools_mng { + struct mlx5_aso_ct_pool **pools; + uint16_t n; /* Total number of pools. */ + uint16_t next; /* Number of pools in use, index of next free pool. */ + rte_spinlock_t ct_sl; /* The ASO CT free list lock. */ + rte_rwlock_t resize_rwl; /* The ASO CT pool resize lock. */ + struct aso_ct_list free_cts; /* Free ASO CT objects list. */ + struct mlx5_aso_sq aso_sq; /* ASO queue objects. */ +}; + /* * Shared Infiniband device context for Master/Representors * which belong to same IB device with multiple IB ports. @@ -996,6 +1042,7 @@ struct mlx5_dev_ctx_shared { uint32_t sq_ts_format:2; /* SQ timestamp formats supported. */ uint32_t qp_ts_format:2; /* QP timestamp formats supported. */ uint32_t meter_aso_en:1; /* Flow Meter ASO is supported. */ + uint32_t ct_aso_en:1; /* Connection Tracking ASO is supported. */ uint32_t max_port; /* Maximal IB device port index. */ struct mlx5_bond_info bond; /* Bonding information. */ void *ctx; /* Verbs/DV/DevX context. */ @@ -1058,6 +1105,8 @@ struct mlx5_dev_ctx_shared { rte_spinlock_t geneve_tlv_opt_sl; /* Lock for geneve tlv resource */ struct mlx5_flow_mtr_mng *mtrmng; /* Meter management structure. */ + struct mlx5_aso_ct_pools_mng *ct_mng; + /* Management data for ASO connection tracking. */ struct mlx5_dev_shared_port port[]; /* per device port data array. */ }; @@ -1355,6 +1404,7 @@ bool mlx5_flex_parser_ecpri_exist(struct rte_eth_dev *dev); int mlx5_flex_parser_ecpri_alloc(struct rte_eth_dev *dev); int mlx5_flow_aso_age_mng_init(struct mlx5_dev_ctx_shared *sh); int mlx5_aso_flow_mtrs_mng_init(struct mlx5_dev_ctx_shared *sh); +int mlx5_flow_aso_ct_mng_init(struct mlx5_dev_ctx_shared *sh); /* mlx5_ethdev.c */ diff --git a/drivers/net/mlx5/mlx5_flow_aso.c b/drivers/net/mlx5/mlx5_flow_aso.c index 300987d0e9..c24d865284 100644 --- a/drivers/net/mlx5/mlx5_flow_aso.c +++ b/drivers/net/mlx5/mlx5_flow_aso.c @@ -186,6 +186,43 @@ mlx5_aso_mtr_init_sq(struct mlx5_aso_sq *sq) } } +/* + * Initialize Send Queue used for ASO connection tracking. + * + * @param[in] sq + * ASO SQ to initialize. + */ +static void +mlx5_aso_ct_init_sq(struct mlx5_aso_sq *sq) +{ + volatile struct mlx5_aso_wqe *restrict wqe; + int i; + int size = 1 << sq->log_desc_n; + uint64_t addr; + + /* All the next fields state should stay constant. */ + for (i = 0, wqe = &sq->sq_obj.aso_wqes[0]; i < size; ++i, ++wqe) { + wqe->general_cseg.sq_ds = rte_cpu_to_be_32((sq->sqn << 8) | + (sizeof(*wqe) >> 4)); + /* One unique MR for the query data. */ + wqe->aso_cseg.lkey = rte_cpu_to_be_32(sq->mr.mkey->id); + /* Magic number 64 represents the length of a ASO CT obj. */ + addr = (uint64_t)((uintptr_t)sq->mr.addr + i * 64); + wqe->aso_cseg.va_h = rte_cpu_to_be_32((uint32_t)(addr >> 32)); + wqe->aso_cseg.va_l_r = rte_cpu_to_be_32((uint32_t)addr | 1u); + /* + * The values of operand_masks are different for modify + * and query. + * And data_mask may be different for each modification. In + * query, it could be zero and ignored. + * CQE generation is always needed, in order to decide when + * it is available to create the flow or read the data. + */ + wqe->general_cseg.flags = RTE_BE32(MLX5_COMP_ALWAYS << + MLX5_COMP_MODE_OFFSET); + } +} + /** * Create Send Queue used for ASO access. * @@ -293,6 +330,19 @@ mlx5_aso_queue_init(struct mlx5_dev_ctx_shared *sh, return -1; mlx5_aso_mtr_init_sq(&sh->mtrmng->pools_mng.sq); break; + case ASO_OPC_MOD_CONNECTION_TRACKING: + /* 64B per object for query. */ + if (mlx5_aso_reg_mr(sh, 64 * sq_desc_n, + &sh->ct_mng->aso_sq.mr, 0)) + return -1; + if (mlx5_aso_sq_create(sh->ctx, &sh->ct_mng->aso_sq, 0, + sh->tx_uar, sh->pdn, MLX5_ASO_QUEUE_LOG_DESC, + sh->sq_ts_format)) { + mlx5_aso_dereg_mr(sh, &sh->ct_mng->aso_sq.mr); + return -1; + } + mlx5_aso_ct_init_sq(&sh->ct_mng->aso_sq); + break; default: DRV_LOG(ERR, "Unknown ASO operation mode"); return -1;