From patchwork Tue Jul 6 13:32:51 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Suanming Mou X-Patchwork-Id: 95407 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 53C53A0C47; Tue, 6 Jul 2021 15:36:13 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 5755841474; Tue, 6 Jul 2021 15:33:58 +0200 (CEST) Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2071.outbound.protection.outlook.com [40.107.94.71]) by mails.dpdk.org (Postfix) with ESMTP id 7112441463 for ; Tue, 6 Jul 2021 15:33:55 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=fHF7xjXiRzcz3eusN7KX1cE4CklMPGsErGv+cQv04BXIY6i9oi9nqLD+1F7h6JMAY54mzotkt0hYAVjL7ymEZH2ItJuKDOCukNTs7YfKMmU9qrAAAhEZQYuq0oWCfnVOGd1ShY582sBUKk5w52YbrxxxVU65M+VrsTiFQ7jXbV/9x9lsbmqKc/MORwRV8mfOMDhBksBaJ+2EHNHptmvroMF6TSUbO7Zh5BLzB1PURSgBkgWOxSJMI8u1jHYU7VkuonDNNPWopknq7957FHuj4ejuSjrs6GZgR/CCTWk8+1kO476g2VhSObi+bjG+vEwNRf+xxdrfSyVK0tWCl5Am8Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=wMxQghiCg6BrkEbZKNp5MffPQT44VK3GyRoSs2dfosI=; b=aE/5VW4UVQYptYLd3AFJ3081mP45lYLc0WFKbKm5UEWOmramgwbffh9rG/5GmOG9yPYjO5l8MngDgte4jEYEavfxaxqHyJ8gRHT0MGFNVXeCZpl3qWH9/KelfrGwxz65j4uvElc8Je7Cy6MEzsaj7EhxU1sDlLXS6o1UyyRfE9iIoHAnFIwlRrCotg4IJJS4ouy6Tyh26o9W2QwFNjgDuwiBFdxHESQu5WzHlgtoYIxZopKQWeVWYTfCAcyyjcSuGVmDdBdhG+0GSJcm+uIErCMKfRUTm4ke7F1dxdydoRzwTJNkLKFuVtef2V/WtiDkbJqPn7i5NqlOM1itAeJPfg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=wMxQghiCg6BrkEbZKNp5MffPQT44VK3GyRoSs2dfosI=; b=YR1usS0I2t35XlnNueQstZYZqCdkVi4f1A+W0vJUnzscbyhwvRkuuAa1P13rEC/fRL7H2GOLpiTQn2F0DBsT7FwLtGyE0YqtUkLlJo/H/diVVXNUZ1b9uRn1qlyeXZl2mXvOxx3VFXV1+ssKXmJ02Zo3IHtnjbBEa6TpQXtLi7SgbSFuGxRSlsRWlZCB0otytranzIlOy9Oc5FrO566Xtu08KNJ1fnHZ/wmfrFh9BgQruRLPqaWCKE9gbx3ZICpNJ2yMepllp1HsIx8aQPVPYeJlCKbChvV92llO8vzRKrACB2a/U+Z2rsPZeXTVzjPdBTHvEEwcGguM0hW0DmogyQ== Received: from BN9PR03CA0629.namprd03.prod.outlook.com (2603:10b6:408:106::34) by DM5PR12MB2551.namprd12.prod.outlook.com (2603:10b6:4:b9::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4287.22; Tue, 6 Jul 2021 13:33:53 +0000 Received: from BN8NAM11FT056.eop-nam11.prod.protection.outlook.com (2603:10b6:408:106:cafe::7e) by BN9PR03CA0629.outlook.office365.com (2603:10b6:408:106::34) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4287.23 via Frontend Transport; Tue, 6 Jul 2021 13:33:53 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; dpdk.org; dkim=none (message not signed) header.d=none;dpdk.org; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by BN8NAM11FT056.mail.protection.outlook.com (10.13.177.26) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4287.22 via Frontend Transport; Tue, 6 Jul 2021 13:33:53 +0000 Received: from nvidia.com (172.20.187.6) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Tue, 6 Jul 2021 13:33:51 +0000 From: Suanming Mou To: , CC: , , Date: Tue, 6 Jul 2021 16:32:51 +0300 Message-ID: <20210706133257.3353-21-suanmingm@nvidia.com> X-Mailer: git-send-email 2.18.1 In-Reply-To: <20210706133257.3353-1-suanmingm@nvidia.com> References: <20210527093403.1153127-1-suanmingm@nvidia.com> <20210706133257.3353-1-suanmingm@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [172.20.187.6] X-ClientProxiedBy: HQMAIL101.nvidia.com (172.20.187.10) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: d8d29838-b6ab-4380-5eed-08d94082b293 X-MS-TrafficTypeDiagnostic: DM5PR12MB2551: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:5797; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: J80TVIaDAVrga1QvrLasSvvA8tXhU4gm6V8GyQi3lI6Dy6WFcaU9sXzSkW85cke4Lpgndc9trHKet7EPEEevP4ynGQEkoO0D1DOs85WTtUqG8QdPCR3ehbn+NzH7ngEq2EkBOJFstbeA2Rgo3kyN3g0har4Y2FZ7YOmkoT3qO/oe7XZHpRKMgtKipuy3pt9s0Z2/QE/2Wx2RI7w+G/EkZmGC9Rv8Widotq9jLRG6gmnLVVdwtaKuLeiaSfsfmRkEiVQ0qY8Zt3uYxfk1nO1TZH0EPkN9KlPCavpFRCrCCCqJZptAI42nkOw/i58L0BxW2L2QBIrP/2T9TuJsFHRmlw7XvCC5PK7bmBUiZqXK/nq8nPOkEGrpkrPjBmEFwabInqn2fyVGlBJFs7EWglSzdkzIBAozhRvoGM+KAHYPaKitplBQ7LZ2bsRR2GBk06kAhvBtj6ia6DNNst0CjS5kIT/uuxhbJwkgGdOyTqb3SY6j4+CZUHTkR6ctkWEdumka/XnoP42MegXr3dxdfdH4UeNM3tJ3WjfOTBRrgARaeL7LEaBGfx2y2cQW6X3JT2yC+QUsIMGr2I4MB0HtEfgcB3JPEfUbSxxQQgYBGBnkmLbF2uwAlt4riNJXWRt7seNCvfk/iBi0PbDIFE58s+4si3jD7+a/DfgvstgrxNeNwHg5rmJCqnCOwJ5e5P+TF6qb7x0jHeJaZoyrwWqc1uuYYQ== X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(346002)(136003)(376002)(396003)(39860400002)(46966006)(36840700001)(83380400001)(86362001)(336012)(356005)(7636003)(54906003)(55016002)(4326008)(110136005)(36756003)(70586007)(6636002)(2616005)(426003)(82740400003)(186003)(36860700001)(1076003)(47076005)(2906002)(16526019)(5660300002)(316002)(6286002)(82310400003)(36906005)(8936002)(8676002)(26005)(7696005)(70206006)(478600001); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Jul 2021 13:33:53.3825 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: d8d29838-b6ab-4380-5eed-08d94082b293 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT056.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR12MB2551 Subject: [dpdk-dev] [PATCH v4 20/26] net/mlx5: move modify header allocator to ipool X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Matan Azrad Modify header actions are allocated by mlx5_malloc which has a big overhead of memory and allocation time. One of the action types under the modify header object is SET_TAG, The SET_TAG action is commonly not reused by the flows and each flow has its own value. Hence, the mlx5_malloc becomes a bottleneck in flow insertion rate in the common cases of SET_TAG. Use ipool allocator for SET_TAG action. Ipool allocator has less overhead of memory and insertion rate and has better synchronization mechanism in multithread cases. Different ipool is created for each optional size of modify header handler. Signed-off-by: Matan Azrad Acked-by: Suanming Mou --- drivers/net/mlx5/mlx5.c | 4 ++ drivers/net/mlx5/mlx5.h | 14 ++++++ drivers/net/mlx5/mlx5_flow.h | 14 +----- drivers/net/mlx5/mlx5_flow_dv.c | 79 ++++++++++++++++++++++++++++----- 4 files changed, 86 insertions(+), 25 deletions(-) diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index 2d4330198b..499d8449ac 100644 --- a/drivers/net/mlx5/mlx5.c +++ b/drivers/net/mlx5/mlx5.c @@ -799,6 +799,7 @@ mlx5_flow_ipool_create(struct mlx5_dev_ctx_shared *sh, } } + /** * Release the flow resources' indexed mempool. * @@ -812,6 +813,9 @@ mlx5_flow_ipool_destroy(struct mlx5_dev_ctx_shared *sh) for (i = 0; i < MLX5_IPOOL_MAX; ++i) mlx5_ipool_destroy(sh->ipool[i]); + for (i = 0; i < MLX5_MAX_MODIFY_NUM; ++i) + if (sh->mdh_ipools[i]) + mlx5_ipool_destroy(sh->mdh_ipools[i]); } /* diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index 56ac018c7e..d88abef12c 100644 --- a/drivers/net/mlx5/mlx5.h +++ b/drivers/net/mlx5/mlx5.h @@ -36,6 +36,19 @@ #define MLX5_SH(dev) (((struct mlx5_priv *)(dev)->data->dev_private)->sh) +/* + * Number of modification commands. + * The maximal actions amount in FW is some constant, and it is 16 in the + * latest releases. In some old releases, it will be limited to 8. + * Since there is no interface to query the capacity, the maximal value should + * be used to allow PMD to create the flow. The validation will be done in the + * lower driver layer or FW. A failure will be returned if exceeds the maximal + * supported actions number on the root table. + * On non-root tables, there is no limitation, but 32 is enough right now. + */ +#define MLX5_MAX_MODIFY_NUM 32 +#define MLX5_ROOT_TBL_MODIFY_NUM 16 + enum mlx5_ipool_index { #if defined(HAVE_IBV_FLOW_DV_SUPPORT) || !defined(HAVE_INFINIBAND_VERBS_H) MLX5_IPOOL_DECAP_ENCAP = 0, /* Pool for encap/decap resource. */ @@ -1123,6 +1136,7 @@ struct mlx5_dev_ctx_shared { struct mlx5_flow_counter_mng cmng; /* Counters management structure. */ void *default_miss_action; /* Default miss action. */ struct mlx5_indexed_pool *ipool[MLX5_IPOOL_MAX]; + struct mlx5_indexed_pool *mdh_ipools[MLX5_MAX_MODIFY_NUM]; /* Memory Pool for mlx5 flow resources. */ struct mlx5_l3t_tbl *cnt_id_tbl; /* Shared counter lookup table. */ /* Shared interrupt handler section. */ diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h index 98e90fec0d..da6367c212 100644 --- a/drivers/net/mlx5/mlx5_flow.h +++ b/drivers/net/mlx5/mlx5_flow.h @@ -504,23 +504,11 @@ struct mlx5_flow_dv_tag_resource { uint32_t tag_id; /**< Tag ID. */ }; -/* - * Number of modification commands. - * The maximal actions amount in FW is some constant, and it is 16 in the - * latest releases. In some old releases, it will be limited to 8. - * Since there is no interface to query the capacity, the maximal value should - * be used to allow PMD to create the flow. The validation will be done in the - * lower driver layer or FW. A failure will be returned if exceeds the maximal - * supported actions number on the root table. - * On non-root tables, there is no limitation, but 32 is enough right now. - */ -#define MLX5_MAX_MODIFY_NUM 32 -#define MLX5_ROOT_TBL_MODIFY_NUM 16 - /* Modify resource structure */ struct mlx5_flow_dv_modify_hdr_resource { struct mlx5_list_entry entry; void *action; /**< Modify header action object. */ + uint32_t idx; /* Key area for hash list matching: */ uint8_t ft_type; /**< Flow table type, Rx or Tx. */ uint8_t actions_num; /**< Number of modification actions. */ diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c index 9116f8b9d4..9666c11861 100644 --- a/drivers/net/mlx5/mlx5_flow_dv.c +++ b/drivers/net/mlx5/mlx5_flow_dv.c @@ -5338,6 +5338,45 @@ flow_dv_modify_match_cb(void *tool_ctx __rte_unused, memcmp(&ref->ft_type, &resource->ft_type, key_len); } +static struct mlx5_indexed_pool * +flow_dv_modify_ipool_get(struct mlx5_dev_ctx_shared *sh, uint8_t index) +{ + struct mlx5_indexed_pool *ipool = __atomic_load_n + (&sh->mdh_ipools[index], __ATOMIC_SEQ_CST); + + if (!ipool) { + struct mlx5_indexed_pool *expected = NULL; + struct mlx5_indexed_pool_config cfg = + (struct mlx5_indexed_pool_config) { + .size = sizeof(struct mlx5_flow_dv_modify_hdr_resource) + + (index + 1) * + sizeof(struct mlx5_modification_cmd), + .trunk_size = 64, + .grow_trunk = 3, + .grow_shift = 2, + .need_lock = 1, + .release_mem_en = 1, + .malloc = mlx5_malloc, + .free = mlx5_free, + .type = "mlx5_modify_action_resource", + }; + + cfg.size = RTE_ALIGN(cfg.size, sizeof(ipool)); + ipool = mlx5_ipool_create(&cfg); + if (!ipool) + return NULL; + if (!__atomic_compare_exchange_n(&sh->mdh_ipools[index], + &expected, ipool, false, + __ATOMIC_SEQ_CST, + __ATOMIC_SEQ_CST)) { + mlx5_ipool_destroy(ipool); + ipool = __atomic_load_n(&sh->mdh_ipools[index], + __ATOMIC_SEQ_CST); + } + } + return ipool; +} + struct mlx5_list_entry * flow_dv_modify_create_cb(void *tool_ctx, void *cb_ctx) { @@ -5346,12 +5385,20 @@ flow_dv_modify_create_cb(void *tool_ctx, void *cb_ctx) struct mlx5dv_dr_domain *ns; struct mlx5_flow_dv_modify_hdr_resource *entry; struct mlx5_flow_dv_modify_hdr_resource *ref = ctx->data; + struct mlx5_indexed_pool *ipool = flow_dv_modify_ipool_get(sh, + ref->actions_num - 1); int ret; uint32_t data_len = ref->actions_num * sizeof(ref->actions[0]); uint32_t key_len = sizeof(*ref) - offsetof(typeof(*ref), ft_type); + uint32_t idx; - entry = mlx5_malloc(MLX5_MEM_ZERO, sizeof(*entry) + data_len, 0, - SOCKET_ID_ANY); + if (unlikely(!ipool)) { + rte_flow_error_set(ctx->error, ENOMEM, + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, + NULL, "cannot allocate modify ipool"); + return NULL; + } + entry = mlx5_ipool_zmalloc(ipool, &idx); if (!entry) { rte_flow_error_set(ctx->error, ENOMEM, RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL, @@ -5371,25 +5418,29 @@ flow_dv_modify_create_cb(void *tool_ctx, void *cb_ctx) (sh->ctx, ns, entry, data_len, &entry->action); if (ret) { - mlx5_free(entry); + mlx5_ipool_free(sh->mdh_ipools[ref->actions_num - 1], idx); rte_flow_error_set(ctx->error, ENOMEM, RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL, "cannot create modification action"); return NULL; } + entry->idx = idx; return &entry->entry; } struct mlx5_list_entry * -flow_dv_modify_clone_cb(void *tool_ctx __rte_unused, - struct mlx5_list_entry *oentry, void *cb_ctx) +flow_dv_modify_clone_cb(void *tool_ctx, struct mlx5_list_entry *oentry, + void *cb_ctx) { + struct mlx5_dev_ctx_shared *sh = tool_ctx; struct mlx5_flow_cb_ctx *ctx = cb_ctx; struct mlx5_flow_dv_modify_hdr_resource *entry; struct mlx5_flow_dv_modify_hdr_resource *ref = ctx->data; uint32_t data_len = ref->actions_num * sizeof(ref->actions[0]); + uint32_t idx; - entry = mlx5_malloc(0, sizeof(*entry) + data_len, 0, SOCKET_ID_ANY); + entry = mlx5_ipool_malloc(sh->mdh_ipools[ref->actions_num - 1], + &idx); if (!entry) { rte_flow_error_set(ctx->error, ENOMEM, RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL, @@ -5397,14 +5448,18 @@ flow_dv_modify_clone_cb(void *tool_ctx __rte_unused, return NULL; } memcpy(entry, oentry, sizeof(*entry) + data_len); + entry->idx = idx; return &entry->entry; } void -flow_dv_modify_clone_free_cb(void *tool_ctx __rte_unused, - struct mlx5_list_entry *entry) +flow_dv_modify_clone_free_cb(void *tool_ctx, struct mlx5_list_entry *entry) { - mlx5_free(entry); + struct mlx5_dev_ctx_shared *sh = tool_ctx; + struct mlx5_flow_dv_modify_hdr_resource *res = + container_of(entry, typeof(*res), entry); + + mlx5_ipool_free(sh->mdh_ipools[res->actions_num - 1], res->idx); } /** @@ -13736,14 +13791,14 @@ flow_dv_jump_tbl_resource_release(struct rte_eth_dev *dev, } void -flow_dv_modify_remove_cb(void *tool_ctx __rte_unused, - struct mlx5_list_entry *entry) +flow_dv_modify_remove_cb(void *tool_ctx, struct mlx5_list_entry *entry) { struct mlx5_flow_dv_modify_hdr_resource *res = container_of(entry, typeof(*res), entry); + struct mlx5_dev_ctx_shared *sh = tool_ctx; claim_zero(mlx5_flow_os_destroy_flow_action(res->action)); - mlx5_free(entry); + mlx5_ipool_free(sh->mdh_ipools[res->actions_num - 1], res->idx); } /**