From patchwork Wed May 5 12:23:19 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bing Zhao X-Patchwork-Id: 92923 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id B3E5DA0524; Wed, 5 May 2021 14:24:45 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 298F841104; Wed, 5 May 2021 14:24:09 +0200 (CEST) Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2047.outbound.protection.outlook.com [40.107.93.47]) by mails.dpdk.org (Postfix) with ESMTP id 6F4FF41145 for ; Wed, 5 May 2021 14:24:07 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=lE7LUvjpMI4Til29L52kbSj6hq4RSAVM1Mib6ZpC+LgpeixyZUD6zuqWWgKCwHbPOFSYPPnPxQ2VjLlDFaCEZjk9p7NJyYiP6yTB/axm6Rv6FOnmZslNxUNL+GY546WyPM7uDMS3LVZp8uH7J/GgeSOVrL4puIlbyP+5UOvRQCEG59hXN0gind+zGDwxT1XT9LQOGEjMcwd4EGZeJRQN+7ZCNSUj7gIDnR3M9Zt8MAVMed1bCiSND+7MJPMEOHGSBG0YEpL8gcbNceHraj99P62vb/obF6r4lFCdx1x3mozJh8jn0OTGDmxMmIwi88voaaK9FICPcpqfsAcCn+l0IQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=zz83slvzD6KOlilMy2PZ98U9nx50EpVlVh2tNZZqVNY=; b=fmArNY8aC+MHFAqdJo913CdFhlCQLAew/HwdtN/uUWU51T3YufMFvKPiIoNl8gJay9ho5jhyOEytZT8p+DrfuOepjfLBjGH6s7IiM8vJn+Z2q0mdx73XdFoucL2n7ANBoQvEBABE/0CgoxO8/rFAn8yMQohfG8mBQSnuMhv7lm1U7ZwFrvsJAlJHe5AqSFLwi0BexS7z4anE1lUPUHfJDxipfQWmTeDP8mPaD0Nnc7eZcatA3hepUxrO01O3FPoaWpOOt2RTCTJ1ym0Lnu7Rp6zd9rHoukDC73CrFEkJ73cV4iKlJLXqB903D0/gU4SGw3XBCYpHAzg3K/AvknZrPg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=zz83slvzD6KOlilMy2PZ98U9nx50EpVlVh2tNZZqVNY=; b=iZmGs0uxRVqWxlMD1ApRs8DwESC9qppPyeypETYLodU/e5yuzcTVcWE4TUU+rJSDLU9l6rEW8Oo43R8EBk20CRorjalf2kT/4kubjKBEsp8JwgV56T1SbNQ5RfALwBX7S9TxQcvgQemPWrsvou4DxNzspKrVyITgQkPhFzCCiOcFlcVUW0doYymFPeC7Z3dwCZUfExXbdvOwmUziEZGaAkaA7rQFxtBnkIqheZeHjSrBVLzD+YRkbxkBcVG9IJfkefXHioAQQX3/1/7Qf95+z7YgMrj84cBdIdiREco2pi3vs6GHFF3O2x3ACvoAjANtv9iqyHJIWq0Sj41hrzdFWw== Received: from DS7PR05CA0012.namprd05.prod.outlook.com (2603:10b6:5:3b9::17) by BN8PR12MB2994.namprd12.prod.outlook.com (2603:10b6:408:42::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4087.43; Wed, 5 May 2021 12:24:06 +0000 Received: from DM6NAM11FT059.eop-nam11.prod.protection.outlook.com (2603:10b6:5:3b9:cafe::52) by DS7PR05CA0012.outlook.office365.com (2603:10b6:5:3b9::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4129.9 via Frontend Transport; Wed, 5 May 2021 12:24:06 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; dpdk.org; dkim=none (message not signed) header.d=none;dpdk.org; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by DM6NAM11FT059.mail.protection.outlook.com (10.13.172.92) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4108.25 via Frontend Transport; Wed, 5 May 2021 12:24:05 +0000 Received: from nvidia.com (172.20.145.6) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 5 May 2021 12:24:01 +0000 From: Bing Zhao To: , , CC: , , Date: Wed, 5 May 2021 15:23:19 +0300 Message-ID: <20210505122328.51129-9-bingz@nvidia.com> X-Mailer: git-send-email 2.26.3 In-Reply-To: <20210505122328.51129-1-bingz@nvidia.com> References: <20210427153811.11554-1-bingz@nvidia.com> <20210505122328.51129-1-bingz@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [172.20.145.6] X-ClientProxiedBy: HQMAIL101.nvidia.com (172.20.187.10) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: ce26412d-5edf-4dac-9c18-08d90fc0acef X-MS-TrafficTypeDiagnostic: BN8PR12MB2994: X-LD-Processed: 43083d15-7273-40c1-b7db-39efd9ccc17a,ExtAddr X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:1169; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: ZjIwvZsC3XUwkY5F9G5c+0xGPwQ9hISfzE38jIAan7aTAV0NdTLncpbKplJ/iwabfHssYjhB7TZYqhnYPQ+mPHdECrv1UebNZQ+Lv/mDbbsktfwxvx9OASOccmMfRfdj74YmboLNlXPdrSD4H92GZTpXmidgJKQGWdCL+ZYMl5jQVGLlT/4OuxQ7oA/dlN67hL9luE6uAQwul31LDPH5ofLThsPXaw/eOBmVqlCIFNsnslgmyyL11qPizmQ/PKXlP8uPYxcZApRZWJrzUtl+MTr2qCAH+YWqz3o53eXBVwXTIFzXSpUIWKQFErjUUOUyL+vwyf1pWQ6ckIg0GXGctq0k+T0k8zZ2U3G1vYUOpdtXyfzw1FtlkFUkZC4Q4ZqsAK1gXy7zYC8cQqf8TbGReMEdvG0EKAHnxO9QC7tHIrNj6L3aLu3lZG/lunfkpH7LKNMN6vCWC7Ef0JEvzRiigNuKr9cREC2+/Dv+8iRhwncQdTFieh6lUOs5auvYNaysDktH7IAzoAqhFZTDUVklOIYNp30nt75xCZqN0syBKRRsm4kWEjWDc5lAEf0vPilcXwzCqRxejsr4AEDtrcIJxJ1Dx0USnilvWF4oyIxaqO5WkI5uHx8B5eC981Sy3pm2vx2kFNskY3S9JjdK86G2QVZGVF+nV7Wn2DqsP9moRkg= X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(136003)(376002)(39860400002)(396003)(346002)(36840700001)(46966006)(6286002)(316002)(8936002)(5660300002)(8676002)(70206006)(110136005)(54906003)(47076005)(478600001)(36906005)(4326008)(7696005)(83380400001)(36860700001)(107886003)(55016002)(26005)(186003)(16526019)(82740400003)(426003)(1076003)(6666004)(336012)(2906002)(7636003)(2616005)(356005)(86362001)(70586007)(36756003)(82310400003); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 05 May 2021 12:24:05.7628 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: ce26412d-5edf-4dac-9c18-08d90fc0acef X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT059.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN8PR12MB2994 Subject: [dpdk-dev] [PATCH v7 08/17] net/mlx5: close CT management structure X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" When freeing the IB shared context during stopping a device, the ASO connection tracking management structure should also be cleaned up. All the DR actions created should be destroyed. The structures need to be freed and ASO CT QP should be released. In the meanwhile, the allocated and registered memory region for query should also be deregistered and then freed. Signed-off-by: Bing Zhao Acked-by: Viacheslav Ovsiienko --- drivers/net/mlx5/mlx5.c | 56 ++++++++++++++++++++++++++++++++ drivers/net/mlx5/mlx5_flow_aso.c | 4 +++ 2 files changed, 60 insertions(+) diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index 7e83d09fec..b610f29a66 100644 --- a/drivers/net/mlx5/mlx5.c +++ b/drivers/net/mlx5/mlx5.c @@ -708,6 +708,60 @@ mlx5_flow_aso_ct_mng_init(struct mlx5_dev_ctx_shared *sh) return 0; } +/* + * Close and release all the resources of the + * ASO connection tracking management structure. + * + * @param[in] sh + * Pointer to mlx5_dev_ctx_shared object to free. + */ +static void +mlx5_flow_aso_ct_mng_close(struct mlx5_dev_ctx_shared *sh) +{ + struct mlx5_aso_ct_pools_mng *mng = sh->ct_mng; + struct mlx5_aso_ct_pool *ct_pool; + struct mlx5_aso_ct_action *ct; + uint32_t idx; + uint32_t val; + uint32_t cnt; + int i; + + mlx5_aso_queue_uninit(sh, ASO_OPC_MOD_CONNECTION_TRACKING); + idx = mng->next; + while (idx--) { + cnt = 0; + ct_pool = mng->pools[idx]; + for (i = 0; i < MLX5_ASO_CT_ACTIONS_PER_POOL; i++) { + ct = &ct_pool->actions[i]; + val = __atomic_fetch_sub(&ct->refcnt, 1, + __ATOMIC_RELAXED); + MLX5_ASSERT(val == 1); + if (val > 1) + cnt++; +#ifdef HAVE_MLX5_DR_ACTION_ASO_CT + if (ct->dr_action_orig) + claim_zero(mlx5_glue->destroy_flow_action + (ct->dr_action_orig)); + if (ct->dr_action_rply) + claim_zero(mlx5_glue->destroy_flow_action + (ct->dr_action_rply)); +#endif + } + claim_zero(mlx5_devx_cmd_destroy(ct_pool->devx_obj)); + if (cnt) { + DRV_LOG(DEBUG, "%u ASO CT objects are being used in the pool %u", + cnt, i); + } + mlx5_free(ct_pool); + /* in case of failure. */ + mng->next--; + } + mlx5_free(mng->pools); + mlx5_free(mng); + /* Management structure must be cleared to 0s during allocation. */ + sh->ct_mng = NULL; +} + /** * Initialize the flow resources' indexed mempool. * @@ -1510,6 +1564,8 @@ mlx5_dev_close(struct rte_eth_dev *dev) if (priv->mreg_cp_tbl) mlx5_hlist_destroy(priv->mreg_cp_tbl); mlx5_mprq_free_mp(dev); + if (priv->sh->ct_mng) + mlx5_flow_aso_ct_mng_close(priv->sh); mlx5_os_free_shared_dr(priv); if (priv->rss_conf.rss_key != NULL) mlx5_free(priv->rss_conf.rss_key); diff --git a/drivers/net/mlx5/mlx5_flow_aso.c b/drivers/net/mlx5/mlx5_flow_aso.c index fbf6e5ef38..37cb43147a 100644 --- a/drivers/net/mlx5/mlx5_flow_aso.c +++ b/drivers/net/mlx5/mlx5_flow_aso.c @@ -372,6 +372,10 @@ mlx5_aso_queue_uninit(struct mlx5_dev_ctx_shared *sh, case ASO_OPC_MOD_POLICER: sq = &sh->mtrmng->pools_mng.sq; break; + case ASO_OPC_MOD_CONNECTION_TRACKING: + mlx5_aso_dereg_mr(sh, &sh->ct_mng->aso_sq.mr); + sq = &sh->ct_mng->aso_sq; + break; default: DRV_LOG(ERR, "Unknown ASO operation mode"); return;