From patchwork Mon Apr 10 10:21:49 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sivaprasad Tummala X-Patchwork-Id: 125875 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 549DD42910; Mon, 10 Apr 2023 12:22:46 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id E5D2340DDC; Mon, 10 Apr 2023 12:22:45 +0200 (CEST) Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2086.outbound.protection.outlook.com [40.107.93.86]) by mails.dpdk.org (Postfix) with ESMTP id 1C0A640A81 for ; Mon, 10 Apr 2023 12:22:44 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=XWIOe5R/miiSE9GpkfFMJZtHHclKwnl3PicFb3FSVYC3A/zv26X1jEK8JNboXfMZHLm6NFEvUGHHRHpeMkCJhEuWwMDwxPpcslBSsOClog9CsOoVM9aI8i5vQNcUKtXbjrb66Bw3b82cuKHI0W1LTPnAncT587Eajxib+V7bEBxL9RXOT6CRsFN4n9NyNOmZmlLDnZJW6h/0f9gPe0pl1ycnOOhQRBVxe81XKexwYytRBH1rLk+FJwfjwFPPv0wWOqCgXS/ukc2dy6uhhQMqvxrWnN3IWwSg7ooaoWRMmPy0K75pG4AnRF/9Z6dkDMdZGRKMjbr/albenVyyyIfDzg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=KiTw48/Do4Hb7RqdI8Yd46KpyvlMTp2P4ArSLhD2ch8=; b=IIETjQnTywEgzgyT0L1SMoeo5i2iO8y+hpVNRnSLtZCpiEsVmyLZBiAwLTfyjdq83Grhw52Lw8s89oyKI4dMWMyke17JVR+lCEpE0xug9hxmgizyVh/l+ZPidbgHn2NoXMBd0Tjx4Fjtf/90WNKpnmHQcyYGp+8CE3AKYQj7hLPh/BnGNpr9+NvH9w2kBlO+colyyD/SnrGKaDZsS3nlfi7MnKwqhlvqPw6lMtgneZajWsLD3gpzbo0OScZxEJrKt2D2tnDaNTtJ+AtAaxq2k5kB4b6AOGaEWFjpn7bjaf/2yNbFmnlHlOJ1y7ysddejKYCRImWQa4pejslTwIX/+A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=intel.com smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=KiTw48/Do4Hb7RqdI8Yd46KpyvlMTp2P4ArSLhD2ch8=; b=1eaSITYQ7nGGsgzwP3mcoCl/n/WYQJOZVKzeBbLRXbUe3Z6Cbif8dgrE3RK3TmPqEDin299c50cH1xgQTjiFViWO63ME6b2XZxl+K5bxQWCwBvmQJxKA52PldI/5O8kc9HD9rSjsqhwtuB/hJLwRKn6lwdw8D40xgPYhfHvVNlQ= Received: from BN0PR02CA0031.namprd02.prod.outlook.com (2603:10b6:408:e5::6) by CY8PR12MB7148.namprd12.prod.outlook.com (2603:10b6:930:5c::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6277.31; Mon, 10 Apr 2023 10:22:42 +0000 Received: from BN8NAM11FT052.eop-nam11.prod.protection.outlook.com (2603:10b6:408:e5:cafe::7b) by BN0PR02CA0031.outlook.office365.com (2603:10b6:408:e5::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6277.34 via Frontend Transport; Mon, 10 Apr 2023 10:22:42 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by BN8NAM11FT052.mail.protection.outlook.com (10.13.177.210) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6298.26 via Frontend Transport; Mon, 10 Apr 2023 10:22:41 +0000 Received: from telco-siena.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Mon, 10 Apr 2023 05:22:35 -0500 From: Sivaprasad Tummala To: CC: Subject: [PATCH v1] power: amd power monitor support Date: Mon, 10 Apr 2023 03:21:49 -0700 Message-ID: <20230410102149.198389-1-Sivaprasad.Tummala@amd.com> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN8NAM11FT052:EE_|CY8PR12MB7148:EE_ X-MS-Office365-Filtering-Correlation-Id: 63811f9d-7dd0-4d12-c2a5-08db39ad8492 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: ASiGqR1niW6c+pUo3xf4WOGEw3vR4Zoqk/XX0XyDVVq70T+Rktn/RVDUlM8dlTz2JD7Nzj9rhVgAQgV9dpkgQrKq7tfEe8vYuYHvImR20+RSEytm5+81S9Z7Twsdr1CpsV6x3WrOPRFKOa1A45q4VPuh456cPenF8wnYzQgdFj17ZoB3pLNvQtt6sPZHdyEmUlYCsV6IXFvRbJN4pSP+7tSA/fJ4n1i2vnpi76Rxdm66u3rD0hT3PmKXUsixJTJGe5SCO0f8Ugy/OmZXflpzrZbdTIIQK86F/nrfj8/Ksj1odSGYHhQwywTXSmZkfLLlxHx6B0dRxUVx94ZPlYqCkfxwu5qEvJreztgtUuVyefc/C4EQZ9lcLr2Vql8XOUu1U2WDq15n1oHoNAXEPupPDF+tDLxESvgOdCX/PylqIQ1DULEsKo+ziDkJeqiZELZASM0ZuAI5y4EVPft0T47fHlaFlzYZ6PRUXPcb+SJiCbOkosFwwelIhCNx9mUr6ELAcF/YkKCjfWPF8USbexRbFCpmNbHYruL0ZKXfF3fZIbcevO8tyjHN/fEmItMz2rfTd/S81KoWGjrMcX/cCeRjJoDSnUCCesp7ze93oESb4ptK/OaMOgbRoq5OEQjjNN27CN4FV99NtAaDqkwxmwRbtYCBJjX1zDH4Xz3ZeT58paYsiWWv3NnGa8k6bKwzhm1rIb0BxRy0g2WRkDztt6tpddwkxDkN74/Jn6jQ4XPB+Hk= X-Forefront-Antispam-Report: CIP:165.204.84.17; CTRY:US; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:SATLEXMB04.amd.com; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230028)(4636009)(396003)(376002)(346002)(39860400002)(136003)(451199021)(46966006)(40470700004)(36840700001)(40480700001)(6916009)(40460700003)(70206006)(82740400003)(70586007)(8676002)(41300700001)(356005)(478600001)(8936002)(316002)(4326008)(81166007)(5660300002)(36860700001)(186003)(336012)(47076005)(16526019)(426003)(83380400001)(7696005)(6666004)(2616005)(1076003)(26005)(2906002)(36756003)(82310400005)(86362001)(36900700001); DIR:OUT; SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Apr 2023 10:22:41.8924 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 63811f9d-7dd0-4d12-c2a5-08db39ad8492 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d; Ip=[165.204.84.17]; Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT052.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY8PR12MB7148 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org mwaitx allows epyc processors to enter a implementation dependent power/performance optimized state (C1 state) for a specific period or until a store to the monitored address range. Signed-off-by: Sivaprasad Tummala --- lib/eal/include/generic/rte_cpuflags.h | 2 + lib/eal/x86/include/rte_cpuflags.h | 1 + lib/eal/x86/rte_cpuflags.c | 3 + lib/eal/x86/rte_power_intrinsics.c | 80 +++++++++++++++++++++++++- lib/power/rte_power_pmd_mgmt.c | 3 +- 5 files changed, 86 insertions(+), 3 deletions(-) diff --git a/lib/eal/include/generic/rte_cpuflags.h b/lib/eal/include/generic/rte_cpuflags.h index d35551e931..db653a8dd7 100644 --- a/lib/eal/include/generic/rte_cpuflags.h +++ b/lib/eal/include/generic/rte_cpuflags.h @@ -26,6 +26,8 @@ struct rte_cpu_intrinsics { /**< indicates support for rte_power_pause function */ uint32_t power_monitor_multi : 1; /**< indicates support for rte_power_monitor_multi function */ + uint32_t amd_power_monitorx : 1; + /**< indicates amd support for rte_power_monitor function */ }; /** diff --git a/lib/eal/x86/include/rte_cpuflags.h b/lib/eal/x86/include/rte_cpuflags.h index 92e90fb6e0..d3e608533a 100644 --- a/lib/eal/x86/include/rte_cpuflags.h +++ b/lib/eal/x86/include/rte_cpuflags.h @@ -133,6 +133,7 @@ enum rte_cpu_flag_t { RTE_CPUFLAG_AVX512VP2INTERSECT, /**< AVX512 Two Register Intersection */ RTE_CPUFLAG_WAITPKG, /**< UMONITOR/UMWAIT/TPAUSE */ + RTE_CPUFLAG_MONITORX, /**< MONITORX */ /* The last item */ RTE_CPUFLAG_NUMFLAGS, /**< This should always be the last! */ diff --git a/lib/eal/x86/rte_cpuflags.c b/lib/eal/x86/rte_cpuflags.c index d6b518251b..ae2e0a8470 100644 --- a/lib/eal/x86/rte_cpuflags.c +++ b/lib/eal/x86/rte_cpuflags.c @@ -133,6 +133,7 @@ const struct feature_entry rte_cpu_feature_table[] = { FEAT_DEF(LAHF_SAHF, 0x80000001, 0, RTE_REG_ECX, 0) FEAT_DEF(LZCNT, 0x80000001, 0, RTE_REG_ECX, 4) + FEAT_DEF(MONITORX, 0x80000001, 0, RTE_REG_ECX, 29) FEAT_DEF(SYSCALL, 0x80000001, 0, RTE_REG_EDX, 11) FEAT_DEF(XD, 0x80000001, 0, RTE_REG_EDX, 20) @@ -191,5 +192,7 @@ rte_cpu_get_intrinsics_support(struct rte_cpu_intrinsics *intrinsics) intrinsics->power_pause = 1; if (rte_cpu_get_flag_enabled(RTE_CPUFLAG_RTM)) intrinsics->power_monitor_multi = 1; + } else if (rte_cpu_get_flag_enabled(RTE_CPUFLAG_MONITORX)) { + intrinsics->amd_power_monitorx = 1; } } diff --git a/lib/eal/x86/rte_power_intrinsics.c b/lib/eal/x86/rte_power_intrinsics.c index f749da9b85..4e81870387 100644 --- a/lib/eal/x86/rte_power_intrinsics.c +++ b/lib/eal/x86/rte_power_intrinsics.c @@ -30,6 +30,7 @@ __umwait_wakeup(volatile void *addr) static bool wait_supported; static bool wait_multi_supported; +static bool amd_mwaitx_supported; static inline uint64_t __get_umwait_val(const volatile void *p, const uint8_t sz) @@ -65,6 +66,76 @@ __check_val_size(const uint8_t sz) } } +/** + * This function uses MONITORX/MWAITX instructions and will enter C1 state. + * For more information about usage of these instructions, please refer to + * AMD64 Architecture Programmer’s Manual. + */ +static inline int +amd_power_monitorx(const struct rte_power_monitor_cond *pmc, + const uint64_t tsc_timestamp) +{ + const unsigned int lcore_id = rte_lcore_id(); + struct power_wait_status *s; + uint64_t cur_value; + + RTE_SET_USED(tsc_timestamp); + + /* prevent non-EAL thread from using this API */ + if (lcore_id >= RTE_MAX_LCORE) + return -EINVAL; + + if (pmc == NULL) + return -EINVAL; + + if (__check_val_size(pmc->size) < 0) + return -EINVAL; + + if (pmc->fn == NULL) + return -EINVAL; + + s = &wait_status[lcore_id]; + + /* update sleep address */ + rte_spinlock_lock(&s->lock); + s->monitor_addr = pmc->addr; + + /* + * we're using raw byte codes for now as only the newest compiler + * versions support this instruction natively. + */ + /* set address for MONITORX */ + asm volatile(".byte 0x0f, 0x01, 0xfa;" + : + : "a"(pmc->addr), + "c"(0), /* no extensions */ + "d"(0)); /* no hints */ + + /* now that we've put this address into monitor, we can unlock */ + rte_spinlock_unlock(&s->lock); + + cur_value = __get_umwait_val(pmc->addr, pmc->size); + + /* check if callback indicates we should abort */ + if (pmc->fn(cur_value, pmc->opaque) != 0) + goto end; + + /* execute MWAITX */ + asm volatile(".byte 0x0f, 0x01, 0xfb;" + : /* ignore rflags */ + : "a"(0), /* enter C1 */ + "c"(0), /* no time-out */ + "b"(0)); + +end: + /* erase sleep address */ + rte_spinlock_lock(&s->lock); + s->monitor_addr = NULL; + rte_spinlock_unlock(&s->lock); + + return 0; +} + /** * This function uses UMONITOR/UMWAIT instructions and will enter C0.2 state. * For more information about usage of these instructions, please refer to @@ -80,6 +151,9 @@ rte_power_monitor(const struct rte_power_monitor_cond *pmc, struct power_wait_status *s; uint64_t cur_value; + if (amd_mwaitx_supported) + return amd_power_monitorx(pmc, tsc_timestamp); + /* prevent user from running this instruction if it's not supported */ if (!wait_supported) return -ENOTSUP; @@ -126,7 +200,7 @@ rte_power_monitor(const struct rte_power_monitor_cond *pmc, asm volatile(".byte 0xf2, 0x0f, 0xae, 0xf7;" : /* ignore rflags */ : "D"(0), /* enter C0.2 */ - "a"(tsc_l), "d"(tsc_h)); + "a"(tsc_l), "d"(tsc_h)); end: /* erase sleep address */ @@ -170,6 +244,8 @@ RTE_INIT(rte_power_intrinsics_init) { wait_supported = 1; if (i.power_monitor_multi) wait_multi_supported = 1; + if (i.amd_power_monitorx) + amd_mwaitx_supported = 1; } int @@ -178,7 +254,7 @@ rte_power_monitor_wakeup(const unsigned int lcore_id) struct power_wait_status *s; /* prevent user from running this instruction if it's not supported */ - if (!wait_supported) + if (!wait_supported && !amd_mwaitx_supported) return -ENOTSUP; /* prevent buffer overrun */ diff --git a/lib/power/rte_power_pmd_mgmt.c b/lib/power/rte_power_pmd_mgmt.c index ca1840387c..43971e6014 100644 --- a/lib/power/rte_power_pmd_mgmt.c +++ b/lib/power/rte_power_pmd_mgmt.c @@ -447,7 +447,8 @@ check_monitor(struct pmd_core_cfg *cfg, const union queue *qdata) bool multimonitor_supported; /* check if rte_power_monitor is supported */ - if (!global_data.intrinsics_support.power_monitor) { + if ((!global_data.intrinsics_support.power_monitor) && + (!global_data.intrinsics_support.amd_power_monitorx)) { RTE_LOG(DEBUG, POWER, "Monitoring intrinsics are not supported\n"); return -ENOTSUP; }