From patchwork Mon Mar 4 20:51:33 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Eads, Gage" X-Patchwork-Id: 50805 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 189C64C96; Mon, 4 Mar 2019 21:51:42 +0100 (CET) Received: from mga06.intel.com (mga06.intel.com [134.134.136.31]) by dpdk.org (Postfix) with ESMTP id B283C493D for ; Mon, 4 Mar 2019 21:51:39 +0100 (CET) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga007.jf.intel.com ([10.7.209.58]) by orsmga104.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 04 Mar 2019 12:51:39 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.58,441,1544515200"; d="scan'208";a="119568339" Received: from txasoft-yocto.an.intel.com ([10.123.72.192]) by orsmga007.jf.intel.com with ESMTP; 04 Mar 2019 12:51:38 -0800 From: Gage Eads To: dev@dpdk.org Cc: olivier.matz@6wind.com, arybchenko@solarflare.com, bruce.richardson@intel.com, konstantin.ananyev@intel.com, gavin.hu@arm.com, Honnappa.Nagarahalli@arm.com, nd@arm.com, chaozhu@linux.vnet.ibm.com, jerinj@marvell.com, hemant.agrawal@nxp.com Date: Mon, 4 Mar 2019 14:51:33 -0600 Message-Id: <20190304205133.2248-2-gage.eads@intel.com> X-Mailer: git-send-email 2.13.6 In-Reply-To: <20190304205133.2248-1-gage.eads@intel.com> References: <20190222154640.22029-1-gage.eads@intel.com> <20190304205133.2248-1-gage.eads@intel.com> Subject: [dpdk-dev] [PATCH v3 1/1] eal: add 128-bit compare exchange (x86-64 only) X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This operation can be used for non-blocking algorithms, such as a non-blocking stack or ring. Signed-off-by: Gage Eads Reviewed-by: Honnappa Nagarahalli --- .../common/include/arch/x86/rte_atomic_64.h | 33 ++++++++++++ lib/librte_eal/common/include/generic/rte_atomic.h | 59 ++++++++++++++++++++++ 2 files changed, 92 insertions(+) diff --git a/lib/librte_eal/common/include/arch/x86/rte_atomic_64.h b/lib/librte_eal/common/include/arch/x86/rte_atomic_64.h index fd2ec9c53..3b061ff7c 100644 --- a/lib/librte_eal/common/include/arch/x86/rte_atomic_64.h +++ b/lib/librte_eal/common/include/arch/x86/rte_atomic_64.h @@ -34,6 +34,7 @@ /* * Inspired from FreeBSD src/sys/amd64/include/atomic.h * Copyright (c) 1998 Doug Rabson + * Copyright (c) 2019 Intel Corporation * All rights reserved. */ @@ -46,6 +47,7 @@ #include #include +#include #include /*------------------------- 64 bit atomic operations -------------------------*/ @@ -208,4 +210,35 @@ static inline void rte_atomic64_clear(rte_atomic64_t *v) } #endif +static inline int __rte_experimental +rte_atomic128_cmp_exchange(rte_int128_t *dst, + rte_int128_t *exp, + const rte_int128_t *src, + unsigned int weak, + int success, + int failure) +{ + RTE_SET_USED(weak); + RTE_SET_USED(success); + RTE_SET_USED(failure); + uint8_t res; + + asm volatile ( + MPLOCKED + "cmpxchg16b %[dst];" + " sete %[res]" + : [dst] "=m" (dst->val[0]), + "=a" (exp->val[0]), + "=d" (exp->val[1]), + [res] "=r" (res) + : "b" (src->val[0]), + "c" (src->val[1]), + "a" (exp->val[0]), + "d" (exp->val[1]), + "m" (dst->val[0]) + : "memory"); + + return res; +} + #endif /* _RTE_ATOMIC_X86_64_H_ */ diff --git a/lib/librte_eal/common/include/generic/rte_atomic.h b/lib/librte_eal/common/include/generic/rte_atomic.h index 4afd1acc3..a7d58c190 100644 --- a/lib/librte_eal/common/include/generic/rte_atomic.h +++ b/lib/librte_eal/common/include/generic/rte_atomic.h @@ -14,6 +14,7 @@ #include #include +#include #ifdef __DOXYGEN__ @@ -1082,4 +1083,62 @@ static inline void rte_atomic64_clear(rte_atomic64_t *v) } #endif +/*------------------------ 128 bit atomic operations -------------------------*/ + +#if defined(RTE_ARCH_X86_64) +/** + * 128-bit integer structure. + */ +RTE_STD_C11 +typedef struct { + RTE_STD_C11 + union { + uint64_t val[2]; + __int128 int128; + }; +} __rte_aligned(16) rte_int128_t; + +/** + * An atomic compare and set function used by the mutex functions. + * (Atomically) Equivalent to: + * if (*dst == *exp) + * *dst = *src + * else + * *exp = *dst + * + * @note The success and failure arguments must be one of the __ATOMIC_* values + * defined in the C++11 standard. For details on their behavior, refer to the + * standard. + * + * @param dst + * The destination into which the value will be written. + * @param exp + * Pointer to the expected value. If the operation fails, this memory is + * updated with the actual value. + * @param src + * Pointer to the new value. + * @param weak + * A value of true allows the comparison to spuriously fail and allows the + * 'exp' update to occur non-atomically (i.e. a torn read may occur). + * Implementations may ignore this argument and only implement the strong + * variant. + * @param success + * If successful, the operation's memory behavior conforms to this (or a + * stronger) model. + * @param failure + * If unsuccessful, the operation's memory behavior conforms to this (or a + * stronger) model. This argument cannot be __ATOMIC_RELEASE, + * __ATOMIC_ACQ_REL, or a stronger model than success. + * @return + * Non-zero on success; 0 on failure. + */ +static inline int __rte_experimental +rte_atomic128_cmp_exchange(rte_int128_t *dst, + rte_int128_t *exp, + const rte_int128_t *src, + unsigned int weak, + int success, + int failure); +#endif + #endif /* _RTE_ATOMIC_H_ */