From patchwork Mon Nov 12 10:47:16 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bruce Richardson X-Patchwork-Id: 47997 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 099FE325F; Mon, 12 Nov 2018 11:47:31 +0100 (CET) Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by dpdk.org (Postfix) with ESMTP id 7A6682B8C; Mon, 12 Nov 2018 11:47:27 +0100 (CET) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by orsmga101.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 12 Nov 2018 02:47:27 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.54,495,1534834800"; d="scan'208";a="105512857" Received: from silpixa00399126.ir.intel.com (HELO silpixa00399126.ger.corp.intel.com) ([10.237.223.223]) by fmsmga004.fm.intel.com with ESMTP; 12 Nov 2018 02:47:26 -0800 From: Bruce Richardson To: dev@dpdk.org Cc: stable@dpdk.org, Yipeng Wang , Bruce Richardson Date: Mon, 12 Nov 2018 10:47:16 +0000 Message-Id: <20181112104719.62568-2-bruce.richardson@intel.com> X-Mailer: git-send-email 2.19.1 In-Reply-To: <20181112104719.62568-1-bruce.richardson@intel.com> References: <20181105173913.61225-1-bruce.richardson@intel.com> <20181112104719.62568-1-bruce.richardson@intel.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v2 1/4] hash: fix TSX aborts with newer gcc X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Yipeng Wang gcc 7 and 8 with O3 will generate vzeroupper from rte_memcpy into TSX region which may abort the TSX transaction. This fix changes rte_memcpy to memcpy which will not insert extra vzeroupper into the library. Fixes: f2e3001b53ec ("hash: support read/write concurrency") Cc: stable@dpdk.org Signed-off-by: Yipeng Wang Signed-off-by: Bruce Richardson --- lib/librte_hash/rte_cmp_x86.h | 2 ++ lib/librte_hash/rte_cuckoo_hash.c | 3 +-- 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/lib/librte_hash/rte_cmp_x86.h b/lib/librte_hash/rte_cmp_x86.h index e82b4c0..13a5836 100644 --- a/lib/librte_hash/rte_cmp_x86.h +++ b/lib/librte_hash/rte_cmp_x86.h @@ -2,6 +2,8 @@ * Copyright(c) 2015 Intel Corporation */ +#include + /* Functions to compare multiple of 16 byte keys (up to 128 bytes) */ static int rte_hash_k16_cmp_eq(const void *key1, const void *key2, size_t key_len __rte_unused) diff --git a/lib/librte_hash/rte_cuckoo_hash.c b/lib/librte_hash/rte_cuckoo_hash.c index 5ddcccd..76f5dc8 100644 --- a/lib/librte_hash/rte_cuckoo_hash.c +++ b/lib/librte_hash/rte_cuckoo_hash.c @@ -13,7 +13,6 @@ #include #include /* for definition of RTE_CACHE_LINE_SIZE */ #include -#include #include #include #include @@ -982,7 +981,7 @@ struct rte_hash * new_k = RTE_PTR_ADD(keys, (uintptr_t)slot_id * h->key_entry_size); new_idx = (uint32_t)((uintptr_t) slot_id); /* Copy key */ - rte_memcpy(new_k->key, key, h->key_len); + memcpy(new_k->key, key, h->key_len); /* Key can be of arbitrary length, so it is not possible to store * it atomically. Hence the new key element's memory stores * (key as well as data) should be complete before it is referenced. From patchwork Mon Nov 12 10:47:17 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bruce Richardson X-Patchwork-Id: 47998 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id CA6AA4CBB; Mon, 12 Nov 2018 11:47:32 +0100 (CET) Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by dpdk.org (Postfix) with ESMTP id 6EA6C325F; Mon, 12 Nov 2018 11:47:29 +0100 (CET) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by orsmga101.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 12 Nov 2018 02:47:29 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.54,495,1534834800"; d="scan'208";a="105512862" Received: from silpixa00399126.ir.intel.com (HELO silpixa00399126.ger.corp.intel.com) ([10.237.223.223]) by fmsmga004.fm.intel.com with ESMTP; 12 Nov 2018 02:47:28 -0800 From: Bruce Richardson To: dev@dpdk.org Cc: stable@dpdk.org, Yipeng Wang Date: Mon, 12 Nov 2018 10:47:17 +0000 Message-Id: <20181112104719.62568-3-bruce.richardson@intel.com> X-Mailer: git-send-email 2.19.1 In-Reply-To: <20181112104719.62568-1-bruce.richardson@intel.com> References: <20181105173913.61225-1-bruce.richardson@intel.com> <20181112104719.62568-1-bruce.richardson@intel.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v2 2/4] hash: add local cache for TSX region X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Yipeng Wang This patch adds back the local cache when TSX support is turned on. When TSX is turned on, free key-data slot ring would be contended by various TSX regions. The purpose of this commit is to reduce possible memory collisions during key insertion. Signed-off-by: Yipeng Wang Acked-by: Bruce Richardson --- lib/librte_hash/rte_cuckoo_hash.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/lib/librte_hash/rte_cuckoo_hash.c b/lib/librte_hash/rte_cuckoo_hash.c index 76f5dc8..5d5f9f1 100644 --- a/lib/librte_hash/rte_cuckoo_hash.c +++ b/lib/librte_hash/rte_cuckoo_hash.c @@ -180,8 +180,10 @@ struct rte_hash * } /* Check extra flags field to check extra options. */ - if (params->extra_flag & RTE_HASH_EXTRA_FLAGS_TRANS_MEM_SUPPORT) + if (params->extra_flag & RTE_HASH_EXTRA_FLAGS_TRANS_MEM_SUPPORT) { + use_local_cache = 1; hw_trans_mem_support = 1; + } if (params->extra_flag & RTE_HASH_EXTRA_FLAGS_MULTI_WRITER_ADD) { use_local_cache = 1; From patchwork Mon Nov 12 10:47:18 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bruce Richardson X-Patchwork-Id: 47999 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 96CD84CC5; Mon, 12 Nov 2018 11:47:35 +0100 (CET) Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by dpdk.org (Postfix) with ESMTP id 83C7B4CAB; Mon, 12 Nov 2018 11:47:31 +0100 (CET) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by orsmga101.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 12 Nov 2018 02:47:31 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.54,495,1534834800"; d="scan'208";a="105512871" Received: from silpixa00399126.ir.intel.com (HELO silpixa00399126.ger.corp.intel.com) ([10.237.223.223]) by fmsmga004.fm.intel.com with ESMTP; 12 Nov 2018 02:47:30 -0800 From: Bruce Richardson To: dev@dpdk.org Cc: stable@dpdk.org, Bruce Richardson Date: Mon, 12 Nov 2018 10:47:18 +0000 Message-Id: <20181112104719.62568-4-bruce.richardson@intel.com> X-Mailer: git-send-email 2.19.1 In-Reply-To: <20181112104719.62568-1-bruce.richardson@intel.com> References: <20181105173913.61225-1-bruce.richardson@intel.com> <20181112104719.62568-1-bruce.richardson@intel.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v2 3/4] eal/x86: reduce contention when retrying TSX X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" When TSX transactions abort, it is generally worth retrying a number of times before falling back to the traditional locking path, as the parallelism benefits from TSX can be worth it when a transaction does succeed. For cases with multiple threads and high contention rates, it can be useful to have increasing delays between retry attempts, so as to avoid having the same threads repeatedly collided. Signed-off-by: Bruce Richardson --- V2: Have retry with backoff for all cases of memory conflicts, not just those where we explicitly abort due to the lock being held. --- .../common/include/arch/x86/rte_spinlock.h | 21 +++++++++++++++++---- lib/librte_eal/linuxapp/eal/eal_alarm.c | 2 ++ 2 files changed, 19 insertions(+), 4 deletions(-) diff --git a/lib/librte_eal/common/include/arch/x86/rte_spinlock.h b/lib/librte_eal/common/include/arch/x86/rte_spinlock.h index 60321da..e2e2b26 100644 --- a/lib/librte_eal/common/include/arch/x86/rte_spinlock.h +++ b/lib/librte_eal/common/include/arch/x86/rte_spinlock.h @@ -15,8 +15,9 @@ #include "rte_branch_prediction.h" #include "rte_common.h" #include "rte_pause.h" +#include "rte_cycles.h" -#define RTE_RTM_MAX_RETRIES (10) +#define RTE_RTM_MAX_RETRIES (20) #define RTE_XABORT_LOCK_BUSY (0xff) #ifndef RTE_FORCE_INTRINSICS @@ -76,7 +77,7 @@ static inline int rte_tm_supported(void) static inline int rte_try_tm(volatile int *lock) { - int retries; + int i, retries; if (!rte_rtm_supported) return 0; @@ -96,9 +97,21 @@ static inline int rte_tm_supported(void) while (*lock) rte_pause(); - if ((status & RTE_XABORT_EXPLICIT) && - (RTE_XABORT_CODE(status) == RTE_XABORT_LOCK_BUSY)) + if ((status & RTE_XABORT_CONFLICT) || + ((status & RTE_XABORT_EXPLICIT) && + (RTE_XABORT_CODE(status) == RTE_XABORT_LOCK_BUSY))) { + /* add a small delay before retrying, basing the + * delay on the number of times we've already tried, + * to give a back-off type of behaviour. We + * randomize trycount by taking bits from the tsc count + */ + int try_count = RTE_RTM_MAX_RETRIES - retries; + int pause_count = (rte_rdtsc() & 0x7) | 1; + pause_count <<= try_count; + for (i = 0; i < pause_count; i++) + rte_pause(); continue; + } if ((status & RTE_XABORT_RETRY) == 0) /* do not retry */ break; diff --git a/lib/librte_eal/linuxapp/eal/eal_alarm.c b/lib/librte_eal/linuxapp/eal/eal_alarm.c index 391d2a6..840ede7 100644 --- a/lib/librte_eal/linuxapp/eal/eal_alarm.c +++ b/lib/librte_eal/linuxapp/eal/eal_alarm.c @@ -30,7 +30,9 @@ #define NS_PER_US 1000 #define US_PER_MS 1000 #define MS_PER_S 1000 +#ifndef US_PER_S #define US_PER_S (US_PER_MS * MS_PER_S) +#endif #ifdef CLOCK_MONOTONIC_RAW /* Defined in glibc bits/time.h */ #define CLOCK_TYPE_ID CLOCK_MONOTONIC_RAW From patchwork Mon Nov 12 10:47:19 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bruce Richardson X-Patchwork-Id: 48000 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 2F833532C; Mon, 12 Nov 2018 11:47:37 +0100 (CET) Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by dpdk.org (Postfix) with ESMTP id 5EA474CC5; Mon, 12 Nov 2018 11:47:34 +0100 (CET) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by orsmga101.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 12 Nov 2018 02:47:34 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.54,495,1534834800"; d="scan'208";a="105512876" Received: from silpixa00399126.ir.intel.com (HELO silpixa00399126.ger.corp.intel.com) ([10.237.223.223]) by fmsmga004.fm.intel.com with ESMTP; 12 Nov 2018 02:47:33 -0800 From: Bruce Richardson To: dev@dpdk.org Cc: stable@dpdk.org, Bruce Richardson Date: Mon, 12 Nov 2018 10:47:19 +0000 Message-Id: <20181112104719.62568-5-bruce.richardson@intel.com> X-Mailer: git-send-email 2.19.1 In-Reply-To: <20181112104719.62568-1-bruce.richardson@intel.com> References: <20181105173913.61225-1-bruce.richardson@intel.com> <20181112104719.62568-1-bruce.richardson@intel.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v2 4/4] test/test: improve output for hash read-write test X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" The hash read-write autotest generates a lot of text, which is very dense on the screen. Even the summary at the end is hard to follow as everything is very compact. We can improve readability by highlighting the starts of the various sections, and by indenting the values within subsections. Signed-off-by: Bruce Richardson --- test/test/test_hash_readwrite.c | 20 +++++++++++--------- 1 file changed, 11 insertions(+), 9 deletions(-) diff --git a/test/test/test_hash_readwrite.c b/test/test/test_hash_readwrite.c index 01f986c..6b695ce 100644 --- a/test/test/test_hash_readwrite.c +++ b/test/test/test_hash_readwrite.c @@ -678,24 +678,26 @@ struct { reader_faster) < 0) return -1; + printf("================\n"); printf("Results summary:\n"); + printf("================\n"); printf("single read: %u\n", htm_results.single_read); printf("single write: %u\n", htm_results.single_write); for (i = 0; i < NUM_TEST; i++) { - printf("core_cnt: %u\n", core_cnt[i]); + printf("+++ core_cnt: %u +++\n", core_cnt[i]); printf("HTM:\n"); - printf("read only: %u\n", htm_results.read_only[i]); - printf("write only: %u\n", htm_results.write_only[i]); - printf("read-write read: %u\n", htm_results.read_write_r[i]); - printf("read-write write: %u\n", htm_results.read_write_w[i]); + printf(" read only: %u\n", htm_results.read_only[i]); + printf(" write only: %u\n", htm_results.write_only[i]); + printf(" read-write read: %u\n", htm_results.read_write_r[i]); + printf(" read-write write: %u\n", htm_results.read_write_w[i]); printf("non HTM:\n"); - printf("read only: %u\n", non_htm_results.read_only[i]); - printf("write only: %u\n", non_htm_results.write_only[i]); - printf("read-write read: %u\n", + printf(" read only: %u\n", non_htm_results.read_only[i]); + printf(" write only: %u\n", non_htm_results.write_only[i]); + printf(" read-write read: %u\n", non_htm_results.read_write_r[i]); - printf("read-write write: %u\n", + printf(" read-write write: %u\n", non_htm_results.read_write_w[i]); }