From patchwork Mon Oct 8 09:11:43 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Phil Yang X-Patchwork-Id: 46239 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id A9C3F49E0; Mon, 8 Oct 2018 11:11:59 +0200 (CEST) Received: from foss.arm.com (foss.arm.com [217.140.101.70]) by dpdk.org (Postfix) with ESMTP id E2D67160 for ; Mon, 8 Oct 2018 11:11:57 +0200 (CEST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 059267A9; Mon, 8 Oct 2018 02:11:57 -0700 (PDT) Received: from localhost.localdomain (unknown [10.171.20.31]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 541CA3F5D3; Mon, 8 Oct 2018 02:11:55 -0700 (PDT) From: Phil Yang To: dev@dpdk.org Cc: jerin.jacob@caviumnetworks.com, Gavin.Hu@arm.com, Honnappa.Nagarahalli@arm.com, Ola.Liljedahl@arm.com, ferruh.yigit@intel.com, phil.yang@arm.com Date: Mon, 8 Oct 2018 17:11:43 +0800 Message-Id: <1538989906-8349-1-git-send-email-phil.yang@arm.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1537364560-4124-1-git-send-email-phil.yang@arm.com> References: <1537364560-4124-1-git-send-email-phil.yang@arm.com> Subject: [dpdk-dev] [PATCH v3 1/4] config: use one single config option for C11 memory model X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Keep only single config option RTE_USE_C11_MEM_MODEL for C11 memory model, so all modules can leverage C11 atomic extension by enable this option. Signed-off-by: Phil Yang Reviewed-by: Honnappa Nagarahalli Reviewed-by: Gavin Hu Acked-by: Jerin Jacob Reviewed-by: Ferruh Yigit --- config/arm/meson.build | 2 +- config/common_armv8a_linuxapp | 2 +- config/common_base | 2 +- config/defconfig_arm64-thunderx-linuxapp-gcc | 2 +- lib/librte_ring/rte_ring.h | 4 ++-- 5 files changed, 6 insertions(+), 6 deletions(-) diff --git a/config/arm/meson.build b/config/arm/meson.build index 94cca49..4b23b39 100644 --- a/config/arm/meson.build +++ b/config/arm/meson.build @@ -53,7 +53,7 @@ flags_cavium = [ ['RTE_MAX_NUMA_NODES', 2], ['RTE_MAX_LCORE', 96], ['RTE_MAX_VFIO_GROUPS', 128], - ['RTE_RING_USE_C11_MEM_MODEL', false]] + ['RTE_USE_C11_MEM_MODEL', false]] flags_dpaa = [ ['RTE_MACHINE', '"dpaa"'], ['RTE_CACHE_LINE_SIZE', 64], diff --git a/config/common_armv8a_linuxapp b/config/common_armv8a_linuxapp index 111c005..54e6987 100644 --- a/config/common_armv8a_linuxapp +++ b/config/common_armv8a_linuxapp @@ -29,7 +29,7 @@ CONFIG_RTE_ARCH_ARM64_MEMCPY=n #CONFIG_RTE_ARM64_MEMCPY_ALIGN_MASK=0xF #CONFIG_RTE_ARM64_MEMCPY_STRICT_ALIGN=n -CONFIG_RTE_RING_USE_C11_MEM_MODEL=y +CONFIG_RTE_USE_C11_MEM_MODEL=y CONFIG_RTE_LIBRTE_FM10K_PMD=n CONFIG_RTE_LIBRTE_SFC_EFX_PMD=n diff --git a/config/common_base b/config/common_base index 155c7d4..ccd2670 100644 --- a/config/common_base +++ b/config/common_base @@ -661,7 +661,7 @@ CONFIG_RTE_LIBRTE_PMD_IFPGA_RAWDEV=y # Compile librte_ring # CONFIG_RTE_LIBRTE_RING=y -CONFIG_RTE_RING_USE_C11_MEM_MODEL=n +CONFIG_RTE_USE_C11_MEM_MODEL=n # # Compile librte_mempool diff --git a/config/defconfig_arm64-thunderx-linuxapp-gcc b/config/defconfig_arm64-thunderx-linuxapp-gcc index 2bed66c..f11e758 100644 --- a/config/defconfig_arm64-thunderx-linuxapp-gcc +++ b/config/defconfig_arm64-thunderx-linuxapp-gcc @@ -10,7 +10,7 @@ CONFIG_RTE_CACHE_LINE_SIZE=128 CONFIG_RTE_MAX_NUMA_NODES=2 CONFIG_RTE_MAX_LCORE=96 CONFIG_RTE_MAX_VFIO_GROUPS=128 -CONFIG_RTE_RING_USE_C11_MEM_MODEL=n +CONFIG_RTE_USE_C11_MEM_MODEL=n # # Compile PMD for octeontx sso event device diff --git a/lib/librte_ring/rte_ring.h b/lib/librte_ring/rte_ring.h index 7a731d0..af5444a 100644 --- a/lib/librte_ring/rte_ring.h +++ b/lib/librte_ring/rte_ring.h @@ -303,11 +303,11 @@ void rte_ring_dump(FILE *f, const struct rte_ring *r); * There are 2 choices for the users * 1.use rmb() memory barrier * 2.use one-direcion load_acquire/store_release barrier,defined by - * CONFIG_RTE_RING_USE_C11_MEM_MODEL=y + * CONFIG_RTE_USE_C11_MEM_MODEL=y * It depends on performance test results. * By default, move common functions to rte_ring_generic.h */ -#ifdef RTE_RING_USE_C11_MEM_MODEL +#ifdef RTE_USE_C11_MEM_MODEL #include "rte_ring_c11_mem.h" #else #include "rte_ring_generic.h" From patchwork Mon Oct 8 09:11:44 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Phil Yang X-Patchwork-Id: 46240 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 583854C8E; Mon, 8 Oct 2018 11:12:01 +0200 (CEST) Received: from foss.arm.com (foss.arm.com [217.140.101.70]) by dpdk.org (Postfix) with ESMTP id CDACD4C80 for ; Mon, 8 Oct 2018 11:11:59 +0200 (CEST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 4477CED1; Mon, 8 Oct 2018 02:11:59 -0700 (PDT) Received: from localhost.localdomain (unknown [10.171.20.31]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 50A5C3F5D3; Mon, 8 Oct 2018 02:11:57 -0700 (PDT) From: Phil Yang To: dev@dpdk.org Cc: jerin.jacob@caviumnetworks.com, Gavin.Hu@arm.com, Honnappa.Nagarahalli@arm.com, Ola.Liljedahl@arm.com, ferruh.yigit@intel.com, phil.yang@arm.com Date: Mon, 8 Oct 2018 17:11:44 +0800 Message-Id: <1538989906-8349-2-git-send-email-phil.yang@arm.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1538989906-8349-1-git-send-email-phil.yang@arm.com> References: <1537364560-4124-1-git-send-email-phil.yang@arm.com> <1538989906-8349-1-git-send-email-phil.yang@arm.com> Subject: [dpdk-dev] [PATCH v3 2/4] kni: fix kni fifo synchronization X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" With existing code in kni_fifo_put, rx_q values are not being updated before updating fifo_write. While reading rx_q in kni_net_rx_normal, This is causing the sync issue on other core. The same situation happens in kni_fifo_get as well. So syncing the values by adding memory barriers to make sure the values being synced before updating fifo_write and fifo_read. Fixes: 3fc5ca2 ("kni: initial import") Signed-off-by: Phil Yang Reviewed-by: Honnappa Nagarahalli Reviewed-by: Gavin Hu Reviewed-by: Ola Liljedahl Reviewed-by: Jerin Jacob --- lib/librte_kni/rte_kni_fifo.h | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/lib/librte_kni/rte_kni_fifo.h b/lib/librte_kni/rte_kni_fifo.h index ac26a8c..70ac14e 100644 --- a/lib/librte_kni/rte_kni_fifo.h +++ b/lib/librte_kni/rte_kni_fifo.h @@ -28,8 +28,9 @@ kni_fifo_put(struct rte_kni_fifo *fifo, void **data, unsigned num) { unsigned i = 0; unsigned fifo_write = fifo->write; - unsigned fifo_read = fifo->read; unsigned new_write = fifo_write; + rte_smp_rmb(); + unsigned fifo_read = fifo->read; for (i = 0; i < num; i++) { new_write = (new_write + 1) & (fifo->len - 1); @@ -39,6 +40,7 @@ kni_fifo_put(struct rte_kni_fifo *fifo, void **data, unsigned num) fifo->buffer[fifo_write] = data[i]; fifo_write = new_write; } + rte_smp_wmb(); fifo->write = fifo_write; return i; } @@ -51,7 +53,9 @@ kni_fifo_get(struct rte_kni_fifo *fifo, void **data, unsigned num) { unsigned i = 0; unsigned new_read = fifo->read; + rte_smp_rmb(); unsigned fifo_write = fifo->write; + for (i = 0; i < num; i++) { if (new_read == fifo_write) break; @@ -59,6 +63,7 @@ kni_fifo_get(struct rte_kni_fifo *fifo, void **data, unsigned num) data[i] = fifo->buffer[new_read]; new_read = (new_read + 1) & (fifo->len - 1); } + rte_smp_rmb(); fifo->read = new_read; return i; } @@ -69,5 +74,8 @@ kni_fifo_get(struct rte_kni_fifo *fifo, void **data, unsigned num) static inline uint32_t kni_fifo_count(struct rte_kni_fifo *fifo) { - return (fifo->len + fifo->write - fifo->read) & (fifo->len - 1); + unsigned fifo_write = fifo->write; + rte_smp_rmb(); + unsigned fifo_read = fifo->read; + return (fifo->len + fifo_write - fifo_read) & (fifo->len - 1); } From patchwork Mon Oct 8 09:11:45 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Phil Yang X-Patchwork-Id: 46241 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id E04104C9F; Mon, 8 Oct 2018 11:12:05 +0200 (CEST) Received: from foss.arm.com (foss.arm.com [217.140.101.70]) by dpdk.org (Postfix) with ESMTP id 05EA51DB8 for ; Mon, 8 Oct 2018 11:12:01 +0200 (CEST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 4F74B7A9; Mon, 8 Oct 2018 02:12:01 -0700 (PDT) Received: from localhost.localdomain (unknown [10.171.20.31]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id A38603F5D3; Mon, 8 Oct 2018 02:11:59 -0700 (PDT) From: Phil Yang To: dev@dpdk.org Cc: jerin.jacob@caviumnetworks.com, Gavin.Hu@arm.com, Honnappa.Nagarahalli@arm.com, Ola.Liljedahl@arm.com, ferruh.yigit@intel.com, phil.yang@arm.com Date: Mon, 8 Oct 2018 17:11:45 +0800 Message-Id: <1538989906-8349-3-git-send-email-phil.yang@arm.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1538989906-8349-1-git-send-email-phil.yang@arm.com> References: <1537364560-4124-1-git-send-email-phil.yang@arm.com> <1538989906-8349-1-git-send-email-phil.yang@arm.com> Subject: [dpdk-dev] [PATCH v3 3/4] kni: fix kni kernel fifo synchronization X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Adding memory barrier to make sure the values being synced before updating fifo_write in kni_fifo_put and fifo_read in kni_fifo_get. Fixes: 3fc5ca2 ("kni: initial import") Signed-off-by: Phil Yang Reviewed-by: Honnappa Nagarahalli Reviewed-by: Gavin Hu --- kernel/linux/kni/kni_fifo.h | 16 ++++++++++------ .../linuxapp/eal/include/exec-env/rte_kni_common.h | 1 + 2 files changed, 11 insertions(+), 6 deletions(-) diff --git a/kernel/linux/kni/kni_fifo.h b/kernel/linux/kni/kni_fifo.h index 9a4762d..2cb3a4a 100644 --- a/kernel/linux/kni/kni_fifo.h +++ b/kernel/linux/kni/kni_fifo.h @@ -16,7 +16,7 @@ kni_fifo_put(struct rte_kni_fifo *fifo, void **data, uint32_t num) { uint32_t i = 0; uint32_t fifo_write = fifo->write; - uint32_t fifo_read = fifo->read; + uint32_t fifo_read = smp_load_acquire(&fifo->read); uint32_t new_write = fifo_write; for (i = 0; i < num; i++) { @@ -27,7 +27,7 @@ kni_fifo_put(struct rte_kni_fifo *fifo, void **data, uint32_t num) fifo->buffer[fifo_write] = data[i]; fifo_write = new_write; } - fifo->write = fifo_write; + smp_store_release(&fifo->write, fifo_write); return i; } @@ -40,7 +40,7 @@ kni_fifo_get(struct rte_kni_fifo *fifo, void **data, uint32_t num) { uint32_t i = 0; uint32_t new_read = fifo->read; - uint32_t fifo_write = fifo->write; + uint32_t fifo_write = smp_load_acquire(&fifo->write); for (i = 0; i < num; i++) { if (new_read == fifo_write) @@ -49,7 +49,7 @@ kni_fifo_get(struct rte_kni_fifo *fifo, void **data, uint32_t num) data[i] = fifo->buffer[new_read]; new_read = (new_read + 1) & (fifo->len - 1); } - fifo->read = new_read; + smp_store_release(&fifo->read, new_read); return i; } @@ -60,7 +60,9 @@ kni_fifo_get(struct rte_kni_fifo *fifo, void **data, uint32_t num) static inline uint32_t kni_fifo_count(struct rte_kni_fifo *fifo) { - return (fifo->len + fifo->write - fifo->read) & (fifo->len - 1); + uint32_t fifo_write = smp_load_acquire(&fifo->write); + uint32_t fifo_read = smp_load_acquire(&fifo->read); + return (fifo->len + fifo_write - fifo_read) & (fifo->len - 1); } /** @@ -69,7 +71,9 @@ kni_fifo_count(struct rte_kni_fifo *fifo) static inline uint32_t kni_fifo_free_count(struct rte_kni_fifo *fifo) { - return (fifo->read - fifo->write - 1) & (fifo->len - 1); + uint32_t fifo_write = smp_load_acquire(&fifo->write); + uint32_t fifo_read = smp_load_acquire(&fifo->read); + return (fifo_read - fifo_write - 1) & (fifo->len - 1); } #endif /* _KNI_FIFO_H_ */ diff --git a/lib/librte_eal/linuxapp/eal/include/exec-env/rte_kni_common.h b/lib/librte_eal/linuxapp/eal/include/exec-env/rte_kni_common.h index cfa9448..58e8533 100644 --- a/lib/librte_eal/linuxapp/eal/include/exec-env/rte_kni_common.h +++ b/lib/librte_eal/linuxapp/eal/include/exec-env/rte_kni_common.h @@ -8,6 +8,7 @@ #ifdef __KERNEL__ #include +#include #define RTE_STD_C11 #else #include From patchwork Mon Oct 8 09:11:46 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Phil Yang X-Patchwork-Id: 46242 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id C48FA4CC3; Mon, 8 Oct 2018 11:12:07 +0200 (CEST) Received: from foss.arm.com (usa-sjc-mx-foss1.foss.arm.com [217.140.101.70]) by dpdk.org (Postfix) with ESMTP id D8EB7160 for ; Mon, 8 Oct 2018 11:12:03 +0200 (CEST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 472157A9; Mon, 8 Oct 2018 02:12:03 -0700 (PDT) Received: from localhost.localdomain (unknown [10.171.20.31]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id A1D7B3F5D3; Mon, 8 Oct 2018 02:12:01 -0700 (PDT) From: Phil Yang To: dev@dpdk.org Cc: jerin.jacob@caviumnetworks.com, Gavin.Hu@arm.com, Honnappa.Nagarahalli@arm.com, Ola.Liljedahl@arm.com, ferruh.yigit@intel.com, phil.yang@arm.com Date: Mon, 8 Oct 2018 17:11:46 +0800 Message-Id: <1538989906-8349-4-git-send-email-phil.yang@arm.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1538989906-8349-1-git-send-email-phil.yang@arm.com> References: <1537364560-4124-1-git-send-email-phil.yang@arm.com> <1538989906-8349-1-git-send-email-phil.yang@arm.com> Subject: [dpdk-dev] [PATCH v3 4/4] kni: introduce c11 atomic into kni fifo synchronization X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Syncing the values by adding c11 atomic memory barriers to make sure the values being synced before updating fifo_write and fifo_read. Signed-off-by: Phil Yang Reviewed-by: Honnappa Nagarahalli Reviewed-by: Gavin Hu Reviewed-by: Ola Liljedahl --- .../linuxapp/eal/include/exec-env/rte_kni_common.h | 5 +++ lib/librte_kni/rte_kni_fifo.h | 47 +++++++++++++++++----- 2 files changed, 41 insertions(+), 11 deletions(-) diff --git a/lib/librte_eal/linuxapp/eal/include/exec-env/rte_kni_common.h b/lib/librte_eal/linuxapp/eal/include/exec-env/rte_kni_common.h index 58e8533..5afa087 100644 --- a/lib/librte_eal/linuxapp/eal/include/exec-env/rte_kni_common.h +++ b/lib/librte_eal/linuxapp/eal/include/exec-env/rte_kni_common.h @@ -55,8 +55,13 @@ struct rte_kni_request { * Writing should never overwrite the read position */ struct rte_kni_fifo { +#ifdef RTE_USE_C11_MEM_MODEL + unsigned write; /**< Next position to be written*/ + unsigned read; /**< Next position to be read */ +#else volatile unsigned write; /**< Next position to be written*/ volatile unsigned read; /**< Next position to be read */ +#endif unsigned len; /**< Circular buffer length */ unsigned elem_size; /**< Pointer size - for 32/64 bit OS */ void *volatile buffer[]; /**< The buffer contains mbuf pointers */ diff --git a/lib/librte_kni/rte_kni_fifo.h b/lib/librte_kni/rte_kni_fifo.h index 70ac14e..83d31b6 100644 --- a/lib/librte_kni/rte_kni_fifo.h +++ b/lib/librte_kni/rte_kni_fifo.h @@ -5,6 +5,36 @@ /** + * @internal when c11 memory model enabled use c11 atomic memory barrier. + * when under non c11 memory model use rte_smp_* memory barrier. + * + * @param src + * Pointer to the source data. + * @param dst + * Pointer to the destination data. + * @param value + * Data value. + */ +#ifdef RTE_USE_C11_MEM_MODEL +#define __KNI_LOAD_ACQUIRE(src) ({ \ + __atomic_load_n((src), __ATOMIC_ACQUIRE); \ + }) +#define __KNI_STORE_RELEASE(dst, value) do { \ + __atomic_store_n((dst), value, __ATOMIC_RELEASE); \ + } while(0) +#else +#define __KNI_LOAD_ACQUIRE(src) ({ \ + typeof (*(src)) val = *(src); \ + rte_smp_rmb(); \ + val; \ + }) +#define __KNI_STORE_RELEASE(dst, value) do { \ + *(dst) = value; \ + rte_smp_wmb(); \ + } while(0) +#endif + +/** * Initializes the kni fifo structure */ static void @@ -29,8 +59,7 @@ kni_fifo_put(struct rte_kni_fifo *fifo, void **data, unsigned num) unsigned i = 0; unsigned fifo_write = fifo->write; unsigned new_write = fifo_write; - rte_smp_rmb(); - unsigned fifo_read = fifo->read; + unsigned fifo_read = __KNI_LOAD_ACQUIRE(&fifo->read); for (i = 0; i < num; i++) { new_write = (new_write + 1) & (fifo->len - 1); @@ -40,8 +69,7 @@ kni_fifo_put(struct rte_kni_fifo *fifo, void **data, unsigned num) fifo->buffer[fifo_write] = data[i]; fifo_write = new_write; } - rte_smp_wmb(); - fifo->write = fifo_write; + __KNI_STORE_RELEASE(&fifo->write, fifo_write); return i; } @@ -53,8 +81,7 @@ kni_fifo_get(struct rte_kni_fifo *fifo, void **data, unsigned num) { unsigned i = 0; unsigned new_read = fifo->read; - rte_smp_rmb(); - unsigned fifo_write = fifo->write; + unsigned fifo_write = __KNI_LOAD_ACQUIRE(&fifo->write); for (i = 0; i < num; i++) { if (new_read == fifo_write) @@ -63,8 +90,7 @@ kni_fifo_get(struct rte_kni_fifo *fifo, void **data, unsigned num) data[i] = fifo->buffer[new_read]; new_read = (new_read + 1) & (fifo->len - 1); } - rte_smp_rmb(); - fifo->read = new_read; + __KNI_STORE_RELEASE(&fifo->read, new_read); return i; } @@ -74,8 +100,7 @@ kni_fifo_get(struct rte_kni_fifo *fifo, void **data, unsigned num) static inline uint32_t kni_fifo_count(struct rte_kni_fifo *fifo) { - unsigned fifo_write = fifo->write; - rte_smp_rmb(); - unsigned fifo_read = fifo->read; + unsigned fifo_write = __KNI_LOAD_ACQUIRE(&fifo->write); + unsigned fifo_read = __KNI_LOAD_ACQUIRE(&fifo->read); return (fifo->len + fifo_write - fifo_read) & (fifo->len - 1); }