git-lng-odp July 2018

git-lng-odp@lists.linaro.org

1 participants
21 discussions

[COMMIT] branch master updated. v1.19.0.1_tigermoth-115-g838c1365

by git＠git-us.linaro.org

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "". The branch, master has been updated via 838c13654c639baa1c54d19056cd771380ab8a58 (commit) from abd50139e35c67ddbc1ec09550ea6b6d27861703 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit 838c13654c639baa1c54d19056cd771380ab8a58 Author: Petri Savolainen <petri.savolainen(a)linaro.org> Date: Fri Jun 29 10:59:30 2018 +0300 linux-gen: ring: ensure head and tail load order in dequeue Acquire memory order is needed when loading r_head. It ensures that load of w_tail cannot get ahead of r_head load, and thus head value cannot get ahead of tail value. Queue empty check assumes that head is always behind or equal to tail (== ring empty). Signed-off-by: Petri Savolainen <petri.savolainen(a)linaro.org> Reviewed-by: Bill Fischofer <bill.fischofer(a)linaro.org> Signed-off-by: Maxim Uvarov <maxim.uvarov(a)linaro.org> diff --git a/platform/linux-generic/include/odp_ring_internal.h b/platform/linux-generic/include/odp_ring_internal.h index 03c0817e..130d74cc 100644 --- a/platform/linux-generic/include/odp_ring_internal.h +++ b/platform/linux-generic/include/odp_ring_internal.h @@ -41,6 +41,16 @@ typedef struct ODP_ALIGNED_CACHE { uint32_t data[0]; } ring_t; +/* 32-bit CAS with memory order selection */ +static inline int cas_mo_u32(odp_atomic_u32_t *atom, uint32_t *old_val, + uint32_t new_val, int mo_success, int mo_failure) +{ + return __atomic_compare_exchange_n(&atom->v, old_val, new_val, + 0 /* strong */, + mo_success, + mo_failure); +} + /* Initialize ring */ static inline void ring_init(ring_t *ring) { @@ -56,7 +66,10 @@ static inline uint32_t ring_deq(ring_t *ring, uint32_t mask) uint32_t head, tail, new_head; uint32_t data; - head = odp_atomic_load_u32(&ring->r_head); + /* Load/CAS acquire of r_head ensures that w_tail load happens after + * r_head load, and thus head value is always behind or equal to tail + * value. */ + head = odp_atomic_load_acq_u32(&ring->r_head); /* Move reader head. This thread owns data at the new head. */ do { @@ -67,8 +80,9 @@ static inline uint32_t ring_deq(ring_t *ring, uint32_t mask) new_head = head + 1; - } while (odp_unlikely(odp_atomic_cas_acq_u32(&ring->r_head, &head, - new_head) == 0)); + } while (odp_unlikely(cas_mo_u32(&ring->r_head, &head, new_head, + __ATOMIC_ACQUIRE, + __ATOMIC_ACQUIRE) == 0)); /* Read queue index */ data = ring->data[new_head & mask]; @@ -89,7 +103,10 @@ static inline uint32_t ring_deq_multi(ring_t *ring, uint32_t mask, { uint32_t head, tail, new_head, i; - head = odp_atomic_load_u32(&ring->r_head); + /* Load/CAS acquire of r_head ensures that w_tail load happens after + * r_head load, and thus head value is always behind or equal to tail + * value. */ + head = odp_atomic_load_acq_u32(&ring->r_head); /* Move reader head. This thread owns data at the new head. */ do { @@ -105,8 +122,9 @@ static inline uint32_t ring_deq_multi(ring_t *ring, uint32_t mask, new_head = head + num; - } while (odp_unlikely(odp_atomic_cas_acq_u32(&ring->r_head, &head, - new_head) == 0)); + } while (odp_unlikely(cas_mo_u32(&ring->r_head, &head, new_head, + __ATOMIC_ACQUIRE, + __ATOMIC_ACQUIRE) == 0)); /* Read queue index */ for (i = 0; i < num; i++) ----------------------------------------------------------------------- Summary of changes: platform/linux-generic/include/odp_ring_internal.h | 30 +++++++++++++++++----- 1 file changed, 24 insertions(+), 6 deletions(-) hooks/post-receive --

6 years, 10 months

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

git-lng-odp July 2018