<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
"http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head><meta http-equiv="content-type" content="text/html; charset=utf-8" />
<title>[205921] trunk/Source/WTF</title>
</head>
<body>
<style type="text/css"><!--
#msg dl.meta { border: 1px #006 solid; background: #369; padding: 6px; color: #fff; }
#msg dl.meta dt { float: left; width: 6em; font-weight: bold; }
#msg dt:after { content:':';}
#msg dl, #msg dt, #msg ul, #msg li, #header, #footer, #logmsg { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt; }
#msg dl a { font-weight: bold}
#msg dl a:link { color:#fc3; }
#msg dl a:active { color:#ff0; }
#msg dl a:visited { color:#cc6; }
h3 { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt; font-weight: bold; }
#msg pre { overflow: auto; background: #ffc; border: 1px #fa0 solid; padding: 6px; }
#logmsg { background: #ffc; border: 1px #fa0 solid; padding: 1em 1em 0 1em; }
#logmsg p, #logmsg pre, #logmsg blockquote { margin: 0 0 1em 0; }
#logmsg p, #logmsg li, #logmsg dt, #logmsg dd { line-height: 14pt; }
#logmsg h1, #logmsg h2, #logmsg h3, #logmsg h4, #logmsg h5, #logmsg h6 { margin: .5em 0; }
#logmsg h1:first-child, #logmsg h2:first-child, #logmsg h3:first-child, #logmsg h4:first-child, #logmsg h5:first-child, #logmsg h6:first-child { margin-top: 0; }
#logmsg ul, #logmsg ol { padding: 0; list-style-position: inside; margin: 0 0 0 1em; }
#logmsg ul { text-indent: -1em; padding-left: 1em; }#logmsg ol { text-indent: -1.5em; padding-left: 1.5em; }
#logmsg > ul, #logmsg > ol { margin: 0 0 1em 0; }
#logmsg pre { background: #eee; padding: 1em; }
#logmsg blockquote { border: 1px solid #fa0; border-left-width: 10px; padding: 1em 1em 0 1em; background: white;}
#logmsg dl { margin: 0; }
#logmsg dt { font-weight: bold; }
#logmsg dd { margin: 0; padding: 0 0 0.5em 0; }
#logmsg dd:before { content:'\00bb';}
#logmsg table { border-spacing: 0px; border-collapse: collapse; border-top: 4px solid #fa0; border-bottom: 1px solid #fa0; background: #fff; }
#logmsg table th { text-align: left; font-weight: normal; padding: 0.2em 0.5em; border-top: 1px dotted #fa0; }
#logmsg table td { text-align: right; border-top: 1px dotted #fa0; padding: 0.2em 0.5em; }
#logmsg table thead th { text-align: center; border-bottom: 1px solid #fa0; }
#logmsg table th.Corner { text-align: left; }
#logmsg hr { border: none 0; border-top: 2px dashed #fa0; height: 1px; }
#header, #footer { color: #fff; background: #636; border: 1px #300 solid; padding: 6px; }
#patch { width: 100%; }
#patch h4 {font-family: verdana,arial,helvetica,sans-serif;font-size:10pt;padding:8px;background:#369;color:#fff;margin:0;}
#patch .propset h4, #patch .binary h4 {margin:0;}
#patch pre {padding:0;line-height:1.2em;margin:0;}
#patch .diff {width:100%;background:#eee;padding: 0 0 10px 0;overflow:auto;}
#patch .propset .diff, #patch .binary .diff {padding:10px 0;}
#patch span {display:block;padding:0 10px;}
#patch .modfile, #patch .addfile, #patch .delfile, #patch .propset, #patch .binary, #patch .copfile {border:1px solid #ccc;margin:10px 0;}
#patch ins {background:#dfd;text-decoration:none;display:block;padding:0 10px;}
#patch del {background:#fdd;text-decoration:none;display:block;padding:0 10px;}
#patch .lines, .info {color:#888;background:#fff;}
--></style>
<div id="msg">
<dl class="meta">
<dt>Revision</dt> <dd><a href="http://trac.webkit.org/projects/webkit/changeset/205921">205921</a></dd>
<dt>Author</dt> <dd>jfbastien@apple.com</dd>
<dt>Date</dt> <dd>2016-09-14 11:51:26 -0700 (Wed, 14 Sep 2016)</dd>
</dl>
<h3>Log Message</h3>
<pre>Atomics on ARM don't require full-system fencing, and other minutiae
https://bugs.webkit.org/show_bug.cgi?id=161928
Reviewed by Geoffrey Garen.
Add cmpxchg versions with both success and failure memory
ordering. In some interesting cases we can craft code which needs
barriers which aren't as strong.
weakCompareAndSwap is super dubious, its 3 uses seem
questionable... but for now I'm just adding debug asserts.
Rename armv7_dmb* functions to arm_dmb* because they apply to v7
and v8 (or more precisely; to ARMv7's ARM and Thumb2, as well as
ARMv8's aarch32 A32/T32 and aarch64).
Use inner-shareability domain for ARM barriers instead of
full-system. This is what C++ uses.
The default case for barriers simply used a compiler barrier. This
is generally wrong, e.g. for MIPS.
* wtf/Atomics.h:
(WTF::Atomic::compareExchangeWeak): offer two-order version
(WTF::Atomic::compareExchangeStrong): offer two-order version
(WTF::weakCompareAndSwap): a few assertions
(WTF::arm_dmb): rename since it applies to ARMv7 and v8; make it innser-shareable
(WTF::arm_dmb_st): rename since it applies to ARMv7 and v8; make it innser-shareable
(WTF::loadLoadFence): incorrect generally
(WTF::loadStoreFence): incorrect generally
(WTF::storeLoadFence): incorrect generally
(WTF::storeStoreFence): incorrect generally
(WTF::memoryBarrierAfterLock): incorrect generally
(WTF::memoryBarrierBeforeUnlock): incorrect generally
(WTF::armV7_dmb): Deleted.
(WTF::armV7_dmb_st): Deleted.</pre>
<h3>Modified Paths</h3>
<ul>
<li><a href="#trunkSourceWTFChangeLog">trunk/Source/WTF/ChangeLog</a></li>
<li><a href="#trunkSourceWTFwtfAtomicsh">trunk/Source/WTF/wtf/Atomics.h</a></li>
</ul>
</div>
<div id="patch">
<h3>Diff</h3>
<a id="trunkSourceWTFChangeLog"></a>
<div class="modfile"><h4>Modified: trunk/Source/WTF/ChangeLog (205920 => 205921)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/WTF/ChangeLog 2016-09-14 18:46:32 UTC (rev 205920)
+++ trunk/Source/WTF/ChangeLog 2016-09-14 18:51:26 UTC (rev 205921)
</span><span class="lines">@@ -1,5 +1,44 @@
</span><span class="cx"> 2016-09-14 JF Bastien <jfbastien@apple.com>
</span><span class="cx">
</span><ins>+ Atomics on ARM don't require full-system fencing, and other minutiae
+ https://bugs.webkit.org/show_bug.cgi?id=161928
+
+ Reviewed by Geoffrey Garen.
+
+ Add cmpxchg versions with both success and failure memory
+ ordering. In some interesting cases we can craft code which needs
+ barriers which aren't as strong.
+
+ weakCompareAndSwap is super dubious, its 3 uses seem
+ questionable... but for now I'm just adding debug asserts.
+
+ Rename armv7_dmb* functions to arm_dmb* because they apply to v7
+ and v8 (or more precisely; to ARMv7's ARM and Thumb2, as well as
+ ARMv8's aarch32 A32/T32 and aarch64).
+
+ Use inner-shareability domain for ARM barriers instead of
+ full-system. This is what C++ uses.
+
+ The default case for barriers simply used a compiler barrier. This
+ is generally wrong, e.g. for MIPS.
+
+ * wtf/Atomics.h:
+ (WTF::Atomic::compareExchangeWeak): offer two-order version
+ (WTF::Atomic::compareExchangeStrong): offer two-order version
+ (WTF::weakCompareAndSwap): a few assertions
+ (WTF::arm_dmb): rename since it applies to ARMv7 and v8; make it innser-shareable
+ (WTF::arm_dmb_st): rename since it applies to ARMv7 and v8; make it innser-shareable
+ (WTF::loadLoadFence): incorrect generally
+ (WTF::loadStoreFence): incorrect generally
+ (WTF::storeLoadFence): incorrect generally
+ (WTF::storeStoreFence): incorrect generally
+ (WTF::memoryBarrierAfterLock): incorrect generally
+ (WTF::memoryBarrierBeforeUnlock): incorrect generally
+ (WTF::armV7_dmb): Deleted.
+ (WTF::armV7_dmb_st): Deleted.
+
+2016-09-14 JF Bastien <jfbastien@apple.com>
+
</ins><span class="cx"> Alwasys inline atomic operations
</span><span class="cx"> https://bugs.webkit.org/show_bug.cgi?id=155371
</span><span class="cx">
</span></span></pre></div>
<a id="trunkSourceWTFwtfAtomicsh"></a>
<div class="modfile"><h4>Modified: trunk/Source/WTF/wtf/Atomics.h (205920 => 205921)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/WTF/wtf/Atomics.h 2016-09-14 18:46:32 UTC (rev 205920)
+++ trunk/Source/WTF/wtf/Atomics.h 2016-09-14 18:51:26 UTC (rev 205921)
</span><span class="lines">@@ -67,6 +67,18 @@
</span><span class="cx"> return value.compare_exchange_weak(expectedOrActual, desired, order);
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+ ALWAYS_INLINE bool compareExchangeWeak(T expected, T desired, std::memory_order order_success, std::memory_order order_failure)
+ {
+#if OS(WINDOWS)
+ // Windows makes strange assertions about the argument to compare_exchange_weak, and anyway,
+ // Windows is X86 so seq_cst is cheap.
+ order_success = std::memory_order_seq_cst;
+ order_failure = std::memory_order_seq_cst;
+#endif
+ T expectedOrActual = expected;
+ return value.compare_exchange_weak(expectedOrActual, desired, order_success, order_failure);
+ }
+
</ins><span class="cx"> ALWAYS_INLINE bool compareExchangeStrong(T expected, T desired, std::memory_order order = std::memory_order_seq_cst)
</span><span class="cx"> {
</span><span class="cx"> #if OS(WINDOWS)
</span><span class="lines">@@ -76,7 +88,18 @@
</span><span class="cx"> T expectedOrActual = expected;
</span><span class="cx"> return value.compare_exchange_strong(expectedOrActual, desired, order);
</span><span class="cx"> }
</span><del>-
</del><ins>+
+ ALWAYS_INLINE bool compareExchangeStrong(T expected, T desired, std::memory_order order_success, std::memory_order order_failure)
+ {
+#if OS(WINDOWS)
+ // See above.
+ order_success = std::memory_order_seq_cst;
+ order_failure = std::memory_order_seq_cst;
+#endif
+ T expectedOrActual = expected;
+ return value.compare_exchange_strong(expectedOrActual, desired, order_success, order_failure);
+ }
+
</ins><span class="cx"> template<typename U>
</span><span class="cx"> ALWAYS_INLINE T exchangeAndAdd(U addend, std::memory_order order = std::memory_order_seq_cst)
</span><span class="cx"> {
</span><span class="lines">@@ -103,6 +126,8 @@
</span><span class="cx"> template<typename T>
</span><span class="cx"> inline bool weakCompareAndSwap(volatile T* location, T expected, T newValue)
</span><span class="cx"> {
</span><ins>+ ASSERT(isPointerTypeAlignmentOkay(location) && "natural alignment required");
+ ASSERT(bitwise_cast<std::atomic<T>*>(location)->is_lock_free() && "expected lock-free type");
</ins><span class="cx"> return bitwise_cast<Atomic<T>*>(location)->compareExchangeWeak(expected, newValue, std::memory_order_relaxed);
</span><span class="cx"> }
</span><span class="cx">
</span><span class="lines">@@ -122,23 +147,23 @@
</span><span class="cx">
</span><span class="cx"> // Full memory fence. No accesses will float above this, and no accesses will sink
</span><span class="cx"> // below it.
</span><del>-inline void armV7_dmb()
</del><ins>+inline void arm_dmb()
</ins><span class="cx"> {
</span><del>- asm volatile("dmb sy" ::: "memory");
</del><ins>+ asm volatile("dmb ish" ::: "memory");
</ins><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> // Like the above, but only affects stores.
</span><del>-inline void armV7_dmb_st()
</del><ins>+inline void arm_dmb_st()
</ins><span class="cx"> {
</span><del>- asm volatile("dmb st" ::: "memory");
</del><ins>+ asm volatile("dmb ishst" ::: "memory");
</ins><span class="cx"> }
</span><span class="cx">
</span><del>-inline void loadLoadFence() { armV7_dmb(); }
-inline void loadStoreFence() { armV7_dmb(); }
-inline void storeLoadFence() { armV7_dmb(); }
-inline void storeStoreFence() { armV7_dmb_st(); }
-inline void memoryBarrierAfterLock() { armV7_dmb(); }
-inline void memoryBarrierBeforeUnlock() { armV7_dmb(); }
</del><ins>+inline void loadLoadFence() { arm_dmb(); }
+inline void loadStoreFence() { arm_dmb(); }
+inline void storeLoadFence() { arm_dmb(); }
+inline void storeStoreFence() { arm_dmb_st(); }
+inline void memoryBarrierAfterLock() { arm_dmb(); }
+inline void memoryBarrierBeforeUnlock() { arm_dmb(); }
</ins><span class="cx">
</span><span class="cx"> #elif CPU(X86) || CPU(X86_64)
</span><span class="cx">
</span><span class="lines">@@ -164,12 +189,12 @@
</span><span class="cx">
</span><span class="cx"> #else
</span><span class="cx">
</span><del>-inline void loadLoadFence() { compilerFence(); }
-inline void loadStoreFence() { compilerFence(); }
-inline void storeLoadFence() { compilerFence(); }
-inline void storeStoreFence() { compilerFence(); }
-inline void memoryBarrierAfterLock() { compilerFence(); }
-inline void memoryBarrierBeforeUnlock() { compilerFence(); }
</del><ins>+inline void loadLoadFence() { std::atomic_thread_fence(std::memory_order_seq_cst); }
+inline void loadStoreFence() { std::atomic_thread_fence(std::memory_order_seq_cst); }
+inline void storeLoadFence() { std::atomic_thread_fence(std::memory_order_seq_cst); }
+inline void storeStoreFence() { std::atomic_thread_fence(std::memory_order_seq_cst); }
+inline void memoryBarrierAfterLock() { std::atomic_thread_fence(std::memory_order_seq_cst); }
+inline void memoryBarrierBeforeUnlock() { std::atomic_thread_fence(std::memory_order_seq_cst); }
</ins><span class="cx">
</span><span class="cx"> #endif
</span><span class="cx">
</span></span></pre>
</div>
</div>
</body>
</html>