<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
"http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head><meta http-equiv="content-type" content="text/html; charset=utf-8" />
<title>[206555] trunk/Source</title>
</head>
<body>
<style type="text/css"><!--
#msg dl.meta { border: 1px #006 solid; background: #369; padding: 6px; color: #fff; }
#msg dl.meta dt { float: left; width: 6em; font-weight: bold; }
#msg dt:after { content:':';}
#msg dl, #msg dt, #msg ul, #msg li, #header, #footer, #logmsg { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt; }
#msg dl a { font-weight: bold}
#msg dl a:link { color:#fc3; }
#msg dl a:active { color:#ff0; }
#msg dl a:visited { color:#cc6; }
h3 { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt; font-weight: bold; }
#msg pre { overflow: auto; background: #ffc; border: 1px #fa0 solid; padding: 6px; }
#logmsg { background: #ffc; border: 1px #fa0 solid; padding: 1em 1em 0 1em; }
#logmsg p, #logmsg pre, #logmsg blockquote { margin: 0 0 1em 0; }
#logmsg p, #logmsg li, #logmsg dt, #logmsg dd { line-height: 14pt; }
#logmsg h1, #logmsg h2, #logmsg h3, #logmsg h4, #logmsg h5, #logmsg h6 { margin: .5em 0; }
#logmsg h1:first-child, #logmsg h2:first-child, #logmsg h3:first-child, #logmsg h4:first-child, #logmsg h5:first-child, #logmsg h6:first-child { margin-top: 0; }
#logmsg ul, #logmsg ol { padding: 0; list-style-position: inside; margin: 0 0 0 1em; }
#logmsg ul { text-indent: -1em; padding-left: 1em; }#logmsg ol { text-indent: -1.5em; padding-left: 1.5em; }
#logmsg > ul, #logmsg > ol { margin: 0 0 1em 0; }
#logmsg pre { background: #eee; padding: 1em; }
#logmsg blockquote { border: 1px solid #fa0; border-left-width: 10px; padding: 1em 1em 0 1em; background: white;}
#logmsg dl { margin: 0; }
#logmsg dt { font-weight: bold; }
#logmsg dd { margin: 0; padding: 0 0 0.5em 0; }
#logmsg dd:before { content:'\00bb';}
#logmsg table { border-spacing: 0px; border-collapse: collapse; border-top: 4px solid #fa0; border-bottom: 1px solid #fa0; background: #fff; }
#logmsg table th { text-align: left; font-weight: normal; padding: 0.2em 0.5em; border-top: 1px dotted #fa0; }
#logmsg table td { text-align: right; border-top: 1px dotted #fa0; padding: 0.2em 0.5em; }
#logmsg table thead th { text-align: center; border-bottom: 1px solid #fa0; }
#logmsg table th.Corner { text-align: left; }
#logmsg hr { border: none 0; border-top: 2px dashed #fa0; height: 1px; }
#header, #footer { color: #fff; background: #636; border: 1px #300 solid; padding: 6px; }
#patch { width: 100%; }
#patch h4 {font-family: verdana,arial,helvetica,sans-serif;font-size:10pt;padding:8px;background:#369;color:#fff;margin:0;}
#patch .propset h4, #patch .binary h4 {margin:0;}
#patch pre {padding:0;line-height:1.2em;margin:0;}
#patch .diff {width:100%;background:#eee;padding: 0 0 10px 0;overflow:auto;}
#patch .propset .diff, #patch .binary .diff {padding:10px 0;}
#patch span {display:block;padding:0 10px;}
#patch .modfile, #patch .addfile, #patch .delfile, #patch .propset, #patch .binary, #patch .copfile {border:1px solid #ccc;margin:10px 0;}
#patch ins {background:#dfd;text-decoration:none;display:block;padding:0 10px;}
#patch del {background:#fdd;text-decoration:none;display:block;padding:0 10px;}
#patch .lines, .info {color:#888;background:#fff;}
--></style>
<div id="msg">
<dl class="meta">
<dt>Revision</dt> <dd><a href="http://trac.webkit.org/projects/webkit/changeset/206555">206555</a></dd>
<dt>Author</dt> <dd>fpizlo@apple.com</dd>
<dt>Date</dt> <dd>2016-09-28 14:55:53 -0700 (Wed, 28 Sep 2016)</dd>
</dl>
<h3>Log Message</h3>
<pre>The write barrier should be down with TSO
https://bugs.webkit.org/show_bug.cgi?id=162316
Reviewed by Geoffrey Garen.
Source/JavaScriptCore:
This makes our write barrier behave correctly when it races with the collector. The
collector wants to do this when visiting:
object->cellState = black
visit(object)
The mutator wants to do this when storing:
object->property = newValue
if (object->cellState == black)
remember(object)
Prior to this change, this didn't work right because the compiler would sometimes place
barriers before the store to the property and because the mutator did not have adequate
fences.
Prior to this change, the DFG and FTL would emit this:
if (object->cellState == black)
remember(object)
object->property = newValue
Which is wrong, because the object could start being scanned just after the cellState
check, at which point the store would be lost. We need to confirm that the state was not
black *after* the store! This change was harder than you'd expect: placing the barrier
after the store broke B3's ability to do its super crazy ninja CSE on some store-load
redundancies. Because the B3 CSE has some moves that the DFG CSE lacks, the DFG CSE's
ability to ignore barriers didn't help. I fixed this by having the FTL convey precise
heap ranges for the patchpoint corresponding to the barrier slow path. It reads the world
(because of the store-load fence) and it writes only cellState (because the B3 heap ranges
don't have any way to represent any of the GC's other state, which means that B3 does not
have to worry about aliasing with any of that).
The collector already uses a store-load fence on x86 just after setting the cellState and
before visiting the object. The mutator needs to do the same. But we cannot put a
store-load fence of any kind before store barriers, because that causes enormous slow
downs. In the worst case, Octane/richards slowed down by 90%! That's crazy! However, the
overall slow downs were small enough (0-15% on benchmark suite aggregates) that it would be
reasonable if the slow down only happened while the GC was running. Then, the concurrent GC
would lift throughput-while-collecting from 0% of peak to 85% of peak. This changes the
barrier so that it looks like this:
if (object->cellState <= heap.sneakyBlackThreshold)
slowPath(object)
Where sneakyBlackThreshold is the normal blackThreshold when we're not collecting, or a
tautoligical threshold (that makes everything look black) when we are collecting. This
turns out to not be any more expensive than the barrier in tip of tree when the GC is not
running, or a 0-15% slow-down when it is "running". (Of course we don't run the GC
concurrently yet. I still have more work to do.) The slowPath() does some extra work to
check if we are concurrently collecting; if so, it does a fence and rechecks if the object
really did need that barrier.
This also reintroduces elimination of redundant store barriers, which was lost in the last
store barrier change. We can only do it when there is no possibility of GC, exit, or
exceptions between the two store barriers. We could remove the exit/exception limitation if
we taught OSR exit how to buffer store barriers, which is an insane thing to do considering
that I've never been able to detect a win from redundant store barrier elimination. I just
want us to have it for stupidly obvious situations, like a tight sequence of stores to the
same object. This same optimization also sometimes strength-reduces the store barrier so
that it uses a constant black threshold rather than the sneaky one, thereby saving one
load.
Even with all of those optimizations, I still had problems with barrier cost. I found that one
of the benchmarks that was being hit particularly hard was JetStream/regexp-2010. Fortunately
that benchmark does most of its barriers in a tight C++ loop in RegExpMatchesArray.h. When we
know what we're doing, we can defer GC around a bunch of object initializations and then remove
all of the barriers between any of the objects allocated within the deferral. Unfortunately,
our GC deferral mechanism isn't really performant enough to make this be a worthwhile
optimization. The most efficient version of such an optimization that I could come up with was
to have a DeferralContext object that houses a boolean that is false by default, but the GC
writes true into it if it would have wanted to GC. You thread a pointer to the deferralContext
through all of your allocations. This kind of mechanism has the overhead of a zero
initialization on the stack on entry and a zero check on exit. This is probably even efficient
enough that we could start thinking about having the DFG use it, for example if we found a
bounded-time section of code with a lot of barriers and entry/exit sites that aren't totally
wacky. This optimization took this patch from 0.68% JetStream regressed to neutral, according
to my latest data.
Finally, an earlier version of this change put the store-load fence in B3 IR, so I ended up
adding FTLOutput support for it and AbstractHeapRepository magic for decorating the heaps.
I think we might as well keep that, it'll be useful.
* CMakeLists.txt:
* JavaScriptCore.xcodeproj/project.pbxproj:
* assembler/MacroAssembler.h:
(JSC::MacroAssembler::branch32):
* assembler/MacroAssemblerX86_64.h:
(JSC::MacroAssemblerX86_64::branch32):
(JSC::MacroAssemblerX86_64::branch64): Deleted.
* bytecode/PolymorphicAccess.cpp:
(JSC::AccessCase::generateImpl):
* dfg/DFGAbstractHeap.h:
* dfg/DFGAbstractInterpreterInlines.h:
(JSC::DFG::AbstractInterpreter<AbstractStateType>::executeEffects):
* dfg/DFGClobberize.h:
(JSC::DFG::clobberize):
* dfg/DFGClobbersExitState.cpp:
(JSC::DFG::clobbersExitState):
* dfg/DFGDoesGC.cpp:
(JSC::DFG::doesGC):
* dfg/DFGFixupPhase.cpp:
(JSC::DFG::FixupPhase::fixupNode):
* dfg/DFGMayExit.cpp:
* dfg/DFGNode.h:
(JSC::DFG::Node::isStoreBarrier):
* dfg/DFGNodeType.h:
* dfg/DFGPlan.cpp:
(JSC::DFG::Plan::compileInThreadImpl):
* dfg/DFGPredictionPropagationPhase.cpp:
* dfg/DFGSafeToExecute.h:
(JSC::DFG::safeToExecute):
* dfg/DFGSpeculativeJIT.cpp:
(JSC::DFG::SpeculativeJIT::compileStoreBarrier):
(JSC::DFG::SpeculativeJIT::storeToWriteBarrierBuffer): Deleted.
(JSC::DFG::SpeculativeJIT::writeBarrier): Deleted.
* dfg/DFGSpeculativeJIT.h:
* dfg/DFGSpeculativeJIT32_64.cpp:
(JSC::DFG::SpeculativeJIT::compile):
(JSC::DFG::SpeculativeJIT::compileBaseValueStoreBarrier): Deleted.
(JSC::DFG::SpeculativeJIT::writeBarrier): Deleted.
* dfg/DFGSpeculativeJIT64.cpp:
(JSC::DFG::SpeculativeJIT::compile):
(JSC::DFG::SpeculativeJIT::compileBaseValueStoreBarrier): Deleted.
(JSC::DFG::SpeculativeJIT::writeBarrier): Deleted.
* dfg/DFGStoreBarrierClusteringPhase.cpp: Added.
(JSC::DFG::performStoreBarrierClustering):
* dfg/DFGStoreBarrierClusteringPhase.h: Added.
* dfg/DFGStoreBarrierInsertionPhase.cpp:
* dfg/DFGStoreBarrierInsertionPhase.h:
* ftl/FTLAbstractHeap.h:
(JSC::FTL::AbsoluteAbstractHeap::at):
(JSC::FTL::AbsoluteAbstractHeap::operator[]):
* ftl/FTLAbstractHeapRepository.cpp:
(JSC::FTL::AbstractHeapRepository::decorateFenceRead):
(JSC::FTL::AbstractHeapRepository::decorateFenceWrite):
(JSC::FTL::AbstractHeapRepository::computeRangesAndDecorateInstructions):
* ftl/FTLAbstractHeapRepository.h:
* ftl/FTLCapabilities.cpp:
(JSC::FTL::canCompile):
* ftl/FTLLowerDFGToB3.cpp:
(JSC::FTL::DFG::LowerDFGToB3::compileNode):
(JSC::FTL::DFG::LowerDFGToB3::compileStoreBarrier):
(JSC::FTL::DFG::LowerDFGToB3::storageForTransition):
(JSC::FTL::DFG::LowerDFGToB3::lazySlowPath):
(JSC::FTL::DFG::LowerDFGToB3::emitStoreBarrier):
* ftl/FTLOutput.cpp:
(JSC::FTL::Output::fence):
(JSC::FTL::Output::absolute):
* ftl/FTLOutput.h:
* heap/CellState.h:
(JSC::isWithinThreshold):
(JSC::isBlack):
* heap/Heap.cpp:
(JSC::Heap::writeBarrierSlowPath):
* heap/Heap.h:
(JSC::Heap::barrierShouldBeFenced):
(JSC::Heap::addressOfBarrierShouldBeFenced):
(JSC::Heap::sneakyBlackThreshold):
(JSC::Heap::addressOfSneakyBlackThreshold):
* heap/HeapInlines.h:
(JSC::Heap::writeBarrier):
(JSC::Heap::writeBarrierWithoutFence):
* jit/AssemblyHelpers.h:
(JSC::AssemblyHelpers::jumpIfIsRememberedOrInEdenWithoutFence):
(JSC::AssemblyHelpers::sneakyJumpIfIsRememberedOrInEden):
(JSC::AssemblyHelpers::jumpIfIsRememberedOrInEden):
(JSC::AssemblyHelpers::storeBarrierStoreLoadFence):
(JSC::AssemblyHelpers::jumpIfStoreBarrierStoreLoadFenceNotNeeded):
* jit/JITOperations.cpp:
* jit/JITOperations.h:
* jit/JITPropertyAccess.cpp:
(JSC::JIT::emit_op_put_by_id):
(JSC::JIT::emitWriteBarrier):
(JSC::JIT::privateCompilePutByVal):
* jit/JITPropertyAccess32_64.cpp:
(JSC::JIT::emit_op_put_by_id):
* llint/LowLevelInterpreter.asm:
* offlineasm/x86.rb:
* runtime/Options.h:
Source/WTF:
Added clearRange(), which quickly clears a range of bits. This turned out to be useful for
a DFG optimization pass.
* wtf/FastBitVector.cpp:
(WTF::FastBitVector::clearRange):
* wtf/FastBitVector.h:</pre>
<h3>Modified Paths</h3>
<ul>
<li><a href="#trunkSourceJavaScriptCoreCMakeListstxt">trunk/Source/JavaScriptCore/CMakeLists.txt</a></li>
<li><a href="#trunkSourceJavaScriptCoreChangeLog">trunk/Source/JavaScriptCore/ChangeLog</a></li>
<li><a href="#trunkSourceJavaScriptCoreJavaScriptCorexcodeprojprojectpbxproj">trunk/Source/JavaScriptCore/JavaScriptCore.xcodeproj/project.pbxproj</a></li>
<li><a href="#trunkSourceJavaScriptCoreassemblerMacroAssemblerh">trunk/Source/JavaScriptCore/assembler/MacroAssembler.h</a></li>
<li><a href="#trunkSourceJavaScriptCoreassemblerMacroAssemblerX86_64h">trunk/Source/JavaScriptCore/assembler/MacroAssemblerX86_64.h</a></li>
<li><a href="#trunkSourceJavaScriptCorebytecodePolymorphicAccesscpp">trunk/Source/JavaScriptCore/bytecode/PolymorphicAccess.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoredfgDFGAbstractHeaph">trunk/Source/JavaScriptCore/dfg/DFGAbstractHeap.h</a></li>
<li><a href="#trunkSourceJavaScriptCoredfgDFGAbstractInterpreterInlinesh">trunk/Source/JavaScriptCore/dfg/DFGAbstractInterpreterInlines.h</a></li>
<li><a href="#trunkSourceJavaScriptCoredfgDFGClobberizeh">trunk/Source/JavaScriptCore/dfg/DFGClobberize.h</a></li>
<li><a href="#trunkSourceJavaScriptCoredfgDFGClobbersExitStatecpp">trunk/Source/JavaScriptCore/dfg/DFGClobbersExitState.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoredfgDFGDoesGCcpp">trunk/Source/JavaScriptCore/dfg/DFGDoesGC.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoredfgDFGFixupPhasecpp">trunk/Source/JavaScriptCore/dfg/DFGFixupPhase.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoredfgDFGMayExitcpp">trunk/Source/JavaScriptCore/dfg/DFGMayExit.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoredfgDFGNodeh">trunk/Source/JavaScriptCore/dfg/DFGNode.h</a></li>
<li><a href="#trunkSourceJavaScriptCoredfgDFGNodeTypeh">trunk/Source/JavaScriptCore/dfg/DFGNodeType.h</a></li>
<li><a href="#trunkSourceJavaScriptCoredfgDFGOSRExitCompilerCommoncpp">trunk/Source/JavaScriptCore/dfg/DFGOSRExitCompilerCommon.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoredfgDFGOperationscpp">trunk/Source/JavaScriptCore/dfg/DFGOperations.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoredfgDFGPlancpp">trunk/Source/JavaScriptCore/dfg/DFGPlan.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoredfgDFGPredictionPropagationPhasecpp">trunk/Source/JavaScriptCore/dfg/DFGPredictionPropagationPhase.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoredfgDFGSafeToExecuteh">trunk/Source/JavaScriptCore/dfg/DFGSafeToExecute.h</a></li>
<li><a href="#trunkSourceJavaScriptCoredfgDFGSpeculativeJITcpp">trunk/Source/JavaScriptCore/dfg/DFGSpeculativeJIT.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoredfgDFGSpeculativeJITh">trunk/Source/JavaScriptCore/dfg/DFGSpeculativeJIT.h</a></li>
<li><a href="#trunkSourceJavaScriptCoredfgDFGSpeculativeJIT32_64cpp">trunk/Source/JavaScriptCore/dfg/DFGSpeculativeJIT32_64.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoredfgDFGSpeculativeJIT64cpp">trunk/Source/JavaScriptCore/dfg/DFGSpeculativeJIT64.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoredfgDFGStoreBarrierInsertionPhasecpp">trunk/Source/JavaScriptCore/dfg/DFGStoreBarrierInsertionPhase.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoredfgDFGStoreBarrierInsertionPhaseh">trunk/Source/JavaScriptCore/dfg/DFGStoreBarrierInsertionPhase.h</a></li>
<li><a href="#trunkSourceJavaScriptCoreftlFTLAbstractHeaph">trunk/Source/JavaScriptCore/ftl/FTLAbstractHeap.h</a></li>
<li><a href="#trunkSourceJavaScriptCoreftlFTLAbstractHeapRepositorycpp">trunk/Source/JavaScriptCore/ftl/FTLAbstractHeapRepository.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoreftlFTLAbstractHeapRepositoryh">trunk/Source/JavaScriptCore/ftl/FTLAbstractHeapRepository.h</a></li>
<li><a href="#trunkSourceJavaScriptCoreftlFTLCapabilitiescpp">trunk/Source/JavaScriptCore/ftl/FTLCapabilities.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoreftlFTLLowerDFGToB3cpp">trunk/Source/JavaScriptCore/ftl/FTLLowerDFGToB3.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoreftlFTLOutputcpp">trunk/Source/JavaScriptCore/ftl/FTLOutput.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoreftlFTLOutputh">trunk/Source/JavaScriptCore/ftl/FTLOutput.h</a></li>
<li><a href="#trunkSourceJavaScriptCoreheapCellStateh">trunk/Source/JavaScriptCore/heap/CellState.h</a></li>
<li><a href="#trunkSourceJavaScriptCoreheapHeapcpp">trunk/Source/JavaScriptCore/heap/Heap.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoreheapHeaph">trunk/Source/JavaScriptCore/heap/Heap.h</a></li>
<li><a href="#trunkSourceJavaScriptCoreheapHeapInlinesh">trunk/Source/JavaScriptCore/heap/HeapInlines.h</a></li>
<li><a href="#trunkSourceJavaScriptCoreheapMarkedAllocatorcpp">trunk/Source/JavaScriptCore/heap/MarkedAllocator.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoreheapMarkedAllocatorh">trunk/Source/JavaScriptCore/heap/MarkedAllocator.h</a></li>
<li><a href="#trunkSourceJavaScriptCoreheapMarkedSpacecpp">trunk/Source/JavaScriptCore/heap/MarkedSpace.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoreheapMarkedSpaceh">trunk/Source/JavaScriptCore/heap/MarkedSpace.h</a></li>
<li><a href="#trunkSourceJavaScriptCorejitAssemblyHelpersh">trunk/Source/JavaScriptCore/jit/AssemblyHelpers.h</a></li>
<li><a href="#trunkSourceJavaScriptCorejitJITOperationscpp">trunk/Source/JavaScriptCore/jit/JITOperations.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCorejitJITOperationsh">trunk/Source/JavaScriptCore/jit/JITOperations.h</a></li>
<li><a href="#trunkSourceJavaScriptCorejitJITPropertyAccesscpp">trunk/Source/JavaScriptCore/jit/JITPropertyAccess.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCorejitJITPropertyAccess32_64cpp">trunk/Source/JavaScriptCore/jit/JITPropertyAccess32_64.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCorellintLowLevelInterpreterasm">trunk/Source/JavaScriptCore/llint/LowLevelInterpreter.asm</a></li>
<li><a href="#trunkSourceJavaScriptCoreofflineasmx86rb">trunk/Source/JavaScriptCore/offlineasm/x86.rb</a></li>
<li><a href="#trunkSourceJavaScriptCoreruntimeJSArraycpp">trunk/Source/JavaScriptCore/runtime/JSArray.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoreruntimeJSArrayh">trunk/Source/JavaScriptCore/runtime/JSArray.h</a></li>
<li><a href="#trunkSourceJavaScriptCoreruntimeJSCellh">trunk/Source/JavaScriptCore/runtime/JSCell.h</a></li>
<li><a href="#trunkSourceJavaScriptCoreruntimeJSCellInlinesh">trunk/Source/JavaScriptCore/runtime/JSCellInlines.h</a></li>
<li><a href="#trunkSourceJavaScriptCoreruntimeJSObjecth">trunk/Source/JavaScriptCore/runtime/JSObject.h</a></li>
<li><a href="#trunkSourceJavaScriptCoreruntimeJSStringh">trunk/Source/JavaScriptCore/runtime/JSString.h</a></li>
<li><a href="#trunkSourceJavaScriptCoreruntimeOptionsh">trunk/Source/JavaScriptCore/runtime/Options.h</a></li>
<li><a href="#trunkSourceJavaScriptCoreruntimeRegExpMatchesArraycpp">trunk/Source/JavaScriptCore/runtime/RegExpMatchesArray.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoreruntimeRegExpMatchesArrayh">trunk/Source/JavaScriptCore/runtime/RegExpMatchesArray.h</a></li>
<li><a href="#trunkSourceWTFChangeLog">trunk/Source/WTF/ChangeLog</a></li>
<li><a href="#trunkSourceWTFwtfFastBitVectorcpp">trunk/Source/WTF/wtf/FastBitVector.cpp</a></li>
<li><a href="#trunkSourceWTFwtfFastBitVectorh">trunk/Source/WTF/wtf/FastBitVector.h</a></li>
</ul>
<h3>Added Paths</h3>
<ul>
<li><a href="#trunkSourceJavaScriptCoredfgDFGStoreBarrierClusteringPhasecpp">trunk/Source/JavaScriptCore/dfg/DFGStoreBarrierClusteringPhase.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoredfgDFGStoreBarrierClusteringPhaseh">trunk/Source/JavaScriptCore/dfg/DFGStoreBarrierClusteringPhase.h</a></li>
<li><a href="#trunkSourceJavaScriptCoreheapGCDeferralContexth">trunk/Source/JavaScriptCore/heap/GCDeferralContext.h</a></li>
<li><a href="#trunkSourceJavaScriptCoreheapGCDeferralContextInlinesh">trunk/Source/JavaScriptCore/heap/GCDeferralContextInlines.h</a></li>
</ul>
</div>
<div id="patch">
<h3>Diff</h3>
<a id="trunkSourceJavaScriptCoreCMakeListstxt"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/CMakeLists.txt (206554 => 206555)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/CMakeLists.txt 2016-09-28 21:53:53 UTC (rev 206554)
+++ trunk/Source/JavaScriptCore/CMakeLists.txt 2016-09-28 21:55:53 UTC (rev 206555)
</span><span class="lines">@@ -361,6 +361,7 @@
</span><span class="cx"> dfg/DFGSpeculativeJIT64.cpp
</span><span class="cx"> dfg/DFGStackLayoutPhase.cpp
</span><span class="cx"> dfg/DFGStaticExecutionCountEstimationPhase.cpp
</span><ins>+ dfg/DFGStoreBarrierClusteringPhase.cpp
</ins><span class="cx"> dfg/DFGStoreBarrierInsertionPhase.cpp
</span><span class="cx"> dfg/DFGStrengthReductionPhase.cpp
</span><span class="cx"> dfg/DFGStructureAbstractValue.cpp
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreChangeLog"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/ChangeLog (206554 => 206555)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/ChangeLog 2016-09-28 21:53:53 UTC (rev 206554)
+++ trunk/Source/JavaScriptCore/ChangeLog 2016-09-28 21:55:53 UTC (rev 206555)
</span><span class="lines">@@ -1,3 +1,191 @@
</span><ins>+2016-09-28 Filip Pizlo <fpizlo@apple.com>
+
+ The write barrier should be down with TSO
+ https://bugs.webkit.org/show_bug.cgi?id=162316
+
+ Reviewed by Geoffrey Garen.
+
+ This makes our write barrier behave correctly when it races with the collector. The
+ collector wants to do this when visiting:
+
+ object->cellState = black
+ visit(object)
+
+ The mutator wants to do this when storing:
+
+ object->property = newValue
+ if (object->cellState == black)
+ remember(object)
+
+ Prior to this change, this didn't work right because the compiler would sometimes place
+ barriers before the store to the property and because the mutator did not have adequate
+ fences.
+
+ Prior to this change, the DFG and FTL would emit this:
+
+ if (object->cellState == black)
+ remember(object)
+ object->property = newValue
+
+ Which is wrong, because the object could start being scanned just after the cellState
+ check, at which point the store would be lost. We need to confirm that the state was not
+ black *after* the store! This change was harder than you'd expect: placing the barrier
+ after the store broke B3's ability to do its super crazy ninja CSE on some store-load
+ redundancies. Because the B3 CSE has some moves that the DFG CSE lacks, the DFG CSE's
+ ability to ignore barriers didn't help. I fixed this by having the FTL convey precise
+ heap ranges for the patchpoint corresponding to the barrier slow path. It reads the world
+ (because of the store-load fence) and it writes only cellState (because the B3 heap ranges
+ don't have any way to represent any of the GC's other state, which means that B3 does not
+ have to worry about aliasing with any of that).
+
+ The collector already uses a store-load fence on x86 just after setting the cellState and
+ before visiting the object. The mutator needs to do the same. But we cannot put a
+ store-load fence of any kind before store barriers, because that causes enormous slow
+ downs. In the worst case, Octane/richards slowed down by 90%! That's crazy! However, the
+ overall slow downs were small enough (0-15% on benchmark suite aggregates) that it would be
+ reasonable if the slow down only happened while the GC was running. Then, the concurrent GC
+ would lift throughput-while-collecting from 0% of peak to 85% of peak. This changes the
+ barrier so that it looks like this:
+
+ if (object->cellState <= heap.sneakyBlackThreshold)
+ slowPath(object)
+
+ Where sneakyBlackThreshold is the normal blackThreshold when we're not collecting, or a
+ tautoligical threshold (that makes everything look black) when we are collecting. This
+ turns out to not be any more expensive than the barrier in tip of tree when the GC is not
+ running, or a 0-15% slow-down when it is "running". (Of course we don't run the GC
+ concurrently yet. I still have more work to do.) The slowPath() does some extra work to
+ check if we are concurrently collecting; if so, it does a fence and rechecks if the object
+ really did need that barrier.
+
+ This also reintroduces elimination of redundant store barriers, which was lost in the last
+ store barrier change. We can only do it when there is no possibility of GC, exit, or
+ exceptions between the two store barriers. We could remove the exit/exception limitation if
+ we taught OSR exit how to buffer store barriers, which is an insane thing to do considering
+ that I've never been able to detect a win from redundant store barrier elimination. I just
+ want us to have it for stupidly obvious situations, like a tight sequence of stores to the
+ same object. This same optimization also sometimes strength-reduces the store barrier so
+ that it uses a constant black threshold rather than the sneaky one, thereby saving one
+ load.
+
+ Even with all of those optimizations, I still had problems with barrier cost. I found that one
+ of the benchmarks that was being hit particularly hard was JetStream/regexp-2010. Fortunately
+ that benchmark does most of its barriers in a tight C++ loop in RegExpMatchesArray.h. When we
+ know what we're doing, we can defer GC around a bunch of object initializations and then remove
+ all of the barriers between any of the objects allocated within the deferral. Unfortunately,
+ our GC deferral mechanism isn't really performant enough to make this be a worthwhile
+ optimization. The most efficient version of such an optimization that I could come up with was
+ to have a DeferralContext object that houses a boolean that is false by default, but the GC
+ writes true into it if it would have wanted to GC. You thread a pointer to the deferralContext
+ through all of your allocations. This kind of mechanism has the overhead of a zero
+ initialization on the stack on entry and a zero check on exit. This is probably even efficient
+ enough that we could start thinking about having the DFG use it, for example if we found a
+ bounded-time section of code with a lot of barriers and entry/exit sites that aren't totally
+ wacky. This optimization took this patch from 0.68% JetStream regressed to neutral, according
+ to my latest data.
+
+ Finally, an earlier version of this change put the store-load fence in B3 IR, so I ended up
+ adding FTLOutput support for it and AbstractHeapRepository magic for decorating the heaps.
+ I think we might as well keep that, it'll be useful.
+
+ * CMakeLists.txt:
+ * JavaScriptCore.xcodeproj/project.pbxproj:
+ * assembler/MacroAssembler.h:
+ (JSC::MacroAssembler::branch32):
+ * assembler/MacroAssemblerX86_64.h:
+ (JSC::MacroAssemblerX86_64::branch32):
+ (JSC::MacroAssemblerX86_64::branch64): Deleted.
+ * bytecode/PolymorphicAccess.cpp:
+ (JSC::AccessCase::generateImpl):
+ * dfg/DFGAbstractHeap.h:
+ * dfg/DFGAbstractInterpreterInlines.h:
+ (JSC::DFG::AbstractInterpreter<AbstractStateType>::executeEffects):
+ * dfg/DFGClobberize.h:
+ (JSC::DFG::clobberize):
+ * dfg/DFGClobbersExitState.cpp:
+ (JSC::DFG::clobbersExitState):
+ * dfg/DFGDoesGC.cpp:
+ (JSC::DFG::doesGC):
+ * dfg/DFGFixupPhase.cpp:
+ (JSC::DFG::FixupPhase::fixupNode):
+ * dfg/DFGMayExit.cpp:
+ * dfg/DFGNode.h:
+ (JSC::DFG::Node::isStoreBarrier):
+ * dfg/DFGNodeType.h:
+ * dfg/DFGPlan.cpp:
+ (JSC::DFG::Plan::compileInThreadImpl):
+ * dfg/DFGPredictionPropagationPhase.cpp:
+ * dfg/DFGSafeToExecute.h:
+ (JSC::DFG::safeToExecute):
+ * dfg/DFGSpeculativeJIT.cpp:
+ (JSC::DFG::SpeculativeJIT::compileStoreBarrier):
+ (JSC::DFG::SpeculativeJIT::storeToWriteBarrierBuffer): Deleted.
+ (JSC::DFG::SpeculativeJIT::writeBarrier): Deleted.
+ * dfg/DFGSpeculativeJIT.h:
+ * dfg/DFGSpeculativeJIT32_64.cpp:
+ (JSC::DFG::SpeculativeJIT::compile):
+ (JSC::DFG::SpeculativeJIT::compileBaseValueStoreBarrier): Deleted.
+ (JSC::DFG::SpeculativeJIT::writeBarrier): Deleted.
+ * dfg/DFGSpeculativeJIT64.cpp:
+ (JSC::DFG::SpeculativeJIT::compile):
+ (JSC::DFG::SpeculativeJIT::compileBaseValueStoreBarrier): Deleted.
+ (JSC::DFG::SpeculativeJIT::writeBarrier): Deleted.
+ * dfg/DFGStoreBarrierClusteringPhase.cpp: Added.
+ (JSC::DFG::performStoreBarrierClustering):
+ * dfg/DFGStoreBarrierClusteringPhase.h: Added.
+ * dfg/DFGStoreBarrierInsertionPhase.cpp:
+ * dfg/DFGStoreBarrierInsertionPhase.h:
+ * ftl/FTLAbstractHeap.h:
+ (JSC::FTL::AbsoluteAbstractHeap::at):
+ (JSC::FTL::AbsoluteAbstractHeap::operator[]):
+ * ftl/FTLAbstractHeapRepository.cpp:
+ (JSC::FTL::AbstractHeapRepository::decorateFenceRead):
+ (JSC::FTL::AbstractHeapRepository::decorateFenceWrite):
+ (JSC::FTL::AbstractHeapRepository::computeRangesAndDecorateInstructions):
+ * ftl/FTLAbstractHeapRepository.h:
+ * ftl/FTLCapabilities.cpp:
+ (JSC::FTL::canCompile):
+ * ftl/FTLLowerDFGToB3.cpp:
+ (JSC::FTL::DFG::LowerDFGToB3::compileNode):
+ (JSC::FTL::DFG::LowerDFGToB3::compileStoreBarrier):
+ (JSC::FTL::DFG::LowerDFGToB3::storageForTransition):
+ (JSC::FTL::DFG::LowerDFGToB3::lazySlowPath):
+ (JSC::FTL::DFG::LowerDFGToB3::emitStoreBarrier):
+ * ftl/FTLOutput.cpp:
+ (JSC::FTL::Output::fence):
+ (JSC::FTL::Output::absolute):
+ * ftl/FTLOutput.h:
+ * heap/CellState.h:
+ (JSC::isWithinThreshold):
+ (JSC::isBlack):
+ * heap/Heap.cpp:
+ (JSC::Heap::writeBarrierSlowPath):
+ * heap/Heap.h:
+ (JSC::Heap::barrierShouldBeFenced):
+ (JSC::Heap::addressOfBarrierShouldBeFenced):
+ (JSC::Heap::sneakyBlackThreshold):
+ (JSC::Heap::addressOfSneakyBlackThreshold):
+ * heap/HeapInlines.h:
+ (JSC::Heap::writeBarrier):
+ (JSC::Heap::writeBarrierWithoutFence):
+ * jit/AssemblyHelpers.h:
+ (JSC::AssemblyHelpers::jumpIfIsRememberedOrInEdenWithoutFence):
+ (JSC::AssemblyHelpers::sneakyJumpIfIsRememberedOrInEden):
+ (JSC::AssemblyHelpers::jumpIfIsRememberedOrInEden):
+ (JSC::AssemblyHelpers::storeBarrierStoreLoadFence):
+ (JSC::AssemblyHelpers::jumpIfStoreBarrierStoreLoadFenceNotNeeded):
+ * jit/JITOperations.cpp:
+ * jit/JITOperations.h:
+ * jit/JITPropertyAccess.cpp:
+ (JSC::JIT::emit_op_put_by_id):
+ (JSC::JIT::emitWriteBarrier):
+ (JSC::JIT::privateCompilePutByVal):
+ * jit/JITPropertyAccess32_64.cpp:
+ (JSC::JIT::emit_op_put_by_id):
+ * llint/LowLevelInterpreter.asm:
+ * offlineasm/x86.rb:
+ * runtime/Options.h:
+
</ins><span class="cx"> 2016-09-27 Joseph Pecoraro <pecoraro@apple.com>
</span><span class="cx">
</span><span class="cx"> Improve useCodeCache Option description string.
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreJavaScriptCorexcodeprojprojectpbxproj"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/JavaScriptCore.xcodeproj/project.pbxproj (206554 => 206555)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/JavaScriptCore.xcodeproj/project.pbxproj 2016-09-28 21:53:53 UTC (rev 206554)
+++ trunk/Source/JavaScriptCore/JavaScriptCore.xcodeproj/project.pbxproj 2016-09-28 21:55:53 UTC (rev 206555)
</span><span class="lines">@@ -503,6 +503,8 @@
</span><span class="cx"> 0F7C39FF1C90C55B00480151 /* DFGOpInfo.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F7C39FE1C90C55B00480151 /* DFGOpInfo.h */; };
</span><span class="cx"> 0F7C5FB81D888A0C0044F5E2 /* MarkedBlockInlines.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F7C5FB71D888A010044F5E2 /* MarkedBlockInlines.h */; };
</span><span class="cx"> 0F7C5FBA1D8895070044F5E2 /* MarkedSpaceInlines.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F7C5FB91D8895050044F5E2 /* MarkedSpaceInlines.h */; };
</span><ins>+ 0F7F988B1D9596C500F4F12E /* DFGStoreBarrierClusteringPhase.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 0F7F98891D9596C300F4F12E /* DFGStoreBarrierClusteringPhase.cpp */; };
+ 0F7F988C1D9596C800F4F12E /* DFGStoreBarrierClusteringPhase.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F7F988A1D9596C300F4F12E /* DFGStoreBarrierClusteringPhase.h */; };
</ins><span class="cx"> 0F8023EA1613832B00A0BA45 /* ByValInfo.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F8023E91613832300A0BA45 /* ByValInfo.h */; settings = {ATTRIBUTES = (Private, ); }; };
</span><span class="cx"> 0F8335B71639C1E6001443B5 /* ArrayAllocationProfile.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 0F8335B41639C1E3001443B5 /* ArrayAllocationProfile.cpp */; };
</span><span class="cx"> 0F8335B81639C1EA001443B5 /* ArrayAllocationProfile.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F8335B51639C1E3001443B5 /* ArrayAllocationProfile.h */; settings = {ATTRIBUTES = (Private, ); }; };
</span><span class="lines">@@ -613,6 +615,8 @@
</span><span class="cx"> 0FB387921BFD31A100E3AB1E /* FTLCompile.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 0FB387911BFD31A100E3AB1E /* FTLCompile.cpp */; };
</span><span class="cx"> 0FB415841D78FB4C00DF8D09 /* ArrayConventions.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 0FB415831D78F98200DF8D09 /* ArrayConventions.cpp */; };
</span><span class="cx"> 0FB438A319270B1D00E1FBC9 /* StructureSet.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 0FB438A219270B1D00E1FBC9 /* StructureSet.cpp */; };
</span><ins>+ 0FB4767E1D99AEA9008EA6CB /* GCDeferralContext.h in Headers */ = {isa = PBXBuildFile; fileRef = 0FB4767C1D99AEA7008EA6CB /* GCDeferralContext.h */; settings = {ATTRIBUTES = (Private, ); }; };
+ 0FB4767F1D99AEAD008EA6CB /* GCDeferralContextInlines.h in Headers */ = {isa = PBXBuildFile; fileRef = 0FB4767D1D99AEA7008EA6CB /* GCDeferralContextInlines.h */; };
</ins><span class="cx"> 0FB4FB731BC843140025CA5A /* FTLLazySlowPath.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 0FB4FB701BC843140025CA5A /* FTLLazySlowPath.cpp */; };
</span><span class="cx"> 0FB4FB741BC843140025CA5A /* FTLLazySlowPath.h in Headers */ = {isa = PBXBuildFile; fileRef = 0FB4FB711BC843140025CA5A /* FTLLazySlowPath.h */; };
</span><span class="cx"> 0FB4FB751BC843140025CA5A /* FTLLazySlowPathCall.h in Headers */ = {isa = PBXBuildFile; fileRef = 0FB4FB721BC843140025CA5A /* FTLLazySlowPathCall.h */; };
</span><span class="lines">@@ -2743,6 +2747,8 @@
</span><span class="cx"> 0F7C39FE1C90C55B00480151 /* DFGOpInfo.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = DFGOpInfo.h; path = dfg/DFGOpInfo.h; sourceTree = "<group>"; };
</span><span class="cx"> 0F7C5FB71D888A010044F5E2 /* MarkedBlockInlines.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = MarkedBlockInlines.h; sourceTree = "<group>"; };
</span><span class="cx"> 0F7C5FB91D8895050044F5E2 /* MarkedSpaceInlines.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = MarkedSpaceInlines.h; sourceTree = "<group>"; };
</span><ins>+ 0F7F98891D9596C300F4F12E /* DFGStoreBarrierClusteringPhase.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; name = DFGStoreBarrierClusteringPhase.cpp; path = dfg/DFGStoreBarrierClusteringPhase.cpp; sourceTree = "<group>"; };
+ 0F7F988A1D9596C300F4F12E /* DFGStoreBarrierClusteringPhase.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = DFGStoreBarrierClusteringPhase.h; path = dfg/DFGStoreBarrierClusteringPhase.h; sourceTree = "<group>"; };
</ins><span class="cx"> 0F8023E91613832300A0BA45 /* ByValInfo.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = ByValInfo.h; sourceTree = "<group>"; };
</span><span class="cx"> 0F8335B41639C1E3001443B5 /* ArrayAllocationProfile.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = ArrayAllocationProfile.cpp; sourceTree = "<group>"; };
</span><span class="cx"> 0F8335B51639C1E3001443B5 /* ArrayAllocationProfile.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = ArrayAllocationProfile.h; sourceTree = "<group>"; };
</span><span class="lines">@@ -2852,6 +2858,8 @@
</span><span class="cx"> 0FB387911BFD31A100E3AB1E /* FTLCompile.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; name = FTLCompile.cpp; path = ftl/FTLCompile.cpp; sourceTree = "<group>"; };
</span><span class="cx"> 0FB415831D78F98200DF8D09 /* ArrayConventions.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = ArrayConventions.cpp; sourceTree = "<group>"; };
</span><span class="cx"> 0FB438A219270B1D00E1FBC9 /* StructureSet.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = StructureSet.cpp; sourceTree = "<group>"; };
</span><ins>+ 0FB4767C1D99AEA7008EA6CB /* GCDeferralContext.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = GCDeferralContext.h; sourceTree = "<group>"; };
+ 0FB4767D1D99AEA7008EA6CB /* GCDeferralContextInlines.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = GCDeferralContextInlines.h; sourceTree = "<group>"; };
</ins><span class="cx"> 0FB4B51016B3A964003F696B /* DFGMinifiedID.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = DFGMinifiedID.h; path = dfg/DFGMinifiedID.h; sourceTree = "<group>"; };
</span><span class="cx"> 0FB4B51916B62772003F696B /* DFGAllocator.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = DFGAllocator.h; path = dfg/DFGAllocator.h; sourceTree = "<group>"; };
</span><span class="cx"> 0FB4B51A16B62772003F696B /* DFGCommon.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; name = DFGCommon.cpp; path = dfg/DFGCommon.cpp; sourceTree = "<group>"; };
</span><span class="lines">@@ -5294,6 +5302,8 @@
</span><span class="cx"> 2AACE63A18CA5A0300ED0191 /* GCActivityCallback.cpp */,
</span><span class="cx"> 2AACE63B18CA5A0300ED0191 /* GCActivityCallback.h */,
</span><span class="cx"> BCBE2CAD14E985AA000593AD /* GCAssertions.h */,
</span><ins>+ 0FB4767C1D99AEA7008EA6CB /* GCDeferralContext.h */,
+ 0FB4767D1D99AEA7008EA6CB /* GCDeferralContextInlines.h */,
</ins><span class="cx"> 0F2B66A817B6B53D00A7AE3F /* GCIncomingRefCounted.h */,
</span><span class="cx"> 0F2B66A917B6B53D00A7AE3F /* GCIncomingRefCountedInlines.h */,
</span><span class="cx"> 0F2B66AA17B6B53D00A7AE3F /* GCIncomingRefCountedSet.h */,
</span><span class="lines">@@ -6555,6 +6565,8 @@
</span><span class="cx"> 0F9FB4F317FCB91700CB67F8 /* DFGStackLayoutPhase.h */,
</span><span class="cx"> 0F4F29DD18B6AD1C0057BC15 /* DFGStaticExecutionCountEstimationPhase.cpp */,
</span><span class="cx"> 0F4F29DE18B6AD1C0057BC15 /* DFGStaticExecutionCountEstimationPhase.h */,
</span><ins>+ 0F7F98891D9596C300F4F12E /* DFGStoreBarrierClusteringPhase.cpp */,
+ 0F7F988A1D9596C300F4F12E /* DFGStoreBarrierClusteringPhase.h */,
</ins><span class="cx"> 0F9E32611B05AB0400801ED5 /* DFGStoreBarrierInsertionPhase.cpp */,
</span><span class="cx"> 0F9E32621B05AB0400801ED5 /* DFGStoreBarrierInsertionPhase.h */,
</span><span class="cx"> 0FC20CB31852E2C600C9E954 /* DFGStrengthReductionPhase.cpp */,
</span><span class="lines">@@ -7948,6 +7960,7 @@
</span><span class="cx"> 0F61832A1C45BF070072450B /* AirCCallingConvention.h in Headers */,
</span><span class="cx"> E33F50791B84225700413856 /* JSInternalPromiseConstructor.h in Headers */,
</span><span class="cx"> E328DAE91D38D005001A2529 /* BytecodeGraph.h in Headers */,
</span><ins>+ 0F7F988C1D9596C800F4F12E /* DFGStoreBarrierClusteringPhase.h in Headers */,
</ins><span class="cx"> E33F50871B8449EF00413856 /* JSInternalPromiseConstructor.lut.h in Headers */,
</span><span class="cx"> E33F50851B8437A000413856 /* JSInternalPromiseDeferred.h in Headers */,
</span><span class="cx"> E33F50751B8421C000413856 /* JSInternalPromisePrototype.h in Headers */,
</span><span class="lines">@@ -8328,6 +8341,7 @@
</span><span class="cx"> A7B601821639FD2A00372BA3 /* UnlinkedCodeBlock.h in Headers */,
</span><span class="cx"> 14142E511B796ECE00F4BF4B /* UnlinkedFunctionExecutable.h in Headers */,
</span><span class="cx"> 0F2E892C16D028AD009E4FD2 /* UnusedPointer.h in Headers */,
</span><ins>+ 0FB4767E1D99AEA9008EA6CB /* GCDeferralContext.h in Headers */,
</ins><span class="cx"> 99DA00B11BD5994E00F4575C /* UpdateContents.py in Headers */,
</span><span class="cx"> 0F963B3813FC6FE90002D9B2 /* ValueProfile.h in Headers */,
</span><span class="cx"> 0F426A481460CBB300131F8F /* ValueRecovery.h in Headers */,
</span><span class="lines">@@ -8370,6 +8384,7 @@
</span><span class="cx"> 451539B912DC994500EF7AC4 /* Yarr.h in Headers */,
</span><span class="cx"> 86704B8512DBA33700A9FE7B /* YarrInterpreter.h in Headers */,
</span><span class="cx"> 86704B8712DBA33700A9FE7B /* YarrJIT.h in Headers */,
</span><ins>+ 0FB4767F1D99AEAD008EA6CB /* GCDeferralContextInlines.h in Headers */,
</ins><span class="cx"> 86704B8812DBA33700A9FE7B /* YarrParser.h in Headers */,
</span><span class="cx"> 86704B8A12DBA33700A9FE7B /* YarrPattern.h in Headers */,
</span><span class="cx"> 262D85B71C0D650F006ACB61 /* AirFixPartialRegisterStalls.h in Headers */,
</span><span class="lines">@@ -9098,6 +9113,7 @@
</span><span class="cx"> A7D89CF717A0B8CC00773AD8 /* DFGFlushFormat.cpp in Sources */,
</span><span class="cx"> 0F69CC88193AC60A0045759E /* DFGFrozenValue.cpp in Sources */,
</span><span class="cx"> 86EC9DC71328DF82002B2AD7 /* DFGGraph.cpp in Sources */,
</span><ins>+ 0F7F988B1D9596C500F4F12E /* DFGStoreBarrierClusteringPhase.cpp in Sources */,
</ins><span class="cx"> 0F2FCCF918A60070001A27F8 /* DFGGraphSafepoint.cpp in Sources */,
</span><span class="cx"> 0FB17660196B8F9E0091052A /* DFGHeapLocation.cpp in Sources */,
</span><span class="cx"> 0FC841681BA8C3210061837D /* DFGInferredTypeCheck.cpp in Sources */,
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreassemblerMacroAssemblerh"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/assembler/MacroAssembler.h (206554 => 206555)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/assembler/MacroAssembler.h 2016-09-28 21:53:53 UTC (rev 206554)
+++ trunk/Source/JavaScriptCore/assembler/MacroAssembler.h 2016-09-28 21:55:53 UTC (rev 206555)
</span><span class="lines">@@ -373,6 +373,11 @@
</span><span class="cx"> branchPtr(cond, op1, imm).linkTo(target, this);
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+ Jump branch32(RelationalCondition cond, RegisterID left, AbsoluteAddress right)
+ {
+ return branch32(flip(cond), right, left);
+ }
+
</ins><span class="cx"> void branch32(RelationalCondition cond, RegisterID op1, RegisterID op2, Label target)
</span><span class="cx"> {
</span><span class="cx"> branch32(cond, op1, op2).linkTo(target, this);
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreassemblerMacroAssemblerX86_64h"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/assembler/MacroAssemblerX86_64.h (206554 => 206555)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/assembler/MacroAssemblerX86_64.h 2016-09-28 21:53:53 UTC (rev 206554)
+++ trunk/Source/JavaScriptCore/assembler/MacroAssemblerX86_64.h 2016-09-28 21:55:53 UTC (rev 206555)
</span><span class="lines">@@ -44,6 +44,7 @@
</span><span class="cx">
</span><span class="cx"> using MacroAssemblerX86Common::add32;
</span><span class="cx"> using MacroAssemblerX86Common::and32;
</span><ins>+ using MacroAssemblerX86Common::branch32;
</ins><span class="cx"> using MacroAssemblerX86Common::branchAdd32;
</span><span class="cx"> using MacroAssemblerX86Common::or32;
</span><span class="cx"> using MacroAssemblerX86Common::sub32;
</span><span class="lines">@@ -827,6 +828,12 @@
</span><span class="cx"> m_assembler.cmpq_rm(right, address.offset, address.base, address.index, address.scale);
</span><span class="cx"> return Jump(m_assembler.jCC(x86Condition(cond)));
</span><span class="cx"> }
</span><ins>+
+ Jump branch32(RelationalCondition cond, AbsoluteAddress left, RegisterID right)
+ {
+ load32(left.m_ptr, scratchRegister());
+ return branch32(cond, scratchRegister(), right);
+ }
</ins><span class="cx">
</span><span class="cx"> Jump branchPtr(RelationalCondition cond, BaseIndex left, RegisterID right)
</span><span class="cx"> {
</span></span></pre></div>
<a id="trunkSourceJavaScriptCorebytecodePolymorphicAccesscpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/bytecode/PolymorphicAccess.cpp (206554 => 206555)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/bytecode/PolymorphicAccess.cpp 2016-09-28 21:53:53 UTC (rev 206554)
+++ trunk/Source/JavaScriptCore/bytecode/PolymorphicAccess.cpp 2016-09-28 21:55:53 UTC (rev 206555)
</span><span class="lines">@@ -1309,29 +1309,7 @@
</span><span class="cx"> CCallHelpers::Address(scratchGPR, offsetInButterfly(m_offset) * sizeof(JSValue)));
</span><span class="cx"> }
</span><span class="cx">
</span><del>- // If we had allocated using an operation then we would have already executed the store
- // barrier and we would have already stored the butterfly into the object.
</del><span class="cx"> if (allocatingInline) {
</span><del>- CCallHelpers::Jump ownerIsRememberedOrInEden = jit.jumpIfIsRememberedOrInEden(baseGPR);
- WriteBarrierBuffer& writeBarrierBuffer = jit.vm()->heap.writeBarrierBuffer();
- jit.load32(writeBarrierBuffer.currentIndexAddress(), scratchGPR2);
- slowPath.append(
- jit.branch32(
- CCallHelpers::AboveOrEqual, scratchGPR2,
- CCallHelpers::TrustedImm32(writeBarrierBuffer.capacity())));
-
- jit.add32(CCallHelpers::TrustedImm32(1), scratchGPR2);
- jit.store32(scratchGPR2, writeBarrierBuffer.currentIndexAddress());
-
- jit.move(CCallHelpers::TrustedImmPtr(writeBarrierBuffer.buffer()), scratchGPR3);
- // We use an offset of -sizeof(void*) because we already added 1 to scratchGPR2.
- jit.storePtr(
- baseGPR,
- CCallHelpers::BaseIndex(
- scratchGPR3, scratchGPR2, CCallHelpers::ScalePtr,
- static_cast<int32_t>(-sizeof(void*))));
- ownerIsRememberedOrInEden.link(&jit);
-
</del><span class="cx"> // We set the new butterfly and the structure last. Doing it this way ensures that
</span><span class="cx"> // whatever we had done up to this point is forgotten if we choose to branch to slow
</span><span class="cx"> // path.
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoredfgDFGAbstractHeaph"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/dfg/DFGAbstractHeap.h (206554 => 206555)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/dfg/DFGAbstractHeap.h 2016-09-28 21:53:53 UTC (rev 206554)
+++ trunk/Source/JavaScriptCore/dfg/DFGAbstractHeap.h 2016-09-28 21:55:53 UTC (rev 206555)
</span><span class="lines">@@ -51,8 +51,9 @@
</span><span class="cx"> macro(Butterfly_vectorLength) \
</span><span class="cx"> macro(GetterSetter_getter) \
</span><span class="cx"> macro(GetterSetter_setter) \
</span><ins>+ macro(JSCell_cellState) \
+ macro(JSCell_indexingType) \
</ins><span class="cx"> macro(JSCell_structureID) \
</span><del>- macro(JSCell_indexingType) \
</del><span class="cx"> macro(JSCell_typeInfoFlags) \
</span><span class="cx"> macro(JSCell_typeInfoType) \
</span><span class="cx"> macro(JSObject_butterfly) \
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoredfgDFGAbstractInterpreterInlinesh"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/dfg/DFGAbstractInterpreterInlines.h (206554 => 206555)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/dfg/DFGAbstractInterpreterInlines.h 2016-09-28 21:53:53 UTC (rev 206554)
+++ trunk/Source/JavaScriptCore/dfg/DFGAbstractInterpreterInlines.h 2016-09-28 21:55:53 UTC (rev 206555)
</span><span class="lines">@@ -2843,11 +2843,12 @@
</span><span class="cx"> break;
</span><span class="cx"> }
</span><span class="cx">
</span><del>- case StoreBarrier: {
</del><ins>+ case StoreBarrier:
+ case FencedStoreBarrier: {
</ins><span class="cx"> filter(node->child1(), SpecCell);
</span><span class="cx"> break;
</span><span class="cx"> }
</span><del>-
</del><ins>+
</ins><span class="cx"> case CheckTierUpAndOSREnter:
</span><span class="cx"> case LoopHint:
</span><span class="cx"> case ZombieHint:
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoredfgDFGClobberizeh"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/dfg/DFGClobberize.h (206554 => 206555)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/dfg/DFGClobberize.h 2016-09-28 21:53:53 UTC (rev 206554)
+++ trunk/Source/JavaScriptCore/dfg/DFGClobberize.h 2016-09-28 21:55:53 UTC (rev 206555)
</span><span class="lines">@@ -404,11 +404,20 @@
</span><span class="cx"> case LoopHint:
</span><span class="cx"> case ProfileType:
</span><span class="cx"> case ProfileControlFlow:
</span><del>- case StoreBarrier:
</del><span class="cx"> case PutHint:
</span><span class="cx"> write(SideState);
</span><span class="cx"> return;
</span><span class="cx">
</span><ins>+ case StoreBarrier:
+ read(JSCell_cellState);
+ write(JSCell_cellState);
+ return;
+
+ case FencedStoreBarrier:
+ read(Heap);
+ write(JSCell_cellState);
+ return;
+
</ins><span class="cx"> case InvalidationPoint:
</span><span class="cx"> write(SideState);
</span><span class="cx"> def(HeapLocation(InvalidationPointLoc, Watchpoint_fire), LazyNode(node));
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoredfgDFGClobbersExitStatecpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/dfg/DFGClobbersExitState.cpp (206554 => 206555)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/dfg/DFGClobbersExitState.cpp 2016-09-28 21:53:53 UTC (rev 206554)
+++ trunk/Source/JavaScriptCore/dfg/DFGClobbersExitState.cpp 2016-09-28 21:55:53 UTC (rev 206555)
</span><span class="lines">@@ -67,6 +67,8 @@
</span><span class="cx"> case CountExecution:
</span><span class="cx"> case AllocatePropertyStorage:
</span><span class="cx"> case ReallocatePropertyStorage:
</span><ins>+ case StoreBarrier:
+ case FencedStoreBarrier:
</ins><span class="cx"> // These do clobber memory, but nothing that is observable. It may be nice to separate the
</span><span class="cx"> // heaps into those that are observable and those that aren't, but we don't do that right now.
</span><span class="cx"> // FIXME: https://bugs.webkit.org/show_bug.cgi?id=148440
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoredfgDFGDoesGCcpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/dfg/DFGDoesGC.cpp (206554 => 206555)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/dfg/DFGDoesGC.cpp 2016-09-28 21:53:53 UTC (rev 206554)
+++ trunk/Source/JavaScriptCore/dfg/DFGDoesGC.cpp 2016-09-28 21:55:53 UTC (rev 206555)
</span><span class="lines">@@ -196,6 +196,7 @@
</span><span class="cx"> case CheckTierUpAndOSREnter:
</span><span class="cx"> case LoopHint:
</span><span class="cx"> case StoreBarrier:
</span><ins>+ case FencedStoreBarrier:
</ins><span class="cx"> case InvalidationPoint:
</span><span class="cx"> case NotifyWrite:
</span><span class="cx"> case CheckInBounds:
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoredfgDFGFixupPhasecpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/dfg/DFGFixupPhase.cpp (206554 => 206555)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/dfg/DFGFixupPhase.cpp 2016-09-28 21:53:53 UTC (rev 206554)
+++ trunk/Source/JavaScriptCore/dfg/DFGFixupPhase.cpp 2016-09-28 21:55:53 UTC (rev 206555)
</span><span class="lines">@@ -1402,6 +1402,7 @@
</span><span class="cx"> case KillStack:
</span><span class="cx"> case GetStack:
</span><span class="cx"> case StoreBarrier:
</span><ins>+ case FencedStoreBarrier:
</ins><span class="cx"> case GetRegExpObjectLastIndex:
</span><span class="cx"> case SetRegExpObjectLastIndex:
</span><span class="cx"> case RecordRegExpCachedResult:
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoredfgDFGMayExitcpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/dfg/DFGMayExit.cpp (206554 => 206555)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/dfg/DFGMayExit.cpp 2016-09-28 21:53:53 UTC (rev 206554)
+++ trunk/Source/JavaScriptCore/dfg/DFGMayExit.cpp 2016-09-28 21:55:53 UTC (rev 206555)
</span><span class="lines">@@ -87,6 +87,7 @@
</span><span class="cx"> case NotifyWrite:
</span><span class="cx"> case PutStructure:
</span><span class="cx"> case StoreBarrier:
</span><ins>+ case FencedStoreBarrier:
</ins><span class="cx"> case PutByOffset:
</span><span class="cx"> case PutClosureVar:
</span><span class="cx"> case RecordRegExpCachedResult:
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoredfgDFGNodeh"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/dfg/DFGNode.h (206554 => 206555)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/dfg/DFGNode.h 2016-09-28 21:53:53 UTC (rev 206554)
+++ trunk/Source/JavaScriptCore/dfg/DFGNode.h 2016-09-28 21:55:53 UTC (rev 206555)
</span><span class="lines">@@ -896,7 +896,7 @@
</span><span class="cx">
</span><span class="cx"> bool isStoreBarrier()
</span><span class="cx"> {
</span><del>- return op() == StoreBarrier;
</del><ins>+ return op() == StoreBarrier || op() == FencedStoreBarrier;
</ins><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> bool hasIdentifier()
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoredfgDFGNodeTypeh"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/dfg/DFGNodeType.h (206554 => 206555)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/dfg/DFGNodeType.h 2016-09-28 21:53:53 UTC (rev 206554)
+++ trunk/Source/JavaScriptCore/dfg/DFGNodeType.h 2016-09-28 21:55:53 UTC (rev 206555)
</span><span class="lines">@@ -375,8 +375,9 @@
</span><span class="cx"> \
</span><span class="cx"> /* Checks the watchdog timer. If the timer has fired, we call operation operationHandleWatchdogTimer*/ \
</span><span class="cx"> macro(CheckWatchdogTimer, NodeMustGenerate) \
</span><del>- /* Write barriers ! */\
</del><ins>+ /* Write barriers */\
</ins><span class="cx"> macro(StoreBarrier, NodeMustGenerate) \
</span><ins>+ macro(FencedStoreBarrier, NodeMustGenerate) \
</ins><span class="cx"> \
</span><span class="cx"> /* For-in enumeration opcodes */\
</span><span class="cx"> macro(GetEnumerableLength, NodeMustGenerate | NodeResultJS) \
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoredfgDFGOSRExitCompilerCommoncpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/dfg/DFGOSRExitCompilerCommon.cpp (206554 => 206555)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/dfg/DFGOSRExitCompilerCommon.cpp 2016-09-28 21:53:53 UTC (rev 206554)
+++ trunk/Source/JavaScriptCore/dfg/DFGOSRExitCompilerCommon.cpp 2016-09-28 21:55:53 UTC (rev 206555)
</span><span class="lines">@@ -249,7 +249,7 @@
</span><span class="cx">
</span><span class="cx"> static void osrWriteBarrier(CCallHelpers& jit, GPRReg owner, GPRReg scratch)
</span><span class="cx"> {
</span><del>- AssemblyHelpers::Jump ownerIsRememberedOrInEden = jit.jumpIfIsRememberedOrInEden(owner);
</del><ins>+ AssemblyHelpers::Jump ownerIsRememberedOrInEden = jit.barrierBranchWithoutFence(owner);
</ins><span class="cx">
</span><span class="cx"> // We need these extra slots because setupArgumentsWithExecState will use poke on x86.
</span><span class="cx"> #if CPU(X86)
</span><span class="lines">@@ -269,6 +269,8 @@
</span><span class="cx">
</span><span class="cx"> void adjustAndJumpToTarget(CCallHelpers& jit, const OSRExitBase& exit)
</span><span class="cx"> {
</span><ins>+ jit.memoryFence();
+
</ins><span class="cx"> jit.move(
</span><span class="cx"> AssemblyHelpers::TrustedImmPtr(
</span><span class="cx"> jit.codeBlock()->baselineAlternative()), GPRInfo::argumentGPR1);
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoredfgDFGOperationscpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/dfg/DFGOperations.cpp (206554 => 206555)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/dfg/DFGOperations.cpp 2016-09-28 21:53:53 UTC (rev 206554)
+++ trunk/Source/JavaScriptCore/dfg/DFGOperations.cpp 2016-09-28 21:55:53 UTC (rev 206555)
</span><span class="lines">@@ -1021,7 +1021,7 @@
</span><span class="cx">
</span><span class="cx"> JSArray* result;
</span><span class="cx"> if (butterfly)
</span><del>- result = JSArray::createWithButterfly(vm, arrayStructure, butterfly);
</del><ins>+ result = JSArray::createWithButterfly(vm, nullptr, arrayStructure, butterfly);
</ins><span class="cx"> else
</span><span class="cx"> result = JSArray::create(vm, arrayStructure, size);
</span><span class="cx"> return bitwise_cast<char*>(result);
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoredfgDFGPlancpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/dfg/DFGPlan.cpp (206554 => 206555)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/dfg/DFGPlan.cpp 2016-09-28 21:53:53 UTC (rev 206554)
+++ trunk/Source/JavaScriptCore/dfg/DFGPlan.cpp 2016-09-28 21:55:53 UTC (rev 206555)
</span><span class="lines">@@ -64,6 +64,7 @@
</span><span class="cx"> #include "DFGSSALoweringPhase.h"
</span><span class="cx"> #include "DFGStackLayoutPhase.h"
</span><span class="cx"> #include "DFGStaticExecutionCountEstimationPhase.h"
</span><ins>+#include "DFGStoreBarrierClusteringPhase.h"
</ins><span class="cx"> #include "DFGStoreBarrierInsertionPhase.h"
</span><span class="cx"> #include "DFGStrengthReductionPhase.h"
</span><span class="cx"> #include "DFGStructureRegistrationPhase.h"
</span><span class="lines">@@ -362,6 +363,7 @@
</span><span class="cx"> performTierUpCheckInjection(dfg);
</span><span class="cx">
</span><span class="cx"> performFastStoreBarrierInsertion(dfg);
</span><ins>+ performStoreBarrierClustering(dfg);
</ins><span class="cx"> performCleanUp(dfg);
</span><span class="cx"> performCPSRethreading(dfg);
</span><span class="cx"> performDCE(dfg);
</span><span class="lines">@@ -451,6 +453,7 @@
</span><span class="cx"> performLivenessAnalysis(dfg);
</span><span class="cx"> performCFA(dfg);
</span><span class="cx"> performGlobalStoreBarrierInsertion(dfg);
</span><ins>+ performStoreBarrierClustering(dfg);
</ins><span class="cx"> if (Options::useMovHintRemoval())
</span><span class="cx"> performMovHintRemoval(dfg);
</span><span class="cx"> performCleanUp(dfg);
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoredfgDFGPredictionPropagationPhasecpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/dfg/DFGPredictionPropagationPhase.cpp (206554 => 206555)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/dfg/DFGPredictionPropagationPhase.cpp 2016-09-28 21:53:53 UTC (rev 206554)
+++ trunk/Source/JavaScriptCore/dfg/DFGPredictionPropagationPhase.cpp 2016-09-28 21:55:53 UTC (rev 206555)
</span><span class="lines">@@ -995,6 +995,7 @@
</span><span class="cx"> case PutStack:
</span><span class="cx"> case KillStack:
</span><span class="cx"> case StoreBarrier:
</span><ins>+ case FencedStoreBarrier:
</ins><span class="cx"> case GetStack:
</span><span class="cx"> case GetRegExpObjectLastIndex:
</span><span class="cx"> case SetRegExpObjectLastIndex:
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoredfgDFGSafeToExecuteh"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/dfg/DFGSafeToExecute.h (206554 => 206555)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/dfg/DFGSafeToExecute.h 2016-09-28 21:53:53 UTC (rev 206554)
+++ trunk/Source/JavaScriptCore/dfg/DFGSafeToExecute.h 2016-09-28 21:55:53 UTC (rev 206555)
</span><span class="lines">@@ -314,7 +314,6 @@
</span><span class="cx"> case CheckTierUpAtReturn:
</span><span class="cx"> case CheckTierUpAndOSREnter:
</span><span class="cx"> case LoopHint:
</span><del>- case StoreBarrier:
</del><span class="cx"> case InvalidationPoint:
</span><span class="cx"> case NotifyWrite:
</span><span class="cx"> case CheckInBounds:
</span><span class="lines">@@ -370,6 +369,14 @@
</span><span class="cx"> // compiling this node.
</span><span class="cx"> return false;
</span><span class="cx">
</span><ins>+ case StoreBarrier:
+ case FencedStoreBarrier:
+ // We conservatively assume that these cannot be put anywhere, which forces the compiler to
+ // keep them exactly where they were. This is sort of overkill since the clobberize effects
+ // already force these things to be ordered precisely. I'm just not confident enough in my
+ // effect based memory model to rely solely on that right now.
+ return false;
+
</ins><span class="cx"> case GetByVal:
</span><span class="cx"> case GetIndexedPropertyStorage:
</span><span class="cx"> case GetArrayLength:
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoredfgDFGSpeculativeJITcpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/dfg/DFGSpeculativeJIT.cpp (206554 => 206555)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/dfg/DFGSpeculativeJIT.cpp 2016-09-28 21:53:53 UTC (rev 206554)
+++ trunk/Source/JavaScriptCore/dfg/DFGSpeculativeJIT.cpp 2016-09-28 21:55:53 UTC (rev 206555)
</span><span class="lines">@@ -8305,46 +8305,51 @@
</span><span class="cx">
</span><span class="cx"> void SpeculativeJIT::compileStoreBarrier(Node* node)
</span><span class="cx"> {
</span><del>- ASSERT(node->op() == StoreBarrier);
</del><ins>+ ASSERT(node->op() == StoreBarrier || node->op() == FencedStoreBarrier);
</ins><span class="cx">
</span><ins>+ bool isFenced = node->op() == FencedStoreBarrier;
+
</ins><span class="cx"> SpeculateCellOperand base(this, node->child1());
</span><span class="cx"> GPRTemporary scratch1(this);
</span><span class="cx"> GPRTemporary scratch2(this);
</span><span class="cx">
</span><del>- writeBarrier(base.gpr(), scratch1.gpr(), scratch2.gpr());
</del><ins>+ GPRReg baseGPR = base.gpr();
+ GPRReg scratch1GPR = scratch1.gpr();
+ GPRReg scratch2GPR = scratch2.gpr();
+
+ JITCompiler::JumpList ok;
+
+ if (isFenced) {
+ ok.append(m_jit.barrierBranch(baseGPR, scratch1GPR));
+
+ JITCompiler::Jump noFence = m_jit.jumpIfBarrierStoreLoadFenceNotNeeded();
+ m_jit.memoryFence();
+ ok.append(m_jit.barrierBranchWithoutFence(baseGPR));
+ noFence.link(&m_jit);
+ } else
+ ok.append(m_jit.barrierBranchWithoutFence(baseGPR));
</ins><span class="cx">
</span><del>- noResult(node);
-}
-
-void SpeculativeJIT::storeToWriteBarrierBuffer(GPRReg cell, GPRReg scratch1, GPRReg scratch2)
-{
- ASSERT(scratch1 != scratch2);
</del><span class="cx"> WriteBarrierBuffer& writeBarrierBuffer = m_jit.vm()->heap.m_writeBarrierBuffer;
</span><del>- m_jit.load32(writeBarrierBuffer.currentIndexAddress(), scratch2);
- JITCompiler::Jump needToFlush = m_jit.branch32(MacroAssembler::AboveOrEqual, scratch2, MacroAssembler::TrustedImm32(writeBarrierBuffer.capacity()));
</del><ins>+ m_jit.load32(writeBarrierBuffer.currentIndexAddress(), scratch2GPR);
+ JITCompiler::Jump needToFlush = m_jit.branch32(MacroAssembler::AboveOrEqual, scratch2GPR, MacroAssembler::TrustedImm32(writeBarrierBuffer.capacity()));
</ins><span class="cx">
</span><del>- m_jit.add32(TrustedImm32(1), scratch2);
- m_jit.store32(scratch2, writeBarrierBuffer.currentIndexAddress());
</del><ins>+ m_jit.add32(TrustedImm32(1), scratch2GPR);
+ m_jit.store32(scratch2GPR, writeBarrierBuffer.currentIndexAddress());
</ins><span class="cx">
</span><del>- m_jit.move(TrustedImmPtr(writeBarrierBuffer.buffer()), scratch1);
</del><ins>+ m_jit.move(TrustedImmPtr(writeBarrierBuffer.buffer()), scratch1GPR);
</ins><span class="cx"> // We use an offset of -sizeof(void*) because we already added 1 to scratch2.
</span><del>- m_jit.storePtr(cell, MacroAssembler::BaseIndex(scratch1, scratch2, MacroAssembler::ScalePtr, static_cast<int32_t>(-sizeof(void*))));
</del><ins>+ m_jit.storePtr(baseGPR, MacroAssembler::BaseIndex(scratch1GPR, scratch2GPR, MacroAssembler::ScalePtr, static_cast<int32_t>(-sizeof(void*))));
</ins><span class="cx">
</span><del>- JITCompiler::Jump done = m_jit.jump();
</del><ins>+ ok.append(m_jit.jump());
</ins><span class="cx"> needToFlush.link(&m_jit);
</span><span class="cx">
</span><span class="cx"> silentSpillAllRegisters(InvalidGPRReg);
</span><del>- callOperation(operationFlushWriteBarrierBuffer, cell);
</del><ins>+ callOperation(operationFlushWriteBarrierBuffer, baseGPR);
</ins><span class="cx"> silentFillAllRegisters(InvalidGPRReg);
</span><span class="cx">
</span><del>- done.link(&m_jit);
-}
</del><ins>+ ok.link(&m_jit);
</ins><span class="cx">
</span><del>-void SpeculativeJIT::writeBarrier(GPRReg ownerGPR, GPRReg scratch1, GPRReg scratch2)
-{
- JITCompiler::Jump ownerIsRememberedOrInEden = m_jit.jumpIfIsRememberedOrInEden(ownerGPR);
- storeToWriteBarrierBuffer(ownerGPR, scratch1, scratch2);
- ownerIsRememberedOrInEden.link(&m_jit);
</del><ins>+ noResult(node);
</ins><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> void SpeculativeJIT::compilePutAccessorById(Node* node)
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoredfgDFGSpeculativeJITh"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/dfg/DFGSpeculativeJIT.h (206554 => 206555)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/dfg/DFGSpeculativeJIT.h 2016-09-28 21:53:53 UTC (rev 206554)
+++ trunk/Source/JavaScriptCore/dfg/DFGSpeculativeJIT.h 2016-09-28 21:55:53 UTC (rev 206555)
</span><span class="lines">@@ -295,12 +295,6 @@
</span><span class="cx"> return masqueradesAsUndefinedWatchpointIsStillValid(m_currentNode->origin.semantic);
</span><span class="cx"> }
</span><span class="cx">
</span><del>- void storeToWriteBarrierBuffer(GPRReg cell, GPRReg scratch1, GPRReg scratch2);
-
- void writeBarrier(GPRReg owner, GPRReg scratch1, GPRReg scratch2);
-
- void writeBarrier(GPRReg owner, GPRReg value, Edge valueUse, GPRReg scratch1, GPRReg scratch2);
-
</del><span class="cx"> void compileStoreBarrier(Node*);
</span><span class="cx">
</span><span class="cx"> static GPRReg selectScratchGPR(GPRReg preserve1 = InvalidGPRReg, GPRReg preserve2 = InvalidGPRReg, GPRReg preserve3 = InvalidGPRReg, GPRReg preserve4 = InvalidGPRReg)
</span><span class="lines">@@ -737,8 +731,6 @@
</span><span class="cx"> void compileTryGetById(Node*);
</span><span class="cx"> void compileIn(Node*);
</span><span class="cx">
</span><del>- void compileBaseValueStoreBarrier(Edge& baseEdge, Edge& valueEdge);
-
</del><span class="cx"> void nonSpeculativeNonPeepholeCompareNullOrUndefined(Edge operand);
</span><span class="cx"> void nonSpeculativePeepholeBranchNullOrUndefined(Edge operand, Node* branchNode);
</span><span class="cx">
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoredfgDFGSpeculativeJIT32_64cpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/dfg/DFGSpeculativeJIT32_64.cpp (206554 => 206555)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/dfg/DFGSpeculativeJIT32_64.cpp 2016-09-28 21:53:53 UTC (rev 206554)
+++ trunk/Source/JavaScriptCore/dfg/DFGSpeculativeJIT32_64.cpp 2016-09-28 21:55:53 UTC (rev 206555)
</span><span class="lines">@@ -1268,18 +1268,6 @@
</span><span class="cx"> }
</span><span class="cx"> }
</span><span class="cx">
</span><del>-void SpeculativeJIT::compileBaseValueStoreBarrier(Edge& baseEdge, Edge& valueEdge)
-{
- ASSERT(!isKnownNotCell(valueEdge.node()));
-
- SpeculateCellOperand base(this, baseEdge);
- JSValueOperand value(this, valueEdge);
- GPRTemporary scratch1(this);
- GPRTemporary scratch2(this);
-
- writeBarrier(base.gpr(), value.tagGPR(), valueEdge, scratch1.gpr(), scratch2.gpr());
-}
-
</del><span class="cx"> void SpeculativeJIT::compileObjectEquality(Node* node)
</span><span class="cx"> {
</span><span class="cx"> SpeculateCellOperand op1(this, node->child1());
</span><span class="lines">@@ -5007,7 +4995,8 @@
</span><span class="cx"> break;
</span><span class="cx"> }
</span><span class="cx">
</span><del>- case StoreBarrier: {
</del><ins>+ case StoreBarrier:
+ case FencedStoreBarrier: {
</ins><span class="cx"> compileStoreBarrier(node);
</span><span class="cx"> break;
</span><span class="cx"> }
</span><span class="lines">@@ -5474,20 +5463,6 @@
</span><span class="cx"> use(node);
</span><span class="cx"> }
</span><span class="cx">
</span><del>-void SpeculativeJIT::writeBarrier(GPRReg ownerGPR, GPRReg valueTagGPR, Edge valueUse, GPRReg scratch1, GPRReg scratch2)
-{
- JITCompiler::Jump isNotCell;
- if (!isKnownCell(valueUse.node()))
- isNotCell = m_jit.branch32(JITCompiler::NotEqual, valueTagGPR, JITCompiler::TrustedImm32(JSValue::CellTag));
-
- JITCompiler::Jump ownerIsRememberedOrInEden = m_jit.jumpIfIsRememberedOrInEden(ownerGPR);
- storeToWriteBarrierBuffer(ownerGPR, scratch1, scratch2);
- ownerIsRememberedOrInEden.link(&m_jit);
-
- if (!isKnownCell(valueUse.node()))
- isNotCell.link(&m_jit);
-}
-
</del><span class="cx"> void SpeculativeJIT::moveTrueTo(GPRReg gpr)
</span><span class="cx"> {
</span><span class="cx"> m_jit.move(TrustedImm32(1), gpr);
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoredfgDFGSpeculativeJIT64cpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/dfg/DFGSpeculativeJIT64.cpp (206554 => 206555)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/dfg/DFGSpeculativeJIT64.cpp 2016-09-28 21:53:53 UTC (rev 206554)
+++ trunk/Source/JavaScriptCore/dfg/DFGSpeculativeJIT64.cpp 2016-09-28 21:55:53 UTC (rev 206555)
</span><span class="lines">@@ -1376,18 +1376,6 @@
</span><span class="cx"> }
</span><span class="cx"> }
</span><span class="cx">
</span><del>-void SpeculativeJIT::compileBaseValueStoreBarrier(Edge& baseEdge, Edge& valueEdge)
-{
- ASSERT(!isKnownNotCell(valueEdge.node()));
-
- SpeculateCellOperand base(this, baseEdge);
- JSValueOperand value(this, valueEdge);
- GPRTemporary scratch1(this);
- GPRTemporary scratch2(this);
-
- writeBarrier(base.gpr(), value.gpr(), valueEdge, scratch1.gpr(), scratch2.gpr());
-}
-
</del><span class="cx"> void SpeculativeJIT::compileObjectEquality(Node* node)
</span><span class="cx"> {
</span><span class="cx"> SpeculateCellOperand op1(this, node->child1());
</span><span class="lines">@@ -5079,11 +5067,12 @@
</span><span class="cx"> unreachable(node);
</span><span class="cx"> break;
</span><span class="cx">
</span><del>- case StoreBarrier: {
</del><ins>+ case StoreBarrier:
+ case FencedStoreBarrier: {
</ins><span class="cx"> compileStoreBarrier(node);
</span><span class="cx"> break;
</span><span class="cx"> }
</span><del>-
</del><ins>+
</ins><span class="cx"> case GetEnumerableLength: {
</span><span class="cx"> SpeculateCellOperand enumerator(this, node->child1());
</span><span class="cx"> GPRFlushedCallResult result(this);
</span><span class="lines">@@ -5575,20 +5564,6 @@
</span><span class="cx"> use(node);
</span><span class="cx"> }
</span><span class="cx">
</span><del>-void SpeculativeJIT::writeBarrier(GPRReg ownerGPR, GPRReg valueGPR, Edge valueUse, GPRReg scratch1, GPRReg scratch2)
-{
- JITCompiler::Jump isNotCell;
- if (!isKnownCell(valueUse.node()))
- isNotCell = m_jit.branchIfNotCell(JSValueRegs(valueGPR));
-
- JITCompiler::Jump ownerIsRememberedOrInEden = m_jit.jumpIfIsRememberedOrInEden(ownerGPR);
- storeToWriteBarrierBuffer(ownerGPR, scratch1, scratch2);
- ownerIsRememberedOrInEden.link(&m_jit);
-
- if (!isKnownCell(valueUse.node()))
- isNotCell.link(&m_jit);
-}
-
</del><span class="cx"> void SpeculativeJIT::moveTrueTo(GPRReg gpr)
</span><span class="cx"> {
</span><span class="cx"> m_jit.move(TrustedImm32(ValueTrue), gpr);
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoredfgDFGStoreBarrierClusteringPhasecpp"></a>
<div class="addfile"><h4>Added: trunk/Source/JavaScriptCore/dfg/DFGStoreBarrierClusteringPhase.cpp (0 => 206555)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/dfg/DFGStoreBarrierClusteringPhase.cpp (rev 0)
+++ trunk/Source/JavaScriptCore/dfg/DFGStoreBarrierClusteringPhase.cpp 2016-09-28 21:55:53 UTC (rev 206555)
</span><span class="lines">@@ -0,0 +1,173 @@
</span><ins>+/*
+ * Copyright (C) 2016 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS''
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS
+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
+ * THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include "config.h"
+#include "DFGStoreBarrierClusteringPhase.h"
+
+#if ENABLE(DFG_JIT)
+
+#include "DFGDoesGC.h"
+#include "DFGGraph.h"
+#include "DFGInsertionSet.h"
+#include "DFGMayExit.h"
+#include "DFGPhase.h"
+#include "JSCInlines.h"
+#include <wtf/FastBitVector.h>
+
+namespace JSC { namespace DFG {
+
+namespace {
+
+class StoreBarrierClusteringPhase : public Phase {
+public:
+ StoreBarrierClusteringPhase(Graph& graph)
+ : Phase(graph, "store barrier fencing")
+ , m_insertionSet(graph)
+ {
+ }
+
+ bool run()
+ {
+ size_t maxSize = 0;
+ for (BasicBlock* block : m_graph.blocksInNaturalOrder())
+ maxSize = std::max(maxSize, block->size());
+ m_barrierPoints.resize(maxSize);
+
+ for (BasicBlock* block : m_graph.blocksInNaturalOrder()) {
+ size_t blockSize = block->size();
+ doBlock(block);
+ m_barrierPoints.clearRange(0, blockSize);
+ }
+
+ return true;
+ }
+
+private:
+ // This summarizes everything we need to remember about a barrier.
+ struct ChildAndOrigin {
+ ChildAndOrigin() { }
+
+ ChildAndOrigin(Node* child, CodeOrigin semanticOrigin)
+ : child(child)
+ , semanticOrigin(semanticOrigin)
+ {
+ }
+
+ Node* child { nullptr };
+ CodeOrigin semanticOrigin;
+ };
+
+ void doBlock(BasicBlock* block)
+ {
+ ASSERT(m_barrierPoints.isEmpty());
+
+ // First identify the places where we would want to place all of the barriers based on a
+ // backwards analysis. We use the futureGC flag to tell us if we had seen a GC. Since this
+ // is a backwards analysis, when we get to a node, futureGC tells us if a GC will happen
+ // in the future after that node.
+ bool futureGC = true;
+ for (unsigned nodeIndex = block->size(); nodeIndex--;) {
+ Node* node = block->at(nodeIndex);
+
+ // This is a backwards analysis, so exits require conservatism. If we exit, then there
+ // probably will be a GC in the future! If we needed to then we could lift that
+ // requirement by either (1) having a StoreBarrierHint that tells OSR exit to barrier that
+ // value or (2) automatically barriering any DFG-live Node on OSR exit. Either way, it
+ // would be weird because it would create a new root for OSR availability analysis. I
+ // don't have evidence that it would be worth it.
+ if (doesGC(m_graph, node) || mayExit(m_graph, node) != DoesNotExit) {
+ futureGC = true;
+ continue;
+ }
+
+ if (node->isStoreBarrier() && futureGC) {
+ m_barrierPoints[nodeIndex] = true;
+ futureGC = false;
+ }
+ }
+
+ // Now we run forward and collect the barriers. When we hit a barrier point, insert all of
+ // them with a fence.
+ for (unsigned nodeIndex = 0; nodeIndex < block->size(); ++nodeIndex) {
+ Node* node = block->at(nodeIndex);
+ if (!node->isStoreBarrier())
+ continue;
+
+ DFG_ASSERT(m_graph, node, !node->origin.wasHoisted);
+ DFG_ASSERT(m_graph, node, node->child1().useKind() == KnownCellUse);
+
+ NodeOrigin origin = node->origin;
+ m_neededBarriers.append(ChildAndOrigin(node->child1().node(), origin.semantic));
+ node->remove();
+
+ if (!m_barrierPoints[nodeIndex])
+ continue;
+
+ std::sort(
+ m_neededBarriers.begin(), m_neededBarriers.end(),
+ [&] (const ChildAndOrigin& a, const ChildAndOrigin& b) -> bool {
+ return a.child < b.child;
+ });
+ auto end = std::unique(
+ m_neededBarriers.begin(), m_neededBarriers.end(),
+ [&] (const ChildAndOrigin& a, const ChildAndOrigin& b) -> bool{
+ return a.child == b.child;
+ });
+ for (auto iter = m_neededBarriers.begin(); iter != end; ++iter) {
+ Node* child = iter->child;
+ CodeOrigin semanticOrigin = iter->semanticOrigin;
+
+ NodeType type;
+ if (Options::useConcurrentBarriers() && iter == m_neededBarriers.begin())
+ type = FencedStoreBarrier;
+ else
+ type = StoreBarrier;
+
+ m_insertionSet.insertNode(
+ nodeIndex, SpecNone, type, origin.withSemantic(semanticOrigin),
+ Edge(child, KnownCellUse));
+ }
+ m_neededBarriers.resize(0);
+ }
+
+ m_insertionSet.execute(block);
+ }
+
+ InsertionSet m_insertionSet;
+ FastBitVector m_barrierPoints;
+ Vector<ChildAndOrigin> m_neededBarriers;
+};
+
+} // anonymous namespace
+
+bool performStoreBarrierClustering(Graph& graph)
+{
+ return runPhase<StoreBarrierClusteringPhase>(graph);
+}
+
+} } // namespace JSC::DFG
+
+#endif // ENABLE(DFG_JIT)
+
</ins></span></pre></div>
<a id="trunkSourceJavaScriptCoredfgDFGStoreBarrierClusteringPhaseh"></a>
<div class="addfile"><h4>Added: trunk/Source/JavaScriptCore/dfg/DFGStoreBarrierClusteringPhase.h (0 => 206555)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/dfg/DFGStoreBarrierClusteringPhase.h (rev 0)
+++ trunk/Source/JavaScriptCore/dfg/DFGStoreBarrierClusteringPhase.h 2016-09-28 21:55:53 UTC (rev 206555)
</span><span class="lines">@@ -0,0 +1,91 @@
</span><ins>+/*
+ * Copyright (C) 2016 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS''
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS
+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
+ * THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#pragma once
+
+#if ENABLE(DFG_JIT)
+
+namespace JSC { namespace DFG {
+
+class Graph;
+
+// Picks up groups of barriers that could be executed in any order with respect to each other and
+// places then at the earliest point in the program where the cluster would be correct. This phase
+// makes only the first of the cluster be a FencedStoreBarrier while the rest are normal
+// StoreBarriers. This phase also removes redundant barriers - for example, the cluster may end up
+// with two or more barriers on the same object, in which case it is totally safe for us to drop
+// one of them. The reason why this is sound hinges on the "earliest point where the cluster would
+// be correct" property. For example, take this input:
+//
+// a: Call()
+// b: PutByOffset(@o, @o, @x)
+// c: FencedStoreBarrier(@o)
+// d: PutByOffset(@o, @o, @y)
+// e: FencedStoreBarrier(@o)
+// f: PutByOffset(@p, @p, @z)
+// g: FencedStoreBarrier(@p)
+// h: GetByOffset(@q)
+// i: Call()
+//
+// The cluster of barriers is @c, @e, and @g. All of the barriers are between two doesGC effects:
+// the calls at @a and @i. Because there are no doesGC effects between @a and @i and there is no
+// possible control flow entry into this sequence between @ and @i, we could could just execute all
+// of the barriers just before @i in any order. The earliest point where the cluster would be
+// correct is just after @f, since that's the last operation that needs a barrier. We use the
+// earliest to reduce register pressure. When the barriers are clustered just after @f, we get:
+//
+// a: Call()
+// b: PutByOffset(@o, @o, @x)
+// d: PutByOffset(@o, @o, @y)
+// f: PutByOffset(@p, @p, @z)
+// c: FencedStoreBarrier(@o)
+// e: FencedStoreBarrier(@o)
+// g: FencedStoreBarrier(@p)
+// h: GetByOffset(@q)
+// i: Call()
+//
+// This phase does more. It takes advantage of the clustering to remove fences and remove redundant
+// barriers. So this phase will output this:
+//
+// a: Call()
+// b: PutByOffset(@o, @o, @x)
+// d: PutByOffset(@o, @o, @y)
+// f: PutByOffset(@p, @p, @z)
+// c: FencedStoreBarrier(@o)
+// g: StoreBarrier(@p)
+// h: GetByOffset(@q)
+// i: Call()
+//
+// This optimization improves both overall throughput and the throughput while the concurrent GC is
+// running. In the former, we are simplifying instruction selection for all but the first fence. In
+// the latter, we are reducing the cost of all but the first barrier. The first barrier will awlays
+// take slow path when there is concurrent GC activity, since the slow path contains the fence. But
+// all of the other barriers will only take slow path if they really need to remember the object.
+bool performStoreBarrierClustering(Graph&);
+
+} } // namespace JSC::DFG
+
+#endif // ENABLE(DFG_JIT)
+
</ins></span></pre></div>
<a id="trunkSourceJavaScriptCoredfgDFGStoreBarrierInsertionPhasecpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/dfg/DFGStoreBarrierInsertionPhase.cpp (206554 => 206555)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/dfg/DFGStoreBarrierInsertionPhase.cpp 2016-09-28 21:53:53 UTC (rev 206554)
+++ trunk/Source/JavaScriptCore/dfg/DFGStoreBarrierInsertionPhase.cpp 2016-09-28 21:55:53 UTC (rev 206555)
</span><span class="lines">@@ -321,7 +321,7 @@
</span><span class="cx"> case AllocatePropertyStorage:
</span><span class="cx"> case ReallocatePropertyStorage:
</span><span class="cx"> // These allocate but then run their own barrier.
</span><del>- insertBarrier(m_nodeIndex + 1, Edge(m_node->child1().node(), KnownCellUse));
</del><ins>+ insertBarrier(m_nodeIndex + 1, m_node->child1());
</ins><span class="cx"> m_node->setEpoch(Epoch());
</span><span class="cx"> break;
</span><span class="cx">
</span><span class="lines">@@ -430,13 +430,14 @@
</span><span class="cx">
</span><span class="cx"> if (verbose)
</span><span class="cx"> dataLog(" Inserting barrier.\n");
</span><del>- insertBarrier(m_nodeIndex, base);
</del><ins>+ insertBarrier(m_nodeIndex + 1, base);
</ins><span class="cx"> }
</span><span class="cx">
</span><del>- void insertBarrier(unsigned nodeIndex, Edge base, bool exitOK = true)
</del><ins>+ void insertBarrier(unsigned nodeIndex, Edge base)
</ins><span class="cx"> {
</span><del>- // Inserting a barrier means that the object may become marked by the GC, which will make
- // the object black.
</del><ins>+ // This is just our way of saying that barriers are not redundant with each other according
+ // to forward analysis: if we proved one time that a barrier was necessary then it'll for
+ // sure be necessary next time.
</ins><span class="cx"> base->setEpoch(Epoch());
</span><span class="cx">
</span><span class="cx"> // If we're in global mode, we should only insert the barriers once we have converged.
</span><span class="lines">@@ -446,17 +447,23 @@
</span><span class="cx"> // FIXME: We could support StoreBarrier(UntypedUse:). That would be sort of cool.
</span><span class="cx"> // But right now we don't need it.
</span><span class="cx">
</span><del>- // If the original edge was unchecked, we should also not have a check. We may be in a context
- // where checks are not allowed. If we ever did have to insert a barrier at an ExitInvalid
- // context and that barrier needed a check, then we could make that work by hoisting the check.
- // That doesn't happen right now.
- if (base.useKind() != KnownCellUse) {
- DFG_ASSERT(m_graph, m_node, m_node->origin.exitOK);
- base.setUseKind(CellUse);
- }
</del><ins>+ DFG_ASSERT(m_graph, m_node, isCell(base.useKind()));
</ins><span class="cx">
</span><del>- m_insertionSet.insertNode(
- nodeIndex, SpecNone, StoreBarrier, m_node->origin.takeValidExit(exitOK), base);
</del><ins>+ // Barriers are always inserted after the node that they service. Therefore, we always know
+ // that the thing is a cell now.
+ base.setUseKind(KnownCellUse);
+
+ NodeOrigin origin = m_node->origin;
+ if (clobbersExitState(m_graph, m_node))
+ origin = origin.withInvalidExit();
+
+ NodeType type;
+ if (Options::useConcurrentBarriers())
+ type = FencedStoreBarrier;
+ else
+ type = StoreBarrier;
+
+ m_insertionSet.insertNode(nodeIndex, SpecNone, type, origin, base);
</ins><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> bool reallyInsertBarriers()
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoredfgDFGStoreBarrierInsertionPhaseh"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/dfg/DFGStoreBarrierInsertionPhase.h (206554 => 206555)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/dfg/DFGStoreBarrierInsertionPhase.h 2016-09-28 21:53:53 UTC (rev 206554)
+++ trunk/Source/JavaScriptCore/dfg/DFGStoreBarrierInsertionPhase.h 2016-09-28 21:55:53 UTC (rev 206555)
</span><span class="lines">@@ -32,14 +32,13 @@
</span><span class="cx"> class Graph;
</span><span class="cx">
</span><span class="cx"> // Inserts store barriers in a block-local manner without consulting the abstract interpreter.
</span><del>-// Uses a simple epoch-based analysis to avoid inserting redundant barriers. This phase requires
-// that we are not in SSA.
</del><ins>+// Uses a simple epoch-based analysis to avoid inserting barriers on newly allocated objects. This
+// phase requires that we are not in SSA.
</ins><span class="cx"> bool performFastStoreBarrierInsertion(Graph&);
</span><span class="cx">
</span><span class="cx"> // Inserts store barriers using a global analysis and consults the abstract interpreter. Uses a
</span><del>-// simple epoch-based analysis to avoid inserting redundant barriers, but only propagates "same
-// epoch as current" property from one block to the next. This phase requires SSA. This phase
-// also requires having valid AI and liveness.
</del><ins>+// simple epoch-based analysis to avoid inserting barriers on newly allocated objects. This phase
+// requires SSA. This phase also requires having valid AI and liveness.
</ins><span class="cx"> bool performGlobalStoreBarrierInsertion(Graph&);
</span><span class="cx">
</span><span class="cx"> } } // namespace JSC::DFG
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreftlFTLAbstractHeaph"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/ftl/FTLAbstractHeap.h (206554 => 206555)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/ftl/FTLAbstractHeap.h 2016-09-28 21:53:53 UTC (rev 206554)
+++ trunk/Source/JavaScriptCore/ftl/FTLAbstractHeap.h 2016-09-28 21:55:53 UTC (rev 206555)
</span><span class="lines">@@ -195,12 +195,12 @@
</span><span class="cx">
</span><span class="cx"> const AbstractHeap& atAnyAddress() const { return m_indexedHeap.atAnyIndex(); }
</span><span class="cx">
</span><del>- const AbstractHeap& at(void* address)
</del><ins>+ const AbstractHeap& at(const void* address)
</ins><span class="cx"> {
</span><span class="cx"> return m_indexedHeap.at(bitwise_cast<ptrdiff_t>(address));
</span><span class="cx"> }
</span><span class="cx">
</span><del>- const AbstractHeap& operator[](void* address) { return at(address); }
</del><ins>+ const AbstractHeap& operator[](const void* address) { return at(address); }
</ins><span class="cx">
</span><span class="cx"> void dump(PrintStream&) const;
</span><span class="cx">
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreftlFTLAbstractHeapRepositorycpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/ftl/FTLAbstractHeapRepository.cpp (206554 => 206555)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/ftl/FTLAbstractHeapRepository.cpp 2016-09-28 21:53:53 UTC (rev 206554)
+++ trunk/Source/JavaScriptCore/ftl/FTLAbstractHeapRepository.cpp 2016-09-28 21:55:53 UTC (rev 206555)
</span><span class="lines">@@ -29,6 +29,7 @@
</span><span class="cx"> #if ENABLE(FTL_JIT)
</span><span class="cx">
</span><span class="cx"> #include "B3CCallValue.h"
</span><ins>+#include "B3FenceValue.h"
</ins><span class="cx"> #include "B3MemoryValue.h"
</span><span class="cx"> #include "B3PatchpointValue.h"
</span><span class="cx"> #include "B3ValueInlines.h"
</span><span class="lines">@@ -116,6 +117,16 @@
</span><span class="cx"> m_heapForPatchpointWrite.append(HeapForValue(heap, value));
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+void AbstractHeapRepository::decorateFenceRead(const AbstractHeap* heap, Value* value)
+{
+ m_heapForFenceRead.append(HeapForValue(heap, value));
+}
+
+void AbstractHeapRepository::decorateFenceWrite(const AbstractHeap* heap, Value* value)
+{
+ m_heapForFenceWrite.append(HeapForValue(heap, value));
+}
+
</ins><span class="cx"> void AbstractHeapRepository::computeRangesAndDecorateInstructions()
</span><span class="cx"> {
</span><span class="cx"> root.compute();
</span><span class="lines">@@ -132,9 +143,13 @@
</span><span class="cx"> for (HeapForValue entry : m_heapForCCallWrite)
</span><span class="cx"> entry.value->as<CCallValue>()->effects.writes = entry.heap->range();
</span><span class="cx"> for (HeapForValue entry : m_heapForPatchpointRead)
</span><del>- entry.value->as<CCallValue>()->effects.reads = entry.heap->range();
</del><ins>+ entry.value->as<PatchpointValue>()->effects.reads = entry.heap->range();
</ins><span class="cx"> for (HeapForValue entry : m_heapForPatchpointWrite)
</span><del>- entry.value->as<CCallValue>()->effects.writes = entry.heap->range();
</del><ins>+ entry.value->as<PatchpointValue>()->effects.writes = entry.heap->range();
+ for (HeapForValue entry : m_heapForFenceRead)
+ entry.value->as<FenceValue>()->read = entry.heap->range();
+ for (HeapForValue entry : m_heapForFenceWrite)
+ entry.value->as<FenceValue>()->write = entry.heap->range();
</ins><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> } } // namespace JSC::FTL
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreftlFTLAbstractHeapRepositoryh"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/ftl/FTLAbstractHeapRepository.h (206554 => 206555)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/ftl/FTLAbstractHeapRepository.h 2016-09-28 21:53:53 UTC (rev 206554)
+++ trunk/Source/JavaScriptCore/ftl/FTLAbstractHeapRepository.h 2016-09-28 21:55:53 UTC (rev 206555)
</span><span class="lines">@@ -215,6 +215,8 @@
</span><span class="cx"> void decorateCCallWrite(const AbstractHeap*, B3::Value*);
</span><span class="cx"> void decoratePatchpointRead(const AbstractHeap*, B3::Value*);
</span><span class="cx"> void decoratePatchpointWrite(const AbstractHeap*, B3::Value*);
</span><ins>+ void decorateFenceRead(const AbstractHeap*, B3::Value*);
+ void decorateFenceWrite(const AbstractHeap*, B3::Value*);
</ins><span class="cx">
</span><span class="cx"> void computeRangesAndDecorateInstructions();
</span><span class="cx">
</span><span class="lines">@@ -240,6 +242,8 @@
</span><span class="cx"> Vector<HeapForValue> m_heapForCCallWrite;
</span><span class="cx"> Vector<HeapForValue> m_heapForPatchpointRead;
</span><span class="cx"> Vector<HeapForValue> m_heapForPatchpointWrite;
</span><ins>+ Vector<HeapForValue> m_heapForFenceRead;
+ Vector<HeapForValue> m_heapForFenceWrite;
</ins><span class="cx"> };
</span><span class="cx">
</span><span class="cx"> } } // namespace JSC::FTL
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreftlFTLCapabilitiescpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/ftl/FTLCapabilities.cpp (206554 => 206555)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/ftl/FTLCapabilities.cpp 2016-09-28 21:53:53 UTC (rev 206554)
+++ trunk/Source/JavaScriptCore/ftl/FTLCapabilities.cpp 2016-09-28 21:55:53 UTC (rev 206555)
</span><span class="lines">@@ -138,6 +138,7 @@
</span><span class="cx"> case GetTypedArrayByteOffset:
</span><span class="cx"> case NotifyWrite:
</span><span class="cx"> case StoreBarrier:
</span><ins>+ case FencedStoreBarrier:
</ins><span class="cx"> case Call:
</span><span class="cx"> case TailCall:
</span><span class="cx"> case TailCallInlinedCaller:
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreftlFTLLowerDFGToB3cpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/ftl/FTLLowerDFGToB3.cpp (206554 => 206555)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/ftl/FTLLowerDFGToB3.cpp 2016-09-28 21:53:53 UTC (rev 206554)
+++ trunk/Source/JavaScriptCore/ftl/FTLLowerDFGToB3.cpp 2016-09-28 21:55:53 UTC (rev 206555)
</span><span class="lines">@@ -31,6 +31,7 @@
</span><span class="cx"> #include "AirGenerationContext.h"
</span><span class="cx"> #include "AllowMacroScratchRegisterUsage.h"
</span><span class="cx"> #include "B3CheckValue.h"
</span><ins>+#include "B3FenceValue.h"
</ins><span class="cx"> #include "B3PatchpointValue.h"
</span><span class="cx"> #include "B3SlotBaseValue.h"
</span><span class="cx"> #include "B3StackmapGenerationParams.h"
</span><span class="lines">@@ -945,6 +946,7 @@
</span><span class="cx"> compileCountExecution();
</span><span class="cx"> break;
</span><span class="cx"> case StoreBarrier:
</span><ins>+ case FencedStoreBarrier:
</ins><span class="cx"> compileStoreBarrier();
</span><span class="cx"> break;
</span><span class="cx"> case HasIndexedProperty:
</span><span class="lines">@@ -6997,9 +6999,9 @@
</span><span class="cx">
</span><span class="cx"> void compileStoreBarrier()
</span><span class="cx"> {
</span><del>- emitStoreBarrier(lowCell(m_node->child1()));
</del><ins>+ emitStoreBarrier(lowCell(m_node->child1()), m_node->op() == FencedStoreBarrier);
</ins><span class="cx"> }
</span><del>-
</del><ins>+
</ins><span class="cx"> void compileHasIndexedProperty()
</span><span class="cx"> {
</span><span class="cx"> switch (m_node->arrayMode().type()) {
</span><span class="lines">@@ -8128,8 +8130,6 @@
</span><span class="cx"> previousStructure, nextStructure);
</span><span class="cx"> }
</span><span class="cx">
</span><del>- emitStoreBarrier(object);
-
</del><span class="cx"> return result;
</span><span class="cx"> }
</span><span class="cx">
</span><span class="lines">@@ -10055,13 +10055,13 @@
</span><span class="cx"> // run after the function that created them returns. Hence, you should not use by-reference
</span><span class="cx"> // capture (i.e. [&]) in any of these lambdas.
</span><span class="cx"> template<typename Functor, typename... ArgumentTypes>
</span><del>- LValue lazySlowPath(const Functor& functor, ArgumentTypes... arguments)
</del><ins>+ PatchpointValue* lazySlowPath(const Functor& functor, ArgumentTypes... arguments)
</ins><span class="cx"> {
</span><span class="cx"> return lazySlowPath(functor, Vector<LValue>{ arguments... });
</span><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> template<typename Functor>
</span><del>- LValue lazySlowPath(const Functor& functor, const Vector<LValue>& userArguments)
</del><ins>+ PatchpointValue* lazySlowPath(const Functor& functor, const Vector<LValue>& userArguments)
</ins><span class="cx"> {
</span><span class="cx"> CodeOrigin origin = m_node->origin.semantic;
</span><span class="cx">
</span><span class="lines">@@ -11431,26 +11431,39 @@
</span><span class="cx"> return m_out.load8ZeroExt32(base, m_heaps.JSCell_cellState);
</span><span class="cx"> }
</span><span class="cx">
</span><del>- void emitStoreBarrier(LValue base)
</del><ins>+ void emitStoreBarrier(LValue base, bool isFenced)
</ins><span class="cx"> {
</span><span class="cx"> LBasicBlock slowPath = m_out.newBlock();
</span><span class="cx"> LBasicBlock continuation = m_out.newBlock();
</span><span class="cx">
</span><ins>+ LValue threshold;
+ if (isFenced)
+ threshold = m_out.load32(m_out.absolute(vm().heap.addressOfBarrierThreshold()));
+ else
+ threshold = m_out.constInt32(blackThreshold);
+
</ins><span class="cx"> m_out.branch(
</span><del>- m_out.above(loadCellState(base), m_out.constInt32(blackThreshold)),
</del><ins>+ m_out.above(loadCellState(base), threshold),
</ins><span class="cx"> usually(continuation), rarely(slowPath));
</span><span class="cx">
</span><span class="cx"> LBasicBlock lastNext = m_out.appendTo(slowPath, continuation);
</span><del>-
</del><ins>+
</ins><span class="cx"> // We emit the store barrier slow path lazily. In a lot of cases, this will never fire. And
</span><span class="cx"> // when it does fire, it makes sense for us to generate this code using our JIT rather than
</span><span class="cx"> // wasting B3's time optimizing it.
</span><del>- lazySlowPath(
</del><ins>+ PatchpointValue* patchpoint = lazySlowPath(
</ins><span class="cx"> [=] (const Vector<Location>& locations) -> RefPtr<LazySlowPath::Generator> {
</span><span class="cx"> GPRReg baseGPR = locations[1].directGPR();
</span><span class="cx">
</span><span class="cx"> return LazySlowPath::createGenerator(
</span><span class="cx"> [=] (CCallHelpers& jit, LazySlowPath::GenerationParams& params) {
</span><ins>+ if (isFenced) {
+ CCallHelpers::Jump noFence = jit.jumpIfBarrierStoreLoadFenceNotNeeded();
+ jit.memoryFence();
+ params.doneJumps.append(jit.barrierBranchWithoutFence(baseGPR));
+ noFence.link(&jit);
+ }
+
</ins><span class="cx"> RegisterSet usedRegisters = params.lazySlowPath->usedRegisters();
</span><span class="cx"> ScratchRegisterAllocator scratchRegisterAllocator(usedRegisters);
</span><span class="cx"> scratchRegisterAllocator.lock(baseGPR);
</span><span class="lines">@@ -11495,6 +11508,13 @@
</span><span class="cx"> });
</span><span class="cx"> },
</span><span class="cx"> base);
</span><ins>+
+ if (isFenced)
+ m_heaps.decoratePatchpointRead(&m_heaps.root, patchpoint);
+ else
+ m_heaps.decoratePatchpointRead(&m_heaps.JSCell_cellState, patchpoint);
+ m_heaps.decoratePatchpointWrite(&m_heaps.JSCell_cellState, patchpoint);
+
</ins><span class="cx"> m_out.jump(continuation);
</span><span class="cx">
</span><span class="cx"> m_out.appendTo(continuation, lastNext);
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreftlFTLOutputcpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/ftl/FTLOutput.cpp (206554 => 206555)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/ftl/FTLOutput.cpp 2016-09-28 21:53:53 UTC (rev 206554)
+++ trunk/Source/JavaScriptCore/ftl/FTLOutput.cpp 2016-09-28 21:55:53 UTC (rev 206555)
</span><span class="lines">@@ -33,6 +33,7 @@
</span><span class="cx"> #include "B3CCallValue.h"
</span><span class="cx"> #include "B3Const32Value.h"
</span><span class="cx"> #include "B3ConstPtrValue.h"
</span><ins>+#include "B3FenceValue.h"
</ins><span class="cx"> #include "B3MathExtras.h"
</span><span class="cx"> #include "B3MemoryValue.h"
</span><span class="cx"> #include "B3SlotBaseValue.h"
</span><span class="lines">@@ -445,6 +446,11 @@
</span><span class="cx"> m_heaps->decorateMemory(pointer.heap(), store);
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+FenceValue* Output::fence()
+{
+ return m_block->appendNew<FenceValue>(m_proc, origin());
+}
+
</ins><span class="cx"> void Output::store32As8(LValue value, TypedPointer pointer)
</span><span class="cx"> {
</span><span class="cx"> LValue store = m_block->appendNew<MemoryValue>(m_proc, Store8, origin(), value, pointer.value());
</span><span class="lines">@@ -795,7 +801,7 @@
</span><span class="cx"> RELEASE_ASSERT_NOT_REACHED();
</span><span class="cx"> }
</span><span class="cx">
</span><del>-TypedPointer Output::absolute(void* address)
</del><ins>+TypedPointer Output::absolute(const void* address)
</ins><span class="cx"> {
</span><span class="cx"> return TypedPointer(m_heaps->absolute[address], constIntPtr(address));
</span><span class="cx"> }
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreftlFTLOutputh"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/ftl/FTLOutput.h (206554 => 206555)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/ftl/FTLOutput.h 2016-09-28 21:53:53 UTC (rev 206554)
+++ trunk/Source/JavaScriptCore/ftl/FTLOutput.h 2016-09-28 21:55:53 UTC (rev 206555)
</span><span class="lines">@@ -61,6 +61,7 @@
</span><span class="cx"> } // namespace DFG
</span><span class="cx">
</span><span class="cx"> namespace B3 {
</span><ins>+class FenceValue;
</ins><span class="cx"> class SlotBaseValue;
</span><span class="cx"> } // namespace B3
</span><span class="cx">
</span><span class="lines">@@ -188,6 +189,7 @@
</span><span class="cx">
</span><span class="cx"> LValue load(TypedPointer, LType);
</span><span class="cx"> void store(LValue, TypedPointer);
</span><ins>+ B3::FenceValue* fence();
</ins><span class="cx">
</span><span class="cx"> LValue load8SignExt32(TypedPointer);
</span><span class="cx"> LValue load8ZeroExt32(TypedPointer);
</span><span class="lines">@@ -284,7 +286,7 @@
</span><span class="cx"> return heap.baseIndex(*this, base, index, indexAsConstant, offset);
</span><span class="cx"> }
</span><span class="cx">
</span><del>- TypedPointer absolute(void* address);
</del><ins>+ TypedPointer absolute(const void* address);
</ins><span class="cx">
</span><span class="cx"> LValue load8SignExt32(LValue base, const AbstractHeap& field) { return load8SignExt32(address(base, field)); }
</span><span class="cx"> LValue load8ZeroExt32(LValue base, const AbstractHeap& field) { return load8ZeroExt32(address(base, field)); }
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreheapCellStateh"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/heap/CellState.h (206554 => 206555)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/heap/CellState.h 2016-09-28 21:53:53 UTC (rev 206554)
+++ trunk/Source/JavaScriptCore/heap/CellState.h 2016-09-28 21:55:53 UTC (rev 206555)
</span><span class="lines">@@ -52,10 +52,16 @@
</span><span class="cx"> };
</span><span class="cx">
</span><span class="cx"> static const unsigned blackThreshold = 1; // x <= blackThreshold means x is black.
</span><ins>+static const unsigned tautologicalThreshold = 100; // x <= tautologicalThreshold is always true.
</ins><span class="cx">
</span><ins>+inline bool isWithinThreshold(CellState cellState, unsigned threshold)
+{
+ return static_cast<unsigned>(cellState) <= threshold;
+}
+
</ins><span class="cx"> inline bool isBlack(CellState cellState)
</span><span class="cx"> {
</span><del>- return static_cast<unsigned>(cellState) <= blackThreshold;
</del><ins>+ return isWithinThreshold(cellState, blackThreshold);
</ins><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> inline CellState blacken(CellState cellState)
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreheapGCDeferralContexth"></a>
<div class="addfile"><h4>Added: trunk/Source/JavaScriptCore/heap/GCDeferralContext.h (0 => 206555)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/heap/GCDeferralContext.h (rev 0)
+++ trunk/Source/JavaScriptCore/heap/GCDeferralContext.h 2016-09-28 21:55:53 UTC (rev 206555)
</span><span class="lines">@@ -0,0 +1,46 @@
</span><ins>+/*
+ * Copyright (C) 2016 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+ * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+ * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+ * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#pragma once
+
+namespace JSC {
+
+class Heap;
+class MarkedAllocator;
+
+class GCDeferralContext {
+ friend class Heap;
+ friend class MarkedAllocator;
+public:
+ inline GCDeferralContext(Heap&);
+ inline ~GCDeferralContext();
+
+private:
+ Heap& m_heap;
+ bool m_shouldGC { false };
+};
+
+} // namespace JSC
+
</ins></span></pre></div>
<a id="trunkSourceJavaScriptCoreheapGCDeferralContextInlinesh"></a>
<div class="addfile"><h4>Added: trunk/Source/JavaScriptCore/heap/GCDeferralContextInlines.h (0 => 206555)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/heap/GCDeferralContextInlines.h (rev 0)
+++ trunk/Source/JavaScriptCore/heap/GCDeferralContextInlines.h 2016-09-28 21:55:53 UTC (rev 206555)
</span><span class="lines">@@ -0,0 +1,49 @@
</span><ins>+/*
+ * Copyright (C) 2016 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+ * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+ * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+ * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#pragma once
+
+#include "GCDeferralContext.h"
+#include "Heap.h"
+
+namespace JSC {
+
+ALWAYS_INLINE GCDeferralContext::GCDeferralContext(Heap& heap)
+ : m_heap(heap)
+{
+}
+
+ALWAYS_INLINE GCDeferralContext::~GCDeferralContext()
+{
+ ASSERT(!DisallowGC::isGCDisallowedOnCurrentThread());
+#if ENABLE(GC_VALIDATION)
+ ASSERT(!m_heap.vm()->isInitializingObject());
+#endif
+ if (UNLIKELY(m_shouldGC))
+ m_heap.collectIfNecessaryOrDefer();
+}
+
+} // namespace JSC
+
</ins></span></pre></div>
<a id="trunkSourceJavaScriptCoreheapHeapcpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/heap/Heap.cpp (206554 => 206555)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/heap/Heap.cpp 2016-09-28 21:53:53 UTC (rev 206554)
+++ trunk/Source/JavaScriptCore/heap/Heap.cpp 2016-09-28 21:55:53 UTC (rev 206555)
</span><span class="lines">@@ -1549,4 +1549,17 @@
</span><span class="cx"> return m_codeBlocks->iterate(func);
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+void Heap::writeBarrierSlowPath(const JSCell* from)
+{
+ if (UNLIKELY(barrierShouldBeFenced())) {
+ // In this case, the barrierThreshold is the tautological threshold, so from could still be
+ // not black. But we can't know for sure until we fire off a fence.
+ WTF::storeLoadFence();
+ if (!isBlack(from->cellState()))
+ return;
+ }
+
+ addToRememberedSet(from);
+}
+
</ins><span class="cx"> } // namespace JSC
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreheapHeaph"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/heap/Heap.h (206554 => 206555)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/heap/Heap.h 2016-09-28 21:53:53 UTC (rev 206554)
+++ trunk/Source/JavaScriptCore/heap/Heap.h 2016-09-28 21:55:53 UTC (rev 206555)
</span><span class="lines">@@ -51,6 +51,7 @@
</span><span class="cx"> class AllocationScope;
</span><span class="cx"> class CodeBlock;
</span><span class="cx"> class CodeBlockSet;
</span><ins>+class GCDeferralContext;
</ins><span class="cx"> class EdenGCActivityCallback;
</span><span class="cx"> class ExecutableBase;
</span><span class="cx"> class FullGCActivityCallback;
</span><span class="lines">@@ -103,9 +104,14 @@
</span><span class="cx">
</span><span class="cx"> static size_t cellSize(const void*);
</span><span class="cx">
</span><del>- void writeBarrier(const JSCell*);
- void writeBarrier(const JSCell*, JSValue);
- void writeBarrier(const JSCell*, JSCell*);
</del><ins>+ void writeBarrier(const JSCell* from);
+ void writeBarrier(const JSCell* from, JSValue to);
+ void writeBarrier(const JSCell* from, JSCell* to);
+
+ void writeBarrierWithoutFence(const JSCell* from);
+
+ // Take this if you know that from->cellState() < barrierThreshold.
+ JS_EXPORT_PRIVATE void writeBarrierSlowPath(const JSCell* from);
</ins><span class="cx">
</span><span class="cx"> WriteBarrierBuffer& writeBarrierBuffer() { return m_writeBarrierBuffer; }
</span><span class="cx"> void flushWriteBarrierBuffer(JSCell*);
</span><span class="lines">@@ -148,6 +154,7 @@
</span><span class="cx"> MarkedAllocator* allocatorForAuxiliaryData(size_t bytes) { return m_objectSpace.auxiliaryAllocatorFor(bytes); }
</span><span class="cx"> void* allocateAuxiliary(JSCell* intendedOwner, size_t);
</span><span class="cx"> void* tryAllocateAuxiliary(JSCell* intendedOwner, size_t);
</span><ins>+ void* tryAllocateAuxiliary(GCDeferralContext*, JSCell* intendedOwner, size_t);
</ins><span class="cx"> void* tryReallocateAuxiliary(JSCell* intendedOwner, void* oldBase, size_t oldSize, size_t newSize);
</span><span class="cx"> void ascribeOwner(JSCell* intendedOwner, void*);
</span><span class="cx">
</span><span class="lines">@@ -165,7 +172,7 @@
</span><span class="cx">
</span><span class="cx"> bool shouldCollect();
</span><span class="cx"> JS_EXPORT_PRIVATE void collect(HeapOperation collectionType = AnyCollection);
</span><del>- bool collectIfNecessaryOrDefer(); // Returns true if it did collect.
</del><ins>+ bool collectIfNecessaryOrDefer(GCDeferralContext* = nullptr); // Returns true if it did collect.
</ins><span class="cx"> void collectAccordingToDeferGCProbability();
</span><span class="cx">
</span><span class="cx"> void completeAllJITPlans();
</span><span class="lines">@@ -253,6 +260,12 @@
</span><span class="cx">
</span><span class="cx"> void didAllocateBlock(size_t capacity);
</span><span class="cx"> void didFreeBlock(size_t capacity);
</span><ins>+
+ bool barrierShouldBeFenced() const { return m_barrierShouldBeFenced; }
+ const bool* addressOfBarrierShouldBeFenced() const { return &m_barrierShouldBeFenced; }
+
+ unsigned barrierThreshold() const { return m_barrierThreshold; }
+ const unsigned* addressOfBarrierThreshold() const { return &m_barrierThreshold; }
</ins><span class="cx">
</span><span class="cx"> private:
</span><span class="cx"> friend class AllocationScope;
</span><span class="lines">@@ -277,12 +290,17 @@
</span><span class="cx"> friend class WeakSet;
</span><span class="cx"> template<typename T> friend void* allocateCell(Heap&);
</span><span class="cx"> template<typename T> friend void* allocateCell(Heap&, size_t);
</span><ins>+ template<typename T> friend void* allocateCell(Heap&, GCDeferralContext*);
+ template<typename T> friend void* allocateCell(Heap&, GCDeferralContext*, size_t);
</ins><span class="cx">
</span><span class="cx"> void collectWithoutAnySweep(HeapOperation collectionType = AnyCollection);
</span><span class="cx">
</span><span class="cx"> void* allocateWithDestructor(size_t); // For use with objects with destructors.
</span><span class="cx"> void* allocateWithoutDestructor(size_t); // For use with objects without destructors.
</span><ins>+ void* allocateWithDestructor(GCDeferralContext*, size_t);
+ void* allocateWithoutDestructor(GCDeferralContext*, size_t);
</ins><span class="cx"> template<typename ClassType> void* allocateObjectOfType(size_t); // Chooses one of the methods above based on type.
</span><ins>+ template<typename ClassType> void* allocateObjectOfType(GCDeferralContext*, size_t);
</ins><span class="cx">
</span><span class="cx"> static const size_t minExtraMemory = 256;
</span><span class="cx">
</span><span class="lines">@@ -410,6 +428,8 @@
</span><span class="cx"> bool m_isSafeToCollect;
</span><span class="cx">
</span><span class="cx"> WriteBarrierBuffer m_writeBarrierBuffer;
</span><ins>+ bool m_barrierShouldBeFenced { Options::forceFencedBarrier() };
+ unsigned m_barrierThreshold { Options::forceFencedBarrier() ? tautologicalThreshold : blackThreshold };
</ins><span class="cx">
</span><span class="cx"> VM* m_vm;
</span><span class="cx"> double m_lastFullGCLength;
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreheapHeapInlinesh"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/heap/HeapInlines.h (206554 => 206555)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/heap/HeapInlines.h 2016-09-28 21:53:53 UTC (rev 206554)
+++ trunk/Source/JavaScriptCore/heap/HeapInlines.h 2016-09-28 21:55:53 UTC (rev 206555)
</span><span class="lines">@@ -25,6 +25,7 @@
</span><span class="cx">
</span><span class="cx"> #pragma once
</span><span class="cx">
</span><ins>+#include "GCDeferralContext.h"
</ins><span class="cx"> #include "Heap.h"
</span><span class="cx"> #include "HeapCellInlines.h"
</span><span class="cx"> #include "IndexingHeader.h"
</span><span class="lines">@@ -124,21 +125,33 @@
</span><span class="cx"> #if ENABLE(WRITE_BARRIER_PROFILING)
</span><span class="cx"> WriteBarrierCounters::countWriteBarrier();
</span><span class="cx"> #endif
</span><del>- if (!from || !isBlack(from->cellState()))
</del><ins>+ if (!from)
</ins><span class="cx"> return;
</span><del>- if (!to || to->cellState() != CellState::NewWhite)
</del><ins>+ if (!isWithinThreshold(from->cellState(), barrierThreshold()))
</ins><span class="cx"> return;
</span><del>- addToRememberedSet(from);
</del><ins>+ if (LIKELY(!to || to->cellState() != CellState::NewWhite))
+ return;
+ writeBarrierSlowPath(from);
</ins><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> inline void Heap::writeBarrier(const JSCell* from)
</span><span class="cx"> {
</span><span class="cx"> ASSERT_GC_OBJECT_LOOKS_VALID(const_cast<JSCell*>(from));
</span><del>- if (!from || !isBlack(from->cellState()))
</del><ins>+ if (!from)
</ins><span class="cx"> return;
</span><del>- addToRememberedSet(from);
</del><ins>+ if (UNLIKELY(isWithinThreshold(from->cellState(), barrierThreshold())))
+ writeBarrierSlowPath(from);
</ins><span class="cx"> }
</span><span class="cx">
</span><ins>+inline void Heap::writeBarrierWithoutFence(const JSCell* from)
+{
+ ASSERT_GC_OBJECT_LOOKS_VALID(const_cast<JSCell*>(from));
+ if (!from)
+ return;
+ if (UNLIKELY(isWithinThreshold(from->cellState(), blackThreshold)))
+ addToRememberedSet(from);
+}
+
</ins><span class="cx"> inline void Heap::reportExtraMemoryAllocated(size_t size)
</span><span class="cx"> {
</span><span class="cx"> if (size > minExtraMemory)
</span><span class="lines">@@ -213,6 +226,18 @@
</span><span class="cx"> return m_objectSpace.allocateWithoutDestructor(bytes);
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+inline void* Heap::allocateWithDestructor(GCDeferralContext* deferralContext, size_t bytes)
+{
+ ASSERT(isValidAllocation(bytes));
+ return m_objectSpace.allocateWithDestructor(deferralContext, bytes);
+}
+
+inline void* Heap::allocateWithoutDestructor(GCDeferralContext* deferralContext, size_t bytes)
+{
+ ASSERT(isValidAllocation(bytes));
+ return m_objectSpace.allocateWithoutDestructor(deferralContext, bytes);
+}
+
</ins><span class="cx"> template<typename ClassType>
</span><span class="cx"> inline void* Heap::allocateObjectOfType(size_t bytes)
</span><span class="cx"> {
</span><span class="lines">@@ -225,6 +250,16 @@
</span><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> template<typename ClassType>
</span><ins>+inline void* Heap::allocateObjectOfType(GCDeferralContext* deferralContext, size_t bytes)
+{
+ ASSERT((!ClassType::needsDestruction || (ClassType::StructureFlags & StructureIsImmortal) || std::is_convertible<ClassType, JSDestructibleObject>::value));
+
+ if (ClassType::needsDestruction)
+ return allocateWithDestructor(deferralContext, bytes);
+ return allocateWithoutDestructor(deferralContext, bytes);
+}
+
+template<typename ClassType>
</ins><span class="cx"> inline MarkedSpace::Subspace& Heap::subspaceForObjectOfType()
</span><span class="cx"> {
</span><span class="cx"> // JSCell::classInfo() expects objects allocated with normal destructor to derive from JSDestructibleObject.
</span><span class="lines">@@ -273,6 +308,17 @@
</span><span class="cx"> return result;
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+inline void* Heap::tryAllocateAuxiliary(GCDeferralContext* deferralContext, JSCell* intendedOwner, size_t bytes)
+{
+ void* result = m_objectSpace.tryAllocateAuxiliary(deferralContext, bytes);
+#if ENABLE(ALLOCATION_LOGGING)
+ dataLogF("JSC GC allocating %lu bytes of auxiliary for %p: %p.\n", bytes, intendedOwner, result);
+#else
+ UNUSED_PARAM(intendedOwner);
+#endif
+ return result;
+}
+
</ins><span class="cx"> inline void* Heap::tryReallocateAuxiliary(JSCell* intendedOwner, void* oldBase, size_t oldSize, size_t newSize)
</span><span class="cx"> {
</span><span class="cx"> void* newBase = tryAllocateAuxiliary(intendedOwner, newSize);
</span><span class="lines">@@ -312,12 +358,15 @@
</span><span class="cx"> m_deferralDepth--;
</span><span class="cx"> }
</span><span class="cx">
</span><del>-inline bool Heap::collectIfNecessaryOrDefer()
</del><ins>+inline bool Heap::collectIfNecessaryOrDefer(GCDeferralContext* deferralContext)
</ins><span class="cx"> {
</span><span class="cx"> if (!shouldCollect())
</span><span class="cx"> return false;
</span><span class="cx">
</span><del>- collect();
</del><ins>+ if (deferralContext)
+ deferralContext->m_shouldGC = true;
+ else
+ collect();
</ins><span class="cx"> return true;
</span><span class="cx"> }
</span><span class="cx">
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreheapMarkedAllocatorcpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/heap/MarkedAllocator.cpp (206554 => 206555)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/heap/MarkedAllocator.cpp 2016-09-28 21:53:53 UTC (rev 206554)
+++ trunk/Source/JavaScriptCore/heap/MarkedAllocator.cpp 2016-09-28 21:55:53 UTC (rev 206555)
</span><span class="lines">@@ -175,7 +175,7 @@
</span><span class="cx"> return result;
</span><span class="cx"> }
</span><span class="cx">
</span><del>-ALWAYS_INLINE void MarkedAllocator::doTestCollectionsIfNeeded()
</del><ins>+ALWAYS_INLINE void MarkedAllocator::doTestCollectionsIfNeeded(GCDeferralContext* deferralContext)
</ins><span class="cx"> {
</span><span class="cx"> if (!Options::slowPathAllocsBetweenGCs())
</span><span class="cx"> return;
</span><span class="lines">@@ -182,8 +182,12 @@
</span><span class="cx">
</span><span class="cx"> static unsigned allocationCount = 0;
</span><span class="cx"> if (!allocationCount) {
</span><del>- if (!m_heap->isDeferred())
- m_heap->collectAllGarbage();
</del><ins>+ if (!m_heap->isDeferred()) {
+ if (deferralContext)
+ deferralContext->m_shouldGC = true;
+ else
+ m_heap->collectAllGarbage();
+ }
</ins><span class="cx"> ASSERT(m_heap->m_operationInProgress == NoOperation);
</span><span class="cx"> }
</span><span class="cx"> if (++allocationCount >= Options::slowPathAllocsBetweenGCs())
</span><span class="lines">@@ -190,23 +194,23 @@
</span><span class="cx"> allocationCount = 0;
</span><span class="cx"> }
</span><span class="cx">
</span><del>-void* MarkedAllocator::allocateSlowCase()
</del><ins>+void* MarkedAllocator::allocateSlowCase(GCDeferralContext* deferralContext)
</ins><span class="cx"> {
</span><span class="cx"> bool crashOnFailure = true;
</span><del>- return allocateSlowCaseImpl(crashOnFailure);
</del><ins>+ return allocateSlowCaseImpl(deferralContext, crashOnFailure);
</ins><span class="cx"> }
</span><span class="cx">
</span><del>-void* MarkedAllocator::tryAllocateSlowCase()
</del><ins>+void* MarkedAllocator::tryAllocateSlowCase(GCDeferralContext* deferralContext)
</ins><span class="cx"> {
</span><span class="cx"> bool crashOnFailure = false;
</span><del>- return allocateSlowCaseImpl(crashOnFailure);
</del><ins>+ return allocateSlowCaseImpl(deferralContext, crashOnFailure);
</ins><span class="cx"> }
</span><span class="cx">
</span><del>-void* MarkedAllocator::allocateSlowCaseImpl(bool crashOnFailure)
</del><ins>+void* MarkedAllocator::allocateSlowCaseImpl(GCDeferralContext* deferralContext, bool crashOnFailure)
</ins><span class="cx"> {
</span><span class="cx"> SuperSamplerScope superSamplerScope(false);
</span><span class="cx"> ASSERT(m_heap->vm()->currentThreadIsHoldingAPILock());
</span><del>- doTestCollectionsIfNeeded();
</del><ins>+ doTestCollectionsIfNeeded(deferralContext);
</ins><span class="cx">
</span><span class="cx"> ASSERT(!m_markedSpace->isIterating());
</span><span class="cx"> m_heap->didAllocate(m_freeList.originalSize);
</span><span class="lines">@@ -213,7 +217,7 @@
</span><span class="cx">
</span><span class="cx"> didConsumeFreeList();
</span><span class="cx">
</span><del>- m_heap->collectIfNecessaryOrDefer();
</del><ins>+ m_heap->collectIfNecessaryOrDefer(deferralContext);
</ins><span class="cx">
</span><span class="cx"> AllocationScope allocationScope(*m_heap);
</span><span class="cx">
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreheapMarkedAllocatorh"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/heap/MarkedAllocator.h (206554 => 206555)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/heap/MarkedAllocator.h 2016-09-28 21:53:53 UTC (rev 206554)
+++ trunk/Source/JavaScriptCore/heap/MarkedAllocator.h 2016-09-28 21:55:53 UTC (rev 206555)
</span><span class="lines">@@ -34,6 +34,7 @@
</span><span class="cx">
</span><span class="cx"> namespace JSC {
</span><span class="cx">
</span><ins>+class GCDeferralContext;
</ins><span class="cx"> class Heap;
</span><span class="cx"> class MarkedSpace;
</span><span class="cx"> class LLIntOffsetsExtractor;
</span><span class="lines">@@ -151,8 +152,8 @@
</span><span class="cx"> bool needsDestruction() const { return m_attributes.destruction == NeedsDestruction; }
</span><span class="cx"> DestructionMode destruction() const { return m_attributes.destruction; }
</span><span class="cx"> HeapCell::Kind cellKind() const { return m_attributes.cellKind; }
</span><del>- void* allocate();
- void* tryAllocate();
</del><ins>+ void* allocate(GCDeferralContext* = nullptr);
+ void* tryAllocate(GCDeferralContext* = nullptr);
</ins><span class="cx"> Heap* heap() { return m_heap; }
</span><span class="cx"> MarkedBlock::Handle* takeLastActiveBlock()
</span><span class="cx"> {
</span><span class="lines">@@ -215,15 +216,15 @@
</span><span class="cx">
</span><span class="cx"> bool shouldStealEmptyBlocksFromOtherAllocators() const;
</span><span class="cx">
</span><del>- JS_EXPORT_PRIVATE void* allocateSlowCase();
- JS_EXPORT_PRIVATE void* tryAllocateSlowCase();
- void* allocateSlowCaseImpl(bool crashOnFailure);
</del><ins>+ JS_EXPORT_PRIVATE void* allocateSlowCase(GCDeferralContext*);
+ JS_EXPORT_PRIVATE void* tryAllocateSlowCase(GCDeferralContext*);
+ void* allocateSlowCaseImpl(GCDeferralContext*, bool crashOnFailure);
</ins><span class="cx"> void didConsumeFreeList();
</span><span class="cx"> void* tryAllocateWithoutCollecting();
</span><span class="cx"> MarkedBlock::Handle* tryAllocateBlock();
</span><span class="cx"> void* tryAllocateIn(MarkedBlock::Handle*);
</span><span class="cx"> void* allocateIn(MarkedBlock::Handle*);
</span><del>- ALWAYS_INLINE void doTestCollectionsIfNeeded();
</del><ins>+ ALWAYS_INLINE void doTestCollectionsIfNeeded(GCDeferralContext*);
</ins><span class="cx">
</span><span class="cx"> void setFreeList(const FreeList&);
</span><span class="cx">
</span><span class="lines">@@ -264,7 +265,7 @@
</span><span class="cx"> return OBJECT_OFFSETOF(MarkedAllocator, m_cellSize);
</span><span class="cx"> }
</span><span class="cx">
</span><del>-ALWAYS_INLINE void* MarkedAllocator::tryAllocate()
</del><ins>+ALWAYS_INLINE void* MarkedAllocator::tryAllocate(GCDeferralContext* deferralContext)
</ins><span class="cx"> {
</span><span class="cx"> unsigned remaining = m_freeList.remaining;
</span><span class="cx"> if (remaining) {
</span><span class="lines">@@ -276,13 +277,13 @@
</span><span class="cx">
</span><span class="cx"> FreeCell* head = m_freeList.head;
</span><span class="cx"> if (UNLIKELY(!head))
</span><del>- return tryAllocateSlowCase();
</del><ins>+ return tryAllocateSlowCase(deferralContext);
</ins><span class="cx">
</span><span class="cx"> m_freeList.head = head->next;
</span><span class="cx"> return head;
</span><span class="cx"> }
</span><span class="cx">
</span><del>-ALWAYS_INLINE void* MarkedAllocator::allocate()
</del><ins>+ALWAYS_INLINE void* MarkedAllocator::allocate(GCDeferralContext* deferralContext)
</ins><span class="cx"> {
</span><span class="cx"> unsigned remaining = m_freeList.remaining;
</span><span class="cx"> if (remaining) {
</span><span class="lines">@@ -294,7 +295,7 @@
</span><span class="cx">
</span><span class="cx"> FreeCell* head = m_freeList.head;
</span><span class="cx"> if (UNLIKELY(!head))
</span><del>- return allocateSlowCase();
</del><ins>+ return allocateSlowCase(deferralContext);
</ins><span class="cx">
</span><span class="cx"> m_freeList.head = head->next;
</span><span class="cx"> return head;
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreheapMarkedSpacecpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/heap/MarkedSpace.cpp (206554 => 206555)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/heap/MarkedSpace.cpp 2016-09-28 21:53:53 UTC (rev 206554)
+++ trunk/Source/JavaScriptCore/heap/MarkedSpace.cpp 2016-09-28 21:55:53 UTC (rev 206555)
</span><span class="lines">@@ -235,32 +235,58 @@
</span><span class="cx">
</span><span class="cx"> void* MarkedSpace::allocate(Subspace& subspace, size_t bytes)
</span><span class="cx"> {
</span><ins>+ if (false)
+ dataLog("Allocating ", bytes, " bytes in ", subspace.attributes, ".\n");
</ins><span class="cx"> if (MarkedAllocator* allocator = allocatorFor(subspace, bytes)) {
</span><span class="cx"> void* result = allocator->allocate();
</span><span class="cx"> return result;
</span><span class="cx"> }
</span><del>- return allocateLarge(subspace, bytes);
</del><ins>+ return allocateLarge(subspace, nullptr, bytes);
</ins><span class="cx"> }
</span><span class="cx">
</span><ins>+void* MarkedSpace::allocate(Subspace& subspace, GCDeferralContext* deferralContext, size_t bytes)
+{
+ if (false)
+ dataLog("Allocating ", bytes, " deferred bytes in ", subspace.attributes, ".\n");
+ if (MarkedAllocator* allocator = allocatorFor(subspace, bytes)) {
+ void* result = allocator->allocate(deferralContext);
+ return result;
+ }
+ return allocateLarge(subspace, deferralContext, bytes);
+}
+
</ins><span class="cx"> void* MarkedSpace::tryAllocate(Subspace& subspace, size_t bytes)
</span><span class="cx"> {
</span><ins>+ if (false)
+ dataLog("Try-allocating ", bytes, " bytes in ", subspace.attributes, ".\n");
</ins><span class="cx"> if (MarkedAllocator* allocator = allocatorFor(subspace, bytes)) {
</span><span class="cx"> void* result = allocator->tryAllocate();
</span><span class="cx"> return result;
</span><span class="cx"> }
</span><del>- return tryAllocateLarge(subspace, bytes);
</del><ins>+ return tryAllocateLarge(subspace, nullptr, bytes);
</ins><span class="cx"> }
</span><span class="cx">
</span><del>-void* MarkedSpace::allocateLarge(Subspace& subspace, size_t size)
</del><ins>+void* MarkedSpace::tryAllocate(Subspace& subspace, GCDeferralContext* deferralContext, size_t bytes)
</ins><span class="cx"> {
</span><del>- void* result = tryAllocateLarge(subspace, size);
</del><ins>+ if (false)
+ dataLog("Try-allocating ", bytes, " deferred bytes in ", subspace.attributes, ".\n");
+ if (MarkedAllocator* allocator = allocatorFor(subspace, bytes)) {
+ void* result = allocator->tryAllocate(deferralContext);
+ return result;
+ }
+ return tryAllocateLarge(subspace, deferralContext, bytes);
+}
+
+void* MarkedSpace::allocateLarge(Subspace& subspace, GCDeferralContext* deferralContext, size_t size)
+{
+ void* result = tryAllocateLarge(subspace, deferralContext, size);
</ins><span class="cx"> RELEASE_ASSERT(result);
</span><span class="cx"> return result;
</span><span class="cx"> }
</span><span class="cx">
</span><del>-void* MarkedSpace::tryAllocateLarge(Subspace& subspace, size_t size)
</del><ins>+void* MarkedSpace::tryAllocateLarge(Subspace& subspace, GCDeferralContext* deferralContext, size_t size)
</ins><span class="cx"> {
</span><del>- m_heap->collectIfNecessaryOrDefer();
</del><ins>+ m_heap->collectIfNecessaryOrDefer(deferralContext);
</ins><span class="cx">
</span><span class="cx"> size = WTF::roundUpToMultipleOf<sizeStep>(size);
</span><span class="cx"> LargeAllocation* allocation = LargeAllocation::tryCreate(*m_heap, size, subspace.attributes);
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreheapMarkedSpaceh"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/heap/MarkedSpace.h (206554 => 206555)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/heap/MarkedSpace.h 2016-09-28 21:53:53 UTC (rev 206554)
+++ trunk/Source/JavaScriptCore/heap/MarkedSpace.h 2016-09-28 21:55:53 UTC (rev 206555)
</span><span class="lines">@@ -111,12 +111,17 @@
</span><span class="cx"> MarkedAllocator* auxiliaryAllocatorFor(size_t);
</span><span class="cx">
</span><span class="cx"> JS_EXPORT_PRIVATE void* allocate(Subspace&, size_t);
</span><ins>+ JS_EXPORT_PRIVATE void* allocate(Subspace&, GCDeferralContext*, size_t);
</ins><span class="cx"> JS_EXPORT_PRIVATE void* tryAllocate(Subspace&, size_t);
</span><ins>+ JS_EXPORT_PRIVATE void* tryAllocate(Subspace&, GCDeferralContext*, size_t);
</ins><span class="cx">
</span><span class="cx"> void* allocateWithDestructor(size_t);
</span><span class="cx"> void* allocateWithoutDestructor(size_t);
</span><ins>+ void* allocateWithDestructor(GCDeferralContext*, size_t);
+ void* allocateWithoutDestructor(GCDeferralContext*, size_t);
</ins><span class="cx"> void* allocateAuxiliary(size_t);
</span><span class="cx"> void* tryAllocateAuxiliary(size_t);
</span><ins>+ void* tryAllocateAuxiliary(GCDeferralContext*, size_t);
</ins><span class="cx">
</span><span class="cx"> Subspace& subspaceForObjectsWithDestructor() { return m_destructorSpace; }
</span><span class="cx"> Subspace& subspaceForObjectsWithoutDestructor() { return m_normalSpace; }
</span><span class="lines">@@ -194,8 +199,8 @@
</span><span class="cx">
</span><span class="cx"> JS_EXPORT_PRIVATE static std::array<size_t, numSizeClasses> s_sizeClassForSizeStep;
</span><span class="cx">
</span><del>- JS_EXPORT_PRIVATE void* allocateLarge(Subspace&, size_t);
- JS_EXPORT_PRIVATE void* tryAllocateLarge(Subspace&, size_t);
</del><ins>+ void* allocateLarge(Subspace&, GCDeferralContext*, size_t);
+ void* tryAllocateLarge(Subspace&, GCDeferralContext*, size_t);
</ins><span class="cx">
</span><span class="cx"> static void initializeSizeClassForStepSize();
</span><span class="cx">
</span><span class="lines">@@ -263,6 +268,16 @@
</span><span class="cx"> return allocate(m_destructorSpace, bytes);
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+inline void* MarkedSpace::allocateWithoutDestructor(GCDeferralContext* deferralContext, size_t bytes)
+{
+ return allocate(m_normalSpace, deferralContext, bytes);
+}
+
+inline void* MarkedSpace::allocateWithDestructor(GCDeferralContext* deferralContext, size_t bytes)
+{
+ return allocate(m_destructorSpace, deferralContext, bytes);
+}
+
</ins><span class="cx"> inline void* MarkedSpace::allocateAuxiliary(size_t bytes)
</span><span class="cx"> {
</span><span class="cx"> return allocate(m_auxiliarySpace, bytes);
</span><span class="lines">@@ -273,6 +288,11 @@
</span><span class="cx"> return tryAllocate(m_auxiliarySpace, bytes);
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+inline void* MarkedSpace::tryAllocateAuxiliary(GCDeferralContext* deferralContext, size_t bytes)
+{
+ return tryAllocate(m_auxiliarySpace, deferralContext, bytes);
+}
+
</ins><span class="cx"> template <typename Functor> inline void MarkedSpace::forEachBlock(const Functor& functor)
</span><span class="cx"> {
</span><span class="cx"> forEachAllocator(
</span></span></pre></div>
<a id="trunkSourceJavaScriptCorejitAssemblyHelpersh"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/jit/AssemblyHelpers.h (206554 => 206555)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/jit/AssemblyHelpers.h 2016-09-28 21:53:53 UTC (rev 206554)
+++ trunk/Source/JavaScriptCore/jit/AssemblyHelpers.h 2016-09-28 21:55:53 UTC (rev 206555)
</span><span class="lines">@@ -1305,17 +1305,44 @@
</span><span class="cx">
</span><span class="cx"> static void emitStoreStructureWithTypeInfo(AssemblyHelpers& jit, TrustedImmPtr structure, RegisterID dest);
</span><span class="cx">
</span><del>- Jump jumpIfIsRememberedOrInEden(GPRReg cell)
</del><ins>+ Jump barrierBranchWithoutFence(GPRReg cell)
</ins><span class="cx"> {
</span><span class="cx"> return branch8(Above, Address(cell, JSCell::cellStateOffset()), TrustedImm32(blackThreshold));
</span><span class="cx"> }
</span><span class="cx">
</span><del>- Jump jumpIfIsRememberedOrInEden(JSCell* cell)
</del><ins>+ Jump barrierBranchWithoutFence(JSCell* cell)
</ins><span class="cx"> {
</span><span class="cx"> uint8_t* address = reinterpret_cast<uint8_t*>(cell) + JSCell::cellStateOffset();
</span><span class="cx"> return branch8(Above, AbsoluteAddress(address), TrustedImm32(blackThreshold));
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+ Jump barrierBranch(GPRReg cell, GPRReg scratchGPR)
+ {
+ load8(Address(cell, JSCell::cellStateOffset()), scratchGPR);
+ return branch32(Above, scratchGPR, AbsoluteAddress(vm()->heap.addressOfBarrierThreshold()));
+ }
+
+ Jump barrierBranch(JSCell* cell, GPRReg scratchGPR)
+ {
+ uint8_t* address = reinterpret_cast<uint8_t*>(cell) + JSCell::cellStateOffset();
+ load8(address, scratchGPR);
+ return branch32(Above, scratchGPR, AbsoluteAddress(vm()->heap.addressOfBarrierThreshold()));
+ }
+
+ void barrierStoreLoadFence()
+ {
+ if (!Options::useConcurrentBarriers())
+ return;
+ Jump ok = jumpIfBarrierStoreLoadFenceNotNeeded();
+ memoryFence();
+ ok.link(this);
+ }
+
+ Jump jumpIfBarrierStoreLoadFenceNotNeeded()
+ {
+ return branchTest8(Zero, AbsoluteAddress(vm()->heap.addressOfBarrierShouldBeFenced()));
+ }
+
</ins><span class="cx"> // Emits the branch structure for typeof. The code emitted by this doesn't fall through. The
</span><span class="cx"> // functor is called at those points where we have pinpointed a type. One way to use this is to
</span><span class="cx"> // have the functor emit the code to put the type string into an appropriate register and then
</span></span></pre></div>
<a id="trunkSourceJavaScriptCorejitJITOperationscpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/jit/JITOperations.cpp (206554 => 206555)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/jit/JITOperations.cpp 2016-09-28 21:53:53 UTC (rev 206554)
+++ trunk/Source/JavaScriptCore/jit/JITOperations.cpp 2016-09-28 21:55:53 UTC (rev 206555)
</span><span class="lines">@@ -2165,14 +2165,11 @@
</span><span class="cx"> vm->heap.writeBarrier(cell);
</span><span class="cx"> }
</span><span class="cx">
</span><del>-// NB: We don't include the value as part of the barrier because the write barrier elision
-// phase in the DFG only tracks whether the object being stored to has been barriered. It
-// would be much more complicated to try to model the value being stored as well.
-void JIT_OPERATION operationUnconditionalWriteBarrier(ExecState* exec, JSCell* cell)
</del><ins>+void JIT_OPERATION operationWriteBarrierSlowPath(ExecState* exec, JSCell* cell)
</ins><span class="cx"> {
</span><span class="cx"> VM* vm = &exec->vm();
</span><span class="cx"> NativeCallFrameTracer tracer(vm, exec);
</span><del>- vm->heap.writeBarrier(cell);
</del><ins>+ vm->heap.writeBarrierSlowPath(cell);
</ins><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> void JIT_OPERATION lookupExceptionHandler(VM* vm, ExecState* exec)
</span></span></pre></div>
<a id="trunkSourceJavaScriptCorejitJITOperationsh"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/jit/JITOperations.h (206554 => 206555)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/jit/JITOperations.h 2016-09-28 21:53:53 UTC (rev 206554)
+++ trunk/Source/JavaScriptCore/jit/JITOperations.h 2016-09-28 21:55:53 UTC (rev 206555)
</span><span class="lines">@@ -410,8 +410,7 @@
</span><span class="cx"> char* JIT_OPERATION operationReallocateButterflyToGrowPropertyStorage(ExecState*, JSObject*, size_t newSize) WTF_INTERNAL;
</span><span class="cx">
</span><span class="cx"> void JIT_OPERATION operationFlushWriteBarrierBuffer(ExecState*, JSCell*);
</span><del>-void JIT_OPERATION operationWriteBarrier(ExecState*, JSCell*, JSCell*);
-void JIT_OPERATION operationUnconditionalWriteBarrier(ExecState*, JSCell*);
</del><ins>+void JIT_OPERATION operationWriteBarrierSlowPath(ExecState*, JSCell*);
</ins><span class="cx"> void JIT_OPERATION operationOSRWriteBarrier(ExecState*, JSCell*);
</span><span class="cx">
</span><span class="cx"> void JIT_OPERATION operationExceptionFuzz(ExecState*);
</span></span></pre></div>
<a id="trunkSourceJavaScriptCorejitJITPropertyAccesscpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/jit/JITPropertyAccess.cpp (206554 => 206555)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/jit/JITPropertyAccess.cpp 2016-09-28 21:53:53 UTC (rev 206554)
+++ trunk/Source/JavaScriptCore/jit/JITPropertyAccess.cpp 2016-09-28 21:55:53 UTC (rev 206555)
</span><span class="lines">@@ -666,8 +666,6 @@
</span><span class="cx"> int valueVReg = currentInstruction[3].u.operand;
</span><span class="cx"> unsigned direct = currentInstruction[8].u.putByIdFlags & PutByIdIsDirect;
</span><span class="cx">
</span><del>- emitWriteBarrier(baseVReg, valueVReg, ShouldFilterBase);
-
</del><span class="cx"> // In order to be able to patch both the Structure, and the object offset, we store one pointer,
</span><span class="cx"> // to just after the arguments have been loaded into registers 'hotPathBegin', and we generate code
</span><span class="cx"> // such that the Structure & offset are always at the same distance from this.
</span><span class="lines">@@ -684,6 +682,8 @@
</span><span class="cx"> gen.generateFastPath(*this);
</span><span class="cx"> addSlowCase(gen.slowPathJump());
</span><span class="cx">
</span><ins>+ emitWriteBarrier(baseVReg, valueVReg, ShouldFilterBase);
+
</ins><span class="cx"> m_putByIds.append(gen);
</span><span class="cx"> }
</span><span class="cx">
</span><span class="lines">@@ -1180,8 +1180,8 @@
</span><span class="cx"> if (mode == ShouldFilterBaseAndValue || mode == ShouldFilterBase)
</span><span class="cx"> ownerNotCell = branchTest64(NonZero, regT0, tagMaskRegister);
</span><span class="cx">
</span><del>- Jump ownerIsRememberedOrInEden = jumpIfIsRememberedOrInEden(regT0);
- callOperation(operationUnconditionalWriteBarrier, regT0);
</del><ins>+ Jump ownerIsRememberedOrInEden = barrierBranch(regT0, regT1);
+ callOperation(operationWriteBarrierSlowPath, regT0);
</ins><span class="cx"> ownerIsRememberedOrInEden.link(this);
</span><span class="cx">
</span><span class="cx"> if (mode == ShouldFilterBaseAndValue || mode == ShouldFilterBase)
</span><span class="lines">@@ -1218,8 +1218,8 @@
</span><span class="cx"> if (mode == ShouldFilterBase || mode == ShouldFilterBaseAndValue)
</span><span class="cx"> ownerNotCell = branch32(NotEqual, regT0, TrustedImm32(JSValue::CellTag));
</span><span class="cx">
</span><del>- Jump ownerIsRememberedOrInEden = jumpIfIsRememberedOrInEden(regT1);
- callOperation(operationUnconditionalWriteBarrier, regT1);
</del><ins>+ Jump ownerIsRememberedOrInEden = barrierBranch(regT1, regT2);
+ callOperation(operationWriteBarrierSlowPath, regT1);
</ins><span class="cx"> ownerIsRememberedOrInEden.link(this);
</span><span class="cx">
</span><span class="cx"> if (mode == ShouldFilterBase || mode == ShouldFilterBaseAndValue)
</span><span class="lines">@@ -1246,12 +1246,9 @@
</span><span class="cx">
</span><span class="cx"> void JIT::emitWriteBarrier(JSCell* owner)
</span><span class="cx"> {
</span><del>- if (!owner->cellContainer().isMarked(owner)) {
- Jump ownerIsRememberedOrInEden = jumpIfIsRememberedOrInEden(owner);
- callOperation(operationUnconditionalWriteBarrier, owner);
- ownerIsRememberedOrInEden.link(this);
- } else
- callOperation(operationUnconditionalWriteBarrier, owner);
</del><ins>+ Jump ownerIsRememberedOrInEden = barrierBranch(owner, regT0);
+ callOperation(operationWriteBarrierSlowPath, owner);
+ ownerIsRememberedOrInEden.link(this);
</ins><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> void JIT::emitByValIdentifierCheck(ByValInfo* byValInfo, RegisterID cell, RegisterID scratch, const Identifier& propertyName, JumpList& slowCases)
</span><span class="lines">@@ -1390,8 +1387,8 @@
</span><span class="cx"> patchBuffer.link(slowCases, CodeLocationLabel(MacroAssemblerCodePtr::createFromExecutableAddress(returnAddress.value())).labelAtOffset(byValInfo->returnAddressToSlowPath));
</span><span class="cx"> patchBuffer.link(done, byValInfo->badTypeJump.labelAtOffset(byValInfo->badTypeJumpToDone));
</span><span class="cx"> if (needsLinkForWriteBarrier) {
</span><del>- ASSERT(m_calls.last().to == operationUnconditionalWriteBarrier);
- patchBuffer.link(m_calls.last().from, operationUnconditionalWriteBarrier);
</del><ins>+ ASSERT(m_calls.last().to == operationWriteBarrierSlowPath);
+ patchBuffer.link(m_calls.last().from, operationWriteBarrierSlowPath);
</ins><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> bool isDirect = m_interpreter->getOpcodeID(currentInstruction->u.opcode) == op_put_by_val_direct;
</span></span></pre></div>
<a id="trunkSourceJavaScriptCorejitJITPropertyAccess32_64cpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/jit/JITPropertyAccess32_64.cpp (206554 => 206555)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/jit/JITPropertyAccess32_64.cpp 2016-09-28 21:53:53 UTC (rev 206554)
+++ trunk/Source/JavaScriptCore/jit/JITPropertyAccess32_64.cpp 2016-09-28 21:55:53 UTC (rev 206555)
</span><span class="lines">@@ -673,8 +673,6 @@
</span><span class="cx"> int value = currentInstruction[3].u.operand;
</span><span class="cx"> int direct = currentInstruction[8].u.putByIdFlags & PutByIdIsDirect;
</span><span class="cx">
</span><del>- emitWriteBarrier(base, value, ShouldFilterBase);
-
</del><span class="cx"> emitLoad2(base, regT1, regT0, value, regT3, regT2);
</span><span class="cx">
</span><span class="cx"> emitJumpSlowCaseIfNotJSCell(base, regT1);
</span><span class="lines">@@ -687,6 +685,8 @@
</span><span class="cx"> gen.generateFastPath(*this);
</span><span class="cx"> addSlowCase(gen.slowPathJump());
</span><span class="cx">
</span><ins>+ emitWriteBarrier(base, value, ShouldFilterBase);
+
</ins><span class="cx"> m_putByIds.append(gen);
</span><span class="cx"> }
</span><span class="cx">
</span></span></pre></div>
<a id="trunkSourceJavaScriptCorellintLowLevelInterpreterasm"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/llint/LowLevelInterpreter.asm (206554 => 206555)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/llint/LowLevelInterpreter.asm 2016-09-28 21:53:53 UTC (rev 206554)
+++ trunk/Source/JavaScriptCore/llint/LowLevelInterpreter.asm 2016-09-28 21:55:53 UTC (rev 206555)
</span><span class="lines">@@ -891,6 +891,7 @@
</span><span class="cx"> end
</span><span class="cx">
</span><span class="cx"> macro skipIfIsRememberedOrInEden(cell, slowPath)
</span><ins>+ memfence
</ins><span class="cx"> bba JSCell::m_cellState[cell], BlackThreshold, .done
</span><span class="cx"> slowPath()
</span><span class="cx"> .done:
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreofflineasmx86rb"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/offlineasm/x86.rb (206554 => 206555)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/offlineasm/x86.rb 2016-09-28 21:53:53 UTC (rev 206554)
+++ trunk/Source/JavaScriptCore/offlineasm/x86.rb 2016-09-28 21:55:53 UTC (rev 206555)
</span><span class="lines">@@ -1527,7 +1527,12 @@
</span><span class="cx"> when "leap"
</span><span class="cx"> $asm.puts "lea#{x86Suffix(:ptr)} #{orderOperands(operands[0].x86AddressOperand(:ptr), operands[1].x86Operand(:ptr))}"
</span><span class="cx"> when "memfence"
</span><del>- $asm.puts "mfence"
</del><ins>+ sp = RegisterID.new(nil, "sp")
+ if isIntelSyntax
+ $asm.puts "mfence"
+ else
+ $asm.puts "lock; orl $0, (#{sp.x86Operand(:ptr)})"
+ end
</ins><span class="cx"> else
</span><span class="cx"> lowerDefault
</span><span class="cx"> end
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreruntimeJSArraycpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/runtime/JSArray.cpp (206554 => 206555)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/runtime/JSArray.cpp 2016-09-28 21:53:53 UTC (rev 206554)
+++ trunk/Source/JavaScriptCore/runtime/JSArray.cpp 2016-09-28 21:55:53 UTC (rev 206555)
</span><span class="lines">@@ -59,7 +59,7 @@
</span><span class="cx"> return butterfly;
</span><span class="cx"> }
</span><span class="cx">
</span><del>-JSArray* JSArray::tryCreateUninitialized(VM& vm, Structure* structure, unsigned initialLength)
</del><ins>+JSArray* JSArray::tryCreateUninitialized(VM& vm, GCDeferralContext* deferralContext, Structure* structure, unsigned initialLength)
</ins><span class="cx"> {
</span><span class="cx"> if (initialLength > MAX_STORAGE_VECTOR_LENGTH)
</span><span class="cx"> return 0;
</span><span class="lines">@@ -76,7 +76,7 @@
</span><span class="cx"> || hasContiguous(indexingType));
</span><span class="cx">
</span><span class="cx"> unsigned vectorLength = Butterfly::optimalContiguousVectorLength(structure, initialLength);
</span><del>- void* temp = vm.heap.tryAllocateAuxiliary(nullptr, Butterfly::totalSize(0, outOfLineStorage, true, vectorLength * sizeof(EncodedJSValue)));
</del><ins>+ void* temp = vm.heap.tryAllocateAuxiliary(deferralContext, nullptr, Butterfly::totalSize(0, outOfLineStorage, true, vectorLength * sizeof(EncodedJSValue)));
</ins><span class="cx"> if (!temp)
</span><span class="cx"> return nullptr;
</span><span class="cx"> butterfly = Butterfly::fromBase(temp, 0, outOfLineStorage);
</span><span class="lines">@@ -104,7 +104,7 @@
</span><span class="cx"> storage->m_vector[i].clear();
</span><span class="cx"> }
</span><span class="cx">
</span><del>- return createWithButterfly(vm, structure, butterfly);
</del><ins>+ return createWithButterfly(vm, deferralContext, structure, butterfly);
</ins><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> void JSArray::setLengthWritable(ExecState* exec, bool writable)
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreruntimeJSArrayh"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/runtime/JSArray.h (206554 => 206555)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/runtime/JSArray.h 2016-09-28 21:53:53 UTC (rev 206554)
+++ trunk/Source/JavaScriptCore/runtime/JSArray.h 2016-09-28 21:55:53 UTC (rev 206555)
</span><span class="lines">@@ -53,13 +53,17 @@
</span><span class="cx">
</span><span class="cx"> public:
</span><span class="cx"> static JSArray* create(VM&, Structure*, unsigned initialLength = 0);
</span><del>- static JSArray* createWithButterfly(VM&, Structure*, Butterfly*);
</del><ins>+ static JSArray* createWithButterfly(VM&, GCDeferralContext*, Structure*, Butterfly*);
</ins><span class="cx">
</span><span class="cx"> // tryCreateUninitialized is used for fast construction of arrays whose size and
</span><span class="cx"> // contents are known at time of creation. Clients of this interface must:
</span><span class="cx"> // - null-check the result (indicating out of memory, or otherwise unable to allocate vector).
</span><span class="cx"> // - call 'initializeIndex' for all properties in sequence, for 0 <= i < initialLength.
</span><del>- JS_EXPORT_PRIVATE static JSArray* tryCreateUninitialized(VM&, Structure*, unsigned initialLength);
</del><ins>+ JS_EXPORT_PRIVATE static JSArray* tryCreateUninitialized(VM&, GCDeferralContext*, Structure*, unsigned initialLength);
+ static JSArray* tryCreateUninitialized(VM& vm, Structure* structure, unsigned initialLength)
+ {
+ return tryCreateUninitialized(vm, nullptr, structure, initialLength);
+ }
</ins><span class="cx">
</span><span class="cx"> JS_EXPORT_PRIVATE static bool defineOwnProperty(JSObject*, ExecState*, PropertyName, const PropertyDescriptor&, bool throwException);
</span><span class="cx">
</span><span class="lines">@@ -230,12 +234,12 @@
</span><span class="cx"> butterfly->arrayStorage()->m_vector[i].clear();
</span><span class="cx"> }
</span><span class="cx">
</span><del>- return createWithButterfly(vm, structure, butterfly);
</del><ins>+ return createWithButterfly(vm, nullptr, structure, butterfly);
</ins><span class="cx"> }
</span><span class="cx">
</span><del>-inline JSArray* JSArray::createWithButterfly(VM& vm, Structure* structure, Butterfly* butterfly)
</del><ins>+inline JSArray* JSArray::createWithButterfly(VM& vm, GCDeferralContext* deferralContext, Structure* structure, Butterfly* butterfly)
</ins><span class="cx"> {
</span><del>- JSArray* array = new (NotNull, allocateCell<JSArray>(vm.heap)) JSArray(vm, structure, butterfly);
</del><ins>+ JSArray* array = new (NotNull, allocateCell<JSArray>(vm.heap, deferralContext)) JSArray(vm, structure, butterfly);
</ins><span class="cx"> array->finishCreation(vm);
</span><span class="cx"> return array;
</span><span class="cx"> }
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreruntimeJSCellh"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/runtime/JSCell.h (206554 => 206555)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/runtime/JSCell.h 2016-09-28 21:53:53 UTC (rev 206554)
+++ trunk/Source/JavaScriptCore/runtime/JSCell.h 2016-09-28 21:55:53 UTC (rev 206555)
</span><span class="lines">@@ -39,6 +39,7 @@
</span><span class="cx"> namespace JSC {
</span><span class="cx">
</span><span class="cx"> class CopyVisitor;
</span><ins>+class GCDeferralContext;
</ins><span class="cx"> class ExecState;
</span><span class="cx"> class Identifier;
</span><span class="cx"> class JSArrayBufferView;
</span><span class="lines">@@ -52,6 +53,9 @@
</span><span class="cx"> template<typename T> void* allocateCell(Heap&);
</span><span class="cx"> template<typename T> void* allocateCell(Heap&, size_t);
</span><span class="cx">
</span><ins>+template<typename T> void* allocateCell(Heap&, GCDeferralContext*);
+template<typename T> void* allocateCell(Heap&, GCDeferralContext*, size_t);
+
</ins><span class="cx"> #define DECLARE_EXPORT_INFO \
</span><span class="cx"> protected: \
</span><span class="cx"> static JS_EXPORTDATA const ::JSC::ClassInfo s_info; \
</span><span class="lines">@@ -69,6 +73,8 @@
</span><span class="cx"> friend class MarkedBlock;
</span><span class="cx"> template<typename T> friend void* allocateCell(Heap&);
</span><span class="cx"> template<typename T> friend void* allocateCell(Heap&, size_t);
</span><ins>+ template<typename T> friend void* allocateCell(Heap&, GCDeferralContext*);
+ template<typename T> friend void* allocateCell(Heap&, GCDeferralContext*, size_t);
</ins><span class="cx">
</span><span class="cx"> public:
</span><span class="cx"> static const unsigned StructureFlags = 0;
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreruntimeJSCellInlinesh"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/runtime/JSCellInlines.h (206554 => 206555)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/runtime/JSCellInlines.h 2016-09-28 21:53:53 UTC (rev 206554)
+++ trunk/Source/JavaScriptCore/runtime/JSCellInlines.h 2016-09-28 21:55:53 UTC (rev 206555)
</span><span class="lines">@@ -141,6 +141,25 @@
</span><span class="cx"> return allocateCell<T>(heap, sizeof(T));
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+template<typename T>
+void* allocateCell(Heap& heap, GCDeferralContext* deferralContext, size_t size)
+{
+ ASSERT(size >= sizeof(T));
+ JSCell* result = static_cast<JSCell*>(heap.allocateObjectOfType<T>(deferralContext, size));
+#if ENABLE(GC_VALIDATION)
+ ASSERT(!heap.vm()->isInitializingObject());
+ heap.vm()->setInitializingObjectClass(T::info());
+#endif
+ result->clearStructure();
+ return result;
+}
+
+template<typename T>
+void* allocateCell(Heap& heap, GCDeferralContext* deferralContext)
+{
+ return allocateCell<T>(heap, deferralContext, sizeof(T));
+}
+
</ins><span class="cx"> inline bool JSCell::isObject() const
</span><span class="cx"> {
</span><span class="cx"> return TypeInfo::isObject(m_type);
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreruntimeJSObjecth"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/runtime/JSObject.h (206554 => 206555)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/runtime/JSObject.h 2016-09-28 21:53:53 UTC (rev 206554)
+++ trunk/Source/JavaScriptCore/runtime/JSObject.h 2016-09-28 21:55:53 UTC (rev 206555)
</span><span class="lines">@@ -417,7 +417,7 @@
</span><span class="cx">
</span><span class="cx"> // NOTE: Clients of this method may call it more than once for any index, and this is supposed
</span><span class="cx"> // to work.
</span><del>- void initializeIndex(VM& vm, unsigned i, JSValue v, IndexingType indexingType)
</del><ins>+ ALWAYS_INLINE void initializeIndex(VM& vm, unsigned i, JSValue v, IndexingType indexingType)
</ins><span class="cx"> {
</span><span class="cx"> Butterfly* butterfly = m_butterfly.get();
</span><span class="cx"> switch (indexingType) {
</span><span class="lines">@@ -467,6 +467,54 @@
</span><span class="cx"> }
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+ void initializeIndexWithoutBarrier(unsigned i, JSValue v)
+ {
+ initializeIndexWithoutBarrier(i, v, indexingType());
+ }
+
+ // This version of initializeIndex is for cases where you know that you will not need any
+ // barriers. This implies not having any data format conversions.
+ ALWAYS_INLINE void initializeIndexWithoutBarrier(unsigned i, JSValue v, IndexingType indexingType)
+ {
+ Butterfly* butterfly = m_butterfly.get();
+ switch (indexingType) {
+ case ALL_UNDECIDED_INDEXING_TYPES: {
+ RELEASE_ASSERT_NOT_REACHED();
+ break;
+ }
+ case ALL_INT32_INDEXING_TYPES: {
+ ASSERT(i < butterfly->publicLength());
+ ASSERT(i < butterfly->vectorLength());
+ RELEASE_ASSERT(v.isInt32());
+ FALLTHROUGH;
+ }
+ case ALL_CONTIGUOUS_INDEXING_TYPES: {
+ ASSERT(i < butterfly->publicLength());
+ ASSERT(i < butterfly->vectorLength());
+ butterfly->contiguous()[i].setWithoutWriteBarrier(v);
+ break;
+ }
+ case ALL_DOUBLE_INDEXING_TYPES: {
+ ASSERT(i < butterfly->publicLength());
+ ASSERT(i < butterfly->vectorLength());
+ RELEASE_ASSERT(v.isNumber());
+ double value = v.asNumber();
+ RELEASE_ASSERT(value == value);
+ butterfly->contiguousDouble()[i] = value;
+ break;
+ }
+ case ALL_ARRAY_STORAGE_INDEXING_TYPES: {
+ ArrayStorage* storage = butterfly->arrayStorage();
+ ASSERT(i < storage->length());
+ ASSERT(i < storage->m_numValuesInVector);
+ storage->m_vector[i].setWithoutWriteBarrier(v);
+ break;
+ }
+ default:
+ RELEASE_ASSERT_NOT_REACHED();
+ }
+ }
+
</ins><span class="cx"> bool hasSparseMap()
</span><span class="cx"> {
</span><span class="cx"> switch (indexingType()) {
</span><span class="lines">@@ -641,6 +689,7 @@
</span><span class="cx"> // Fast access to known property offsets.
</span><span class="cx"> JSValue getDirect(PropertyOffset offset) const { return locationForOffset(offset)->get(); }
</span><span class="cx"> void putDirect(VM& vm, PropertyOffset offset, JSValue value) { locationForOffset(offset)->set(vm, this, value); }
</span><ins>+ void putDirectWithoutBarrier(PropertyOffset offset, JSValue value) { locationForOffset(offset)->setWithoutWriteBarrier(value); }
</ins><span class="cx"> void putDirectUndefined(PropertyOffset offset) { locationForOffset(offset)->setUndefined(); }
</span><span class="cx">
</span><span class="cx"> JS_EXPORT_PRIVATE bool putDirectNativeIntrinsicGetter(VM&, JSGlobalObject*, Identifier, NativeFunction, Intrinsic, unsigned attributes);
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreruntimeJSStringh"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/runtime/JSString.h (206554 => 206555)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/runtime/JSString.h 2016-09-28 21:53:53 UTC (rev 206554)
+++ trunk/Source/JavaScriptCore/runtime/JSString.h 2016-09-28 21:55:53 UTC (rev 206555)
</span><span class="lines">@@ -365,13 +365,18 @@
</span><span class="cx"> return newString;
</span><span class="cx"> }
</span><span class="cx">
</span><del>- ALWAYS_INLINE static JSString* createSubstringOfResolved(VM& vm, JSString* base, unsigned offset, unsigned length)
</del><ins>+ ALWAYS_INLINE static JSString* createSubstringOfResolved(VM& vm, GCDeferralContext* deferralContext, JSString* base, unsigned offset, unsigned length)
</ins><span class="cx"> {
</span><del>- JSRopeString* newString = new (NotNull, allocateCell<JSRopeString>(vm.heap)) JSRopeString(vm);
</del><ins>+ JSRopeString* newString = new (NotNull, allocateCell<JSRopeString>(vm.heap, deferralContext)) JSRopeString(vm);
</ins><span class="cx"> newString->finishCreationSubstringOfResolved(vm, base, offset, length);
</span><span class="cx"> return newString;
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+ ALWAYS_INLINE static JSString* createSubstringOfResolved(VM& vm, JSString* base, unsigned offset, unsigned length)
+ {
+ return createSubstringOfResolved(vm, nullptr, base, offset, length);
+ }
+
</ins><span class="cx"> void visitFibers(SlotVisitor&);
</span><span class="cx">
</span><span class="cx"> static ptrdiff_t offsetOfFibers() { return OBJECT_OFFSETOF(JSRopeString, u); }
</span><span class="lines">@@ -491,6 +496,7 @@
</span><span class="cx"> return jsCast<JSString*>(value.asCell());
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+// This MUST NOT GC.
</ins><span class="cx"> inline JSString* jsEmptyString(VM* vm)
</span><span class="cx"> {
</span><span class="cx"> return vm->smallStrings.emptyString();
</span><span class="lines">@@ -581,7 +587,7 @@
</span><span class="cx"> return JSRopeString::create(vm, exec, s, offset, length);
</span><span class="cx"> }
</span><span class="cx">
</span><del>-inline JSString* jsSubstringOfResolved(VM& vm, JSString* s, unsigned offset, unsigned length)
</del><ins>+inline JSString* jsSubstringOfResolved(VM& vm, GCDeferralContext* deferralContext, JSString* s, unsigned offset, unsigned length)
</ins><span class="cx"> {
</span><span class="cx"> ASSERT(offset <= static_cast<unsigned>(s->length()));
</span><span class="cx"> ASSERT(length <= static_cast<unsigned>(s->length()));
</span><span class="lines">@@ -590,9 +596,14 @@
</span><span class="cx"> return vm.smallStrings.emptyString();
</span><span class="cx"> if (!offset && length == s->length())
</span><span class="cx"> return s;
</span><del>- return JSRopeString::createSubstringOfResolved(vm, s, offset, length);
</del><ins>+ return JSRopeString::createSubstringOfResolved(vm, deferralContext, s, offset, length);
</ins><span class="cx"> }
</span><span class="cx">
</span><ins>+inline JSString* jsSubstringOfResolved(VM& vm, JSString* s, unsigned offset, unsigned length)
+{
+ return jsSubstringOfResolved(vm, nullptr, s, offset, length);
+}
+
</ins><span class="cx"> inline JSString* jsSubstring(ExecState* exec, JSString* s, unsigned offset, unsigned length)
</span><span class="cx"> {
</span><span class="cx"> return jsSubstring(exec->vm(), exec, s, offset, length);
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreruntimeOptionsh"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/runtime/Options.h (206554 => 206555)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/runtime/Options.h 2016-09-28 21:53:53 UTC (rev 206554)
+++ trunk/Source/JavaScriptCore/runtime/Options.h 2016-09-28 21:55:53 UTC (rev 206555)
</span><span class="lines">@@ -181,6 +181,8 @@
</span><span class="cx"> v(bool, testTheFTL, false, Normal, nullptr) \
</span><span class="cx"> v(bool, verboseSanitizeStack, false, Normal, nullptr) \
</span><span class="cx"> v(bool, useGenerationalGC, true, Normal, nullptr) \
</span><ins>+ v(bool, useConcurrentBarriers, true, Normal, nullptr) \
+ v(bool, forceFencedBarrier, false, Normal, nullptr) \
</ins><span class="cx"> v(bool, scribbleFreeCells, false, Normal, nullptr) \
</span><span class="cx"> v(double, sizeClassProgression, 1.4, Normal, nullptr) \
</span><span class="cx"> v(unsigned, largeAllocationCutoff, 100000, Normal, nullptr) \
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreruntimeRegExpMatchesArraycpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/runtime/RegExpMatchesArray.cpp (206554 => 206555)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/runtime/RegExpMatchesArray.cpp 2016-09-28 21:53:53 UTC (rev 206554)
+++ trunk/Source/JavaScriptCore/runtime/RegExpMatchesArray.cpp 2016-09-28 21:55:53 UTC (rev 206555)
</span><span class="lines">@@ -36,29 +36,31 @@
</span><span class="cx"> // FIXME: This should handle array allocation errors gracefully.
</span><span class="cx"> // https://bugs.webkit.org/show_bug.cgi?id=155144
</span><span class="cx">
</span><ins>+ GCDeferralContext deferralContext(vm.heap);
+
</ins><span class="cx"> if (UNLIKELY(globalObject->isHavingABadTime())) {
</span><del>- array = JSArray::tryCreateUninitialized(vm, globalObject->regExpMatchesArrayStructure(), regExp->numSubpatterns() + 1);
</del><ins>+ array = JSArray::tryCreateUninitialized(vm, &deferralContext, globalObject->regExpMatchesArrayStructure(), regExp->numSubpatterns() + 1);
</ins><span class="cx">
</span><del>- array->initializeIndex(vm, 0, jsEmptyString(&vm));
</del><ins>+ array->initializeIndexWithoutBarrier(0, jsEmptyString(&vm));
</ins><span class="cx">
</span><span class="cx"> if (unsigned numSubpatterns = regExp->numSubpatterns()) {
</span><span class="cx"> for (unsigned i = 1; i <= numSubpatterns; ++i)
</span><del>- array->initializeIndex(vm, i, jsUndefined());
</del><ins>+ array->initializeIndexWithoutBarrier(i, jsUndefined());
</ins><span class="cx"> }
</span><span class="cx"> } else {
</span><del>- array = tryCreateUninitializedRegExpMatchesArray(vm, globalObject->regExpMatchesArrayStructure(), regExp->numSubpatterns() + 1);
</del><ins>+ array = tryCreateUninitializedRegExpMatchesArray(vm, &deferralContext, globalObject->regExpMatchesArrayStructure(), regExp->numSubpatterns() + 1);
</ins><span class="cx"> RELEASE_ASSERT(array);
</span><span class="cx">
</span><del>- array->initializeIndex(vm, 0, jsEmptyString(&vm), ArrayWithContiguous);
</del><ins>+ array->initializeIndexWithoutBarrier(0, jsEmptyString(&vm), ArrayWithContiguous);
</ins><span class="cx">
</span><span class="cx"> if (unsigned numSubpatterns = regExp->numSubpatterns()) {
</span><span class="cx"> for (unsigned i = 1; i <= numSubpatterns; ++i)
</span><del>- array->initializeIndex(vm, i, jsUndefined(), ArrayWithContiguous);
</del><ins>+ array->initializeIndexWithoutBarrier(i, jsUndefined(), ArrayWithContiguous);
</ins><span class="cx"> }
</span><span class="cx"> }
</span><span class="cx">
</span><del>- array->putDirect(vm, RegExpMatchesArrayIndexPropertyOffset, jsNumber(-1));
- array->putDirect(vm, RegExpMatchesArrayInputPropertyOffset, input);
</del><ins>+ array->putDirectWithoutBarrier(RegExpMatchesArrayIndexPropertyOffset, jsNumber(-1));
+ array->putDirectWithoutBarrier(RegExpMatchesArrayInputPropertyOffset, input);
</ins><span class="cx"> return array;
</span><span class="cx"> }
</span><span class="cx">
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreruntimeRegExpMatchesArrayh"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/runtime/RegExpMatchesArray.h (206554 => 206555)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/runtime/RegExpMatchesArray.h 2016-09-28 21:53:53 UTC (rev 206554)
+++ trunk/Source/JavaScriptCore/runtime/RegExpMatchesArray.h 2016-09-28 21:55:53 UTC (rev 206555)
</span><span class="lines">@@ -20,6 +20,7 @@
</span><span class="cx"> #pragma once
</span><span class="cx">
</span><span class="cx"> #include "ButterflyInlines.h"
</span><ins>+#include "GCDeferralContextInlines.h"
</ins><span class="cx"> #include "JSArray.h"
</span><span class="cx"> #include "JSCInlines.h"
</span><span class="cx"> #include "JSGlobalObject.h"
</span><span class="lines">@@ -31,13 +32,13 @@
</span><span class="cx"> static const PropertyOffset RegExpMatchesArrayIndexPropertyOffset = 100;
</span><span class="cx"> static const PropertyOffset RegExpMatchesArrayInputPropertyOffset = 101;
</span><span class="cx">
</span><del>-ALWAYS_INLINE JSArray* tryCreateUninitializedRegExpMatchesArray(VM& vm, Structure* structure, unsigned initialLength)
</del><ins>+ALWAYS_INLINE JSArray* tryCreateUninitializedRegExpMatchesArray(VM& vm, GCDeferralContext* deferralContext, Structure* structure, unsigned initialLength)
</ins><span class="cx"> {
</span><span class="cx"> unsigned vectorLength = initialLength;
</span><span class="cx"> if (vectorLength > MAX_STORAGE_VECTOR_LENGTH)
</span><span class="cx"> return 0;
</span><span class="cx">
</span><del>- void* temp = vm.heap.tryAllocateAuxiliary(nullptr, Butterfly::totalSize(0, structure->outOfLineCapacity(), true, vectorLength * sizeof(EncodedJSValue)));
</del><ins>+ void* temp = vm.heap.tryAllocateAuxiliary(deferralContext, nullptr, Butterfly::totalSize(0, structure->outOfLineCapacity(), true, vectorLength * sizeof(EncodedJSValue)));
</ins><span class="cx"> if (!temp)
</span><span class="cx"> return nullptr;
</span><span class="cx"> Butterfly* butterfly = Butterfly::fromBase(temp, 0, structure->outOfLineCapacity());
</span><span class="lines">@@ -47,7 +48,7 @@
</span><span class="cx"> for (unsigned i = initialLength; i < vectorLength; ++i)
</span><span class="cx"> butterfly->contiguous()[i].clear();
</span><span class="cx">
</span><del>- return JSArray::createWithButterfly(vm, structure, butterfly);
</del><ins>+ return JSArray::createWithButterfly(vm, deferralContext, structure, butterfly);
</ins><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> ALWAYS_INLINE JSArray* createRegExpMatchesArray(
</span><span class="lines">@@ -76,47 +77,44 @@
</span><span class="cx">
</span><span class="cx"> unsigned numSubpatterns = regExp->numSubpatterns();
</span><span class="cx">
</span><ins>+ GCDeferralContext deferralContext(vm.heap);
+
</ins><span class="cx"> if (UNLIKELY(globalObject->isHavingABadTime())) {
</span><del>- array = JSArray::tryCreateUninitialized(vm, globalObject->regExpMatchesArrayStructure(), numSubpatterns + 1);
</del><ins>+ array = JSArray::tryCreateUninitialized(vm, &deferralContext, globalObject->regExpMatchesArrayStructure(), numSubpatterns + 1);
</ins><span class="cx">
</span><span class="cx"> setProperties();
</span><span class="cx">
</span><del>- array->initializeIndex(vm, 0, jsUndefined());
</del><ins>+ array->initializeIndexWithoutBarrier(0, jsSubstringOfResolved(vm, &deferralContext, input, result.start, result.end - result.start));
</ins><span class="cx">
</span><del>- for (unsigned i = 1; i <= numSubpatterns; ++i)
- array->initializeIndex(vm, i, jsUndefined());
-
- // Now the object is safe to scan by GC.
-
- array->initializeIndex(vm, 0, jsSubstringOfResolved(vm, input, result.start, result.end - result.start));
-
</del><span class="cx"> for (unsigned i = 1; i <= numSubpatterns; ++i) {
</span><span class="cx"> int start = subpatternResults[2 * i];
</span><ins>+ JSValue value;
</ins><span class="cx"> if (start >= 0)
</span><del>- array->initializeIndex(vm, i, JSRopeString::createSubstringOfResolved(vm, input, start, subpatternResults[2 * i + 1] - start));
</del><ins>+ value = JSRopeString::createSubstringOfResolved(vm, &deferralContext, input, start, subpatternResults[2 * i + 1] - start);
+ else
+ value = jsUndefined();
+ array->initializeIndexWithoutBarrier(i, value);
</ins><span class="cx"> }
</span><span class="cx"> } else {
</span><del>- array = tryCreateUninitializedRegExpMatchesArray(vm, globalObject->regExpMatchesArrayStructure(), numSubpatterns + 1);
</del><ins>+ array = tryCreateUninitializedRegExpMatchesArray(vm, &deferralContext, globalObject->regExpMatchesArrayStructure(), numSubpatterns + 1);
</ins><span class="cx"> RELEASE_ASSERT(array);
</span><span class="cx">
</span><span class="cx"> setProperties();
</span><span class="cx">
</span><del>- array->initializeIndex(vm, 0, jsUndefined(), ArrayWithContiguous);
-
- for (unsigned i = 1; i <= numSubpatterns; ++i)
- array->initializeIndex(vm, i, jsUndefined(), ArrayWithContiguous);
-
</del><span class="cx"> // Now the object is safe to scan by GC.
</span><span class="cx">
</span><del>- array->initializeIndex(vm, 0, jsSubstringOfResolved(vm, input, result.start, result.end - result.start), ArrayWithContiguous);
</del><ins>+ array->initializeIndexWithoutBarrier(0, jsSubstringOfResolved(vm, &deferralContext, input, result.start, result.end - result.start), ArrayWithContiguous);
</ins><span class="cx">
</span><span class="cx"> for (unsigned i = 1; i <= numSubpatterns; ++i) {
</span><span class="cx"> int start = subpatternResults[2 * i];
</span><ins>+ JSValue value;
</ins><span class="cx"> if (start >= 0)
</span><del>- array->initializeIndex(vm, i, JSRopeString::createSubstringOfResolved(vm, input, start, subpatternResults[2 * i + 1] - start), ArrayWithContiguous);
</del><ins>+ value = JSRopeString::createSubstringOfResolved(vm, &deferralContext, input, start, subpatternResults[2 * i + 1] - start);
+ else
+ value = jsUndefined();
+ array->initializeIndexWithoutBarrier(i, value, ArrayWithContiguous);
</ins><span class="cx"> }
</span><span class="cx"> }
</span><del>-
</del><span class="cx"> return array;
</span><span class="cx"> }
</span><span class="cx">
</span></span></pre></div>
<a id="trunkSourceWTFChangeLog"></a>
<div class="modfile"><h4>Modified: trunk/Source/WTF/ChangeLog (206554 => 206555)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/WTF/ChangeLog 2016-09-28 21:53:53 UTC (rev 206554)
+++ trunk/Source/WTF/ChangeLog 2016-09-28 21:55:53 UTC (rev 206555)
</span><span class="lines">@@ -1,3 +1,17 @@
</span><ins>+2016-09-28 Filip Pizlo <fpizlo@apple.com>
+
+ The write barrier should be down with TSO
+ https://bugs.webkit.org/show_bug.cgi?id=162316
+
+ Reviewed by Geoffrey Garen.
+
+ Added clearRange(), which quickly clears a range of bits. This turned out to be useful for
+ a DFG optimization pass.
+
+ * wtf/FastBitVector.cpp:
+ (WTF::FastBitVector::clearRange):
+ * wtf/FastBitVector.h:
+
</ins><span class="cx"> 2016-09-28 Mark Lam <mark.lam@apple.com>
</span><span class="cx">
</span><span class="cx"> Fix race condition in StringView's UnderlyingString lifecycle management.
</span></span></pre></div>
<a id="trunkSourceWTFwtfFastBitVectorcpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/WTF/wtf/FastBitVector.cpp (206554 => 206555)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/WTF/wtf/FastBitVector.cpp 2016-09-28 21:53:53 UTC (rev 206554)
+++ trunk/Source/WTF/wtf/FastBitVector.cpp 2016-09-28 21:55:53 UTC (rev 206555)
</span><span class="lines">@@ -53,5 +53,24 @@
</span><span class="cx"> m_words = newArray;
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+void FastBitVector::clearRange(size_t begin, size_t end)
+{
+ if (end - begin < 32) {
+ for (size_t i = begin; i < end; ++i)
+ at(i) = false;
+ return;
+ }
+
+ size_t endBeginSlop = (begin + 31) & ~31;
+ size_t beginEndSlop = end & ~31;
+
+ for (size_t i = begin; i < endBeginSlop; ++i)
+ at(i) = false;
+ for (size_t i = beginEndSlop; i < end; ++i)
+ at(i) = false;
+ for (size_t i = endBeginSlop / 32; i < beginEndSlop / 32; ++i)
+ m_words.word(i) = 0;
+}
+
</ins><span class="cx"> } // namespace WTF
</span><span class="cx">
</span></span></pre></div>
<a id="trunkSourceWTFwtfFastBitVectorh"></a>
<div class="modfile"><h4>Modified: trunk/Source/WTF/wtf/FastBitVector.h (206554 => 206555)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/WTF/wtf/FastBitVector.h 2016-09-28 21:53:53 UTC (rev 206554)
+++ trunk/Source/WTF/wtf/FastBitVector.h 2016-09-28 21:55:53 UTC (rev 206555)
</span><span class="lines">@@ -468,6 +468,8 @@
</span><span class="cx"> {
</span><span class="cx"> m_words.clearAll();
</span><span class="cx"> }
</span><ins>+
+ WTF_EXPORT_PRIVATE void clearRange(size_t begin, size_t end);
</ins><span class="cx">
</span><span class="cx"> // Returns true if the contents of this bitvector changed.
</span><span class="cx"> template<typename OtherWords>
</span></span></pre>
</div>
</div>
</body>
</html>