<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
"http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head><meta http-equiv="content-type" content="text/html; charset=utf-8" />
<title>[197861] trunk/Source/JavaScriptCore</title>
</head>
<body>
<style type="text/css"><!--
#msg dl.meta { border: 1px #006 solid; background: #369; padding: 6px; color: #fff; }
#msg dl.meta dt { float: left; width: 6em; font-weight: bold; }
#msg dt:after { content:':';}
#msg dl, #msg dt, #msg ul, #msg li, #header, #footer, #logmsg { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt; }
#msg dl a { font-weight: bold}
#msg dl a:link { color:#fc3; }
#msg dl a:active { color:#ff0; }
#msg dl a:visited { color:#cc6; }
h3 { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt; font-weight: bold; }
#msg pre { overflow: auto; background: #ffc; border: 1px #fa0 solid; padding: 6px; }
#logmsg { background: #ffc; border: 1px #fa0 solid; padding: 1em 1em 0 1em; }
#logmsg p, #logmsg pre, #logmsg blockquote { margin: 0 0 1em 0; }
#logmsg p, #logmsg li, #logmsg dt, #logmsg dd { line-height: 14pt; }
#logmsg h1, #logmsg h2, #logmsg h3, #logmsg h4, #logmsg h5, #logmsg h6 { margin: .5em 0; }
#logmsg h1:first-child, #logmsg h2:first-child, #logmsg h3:first-child, #logmsg h4:first-child, #logmsg h5:first-child, #logmsg h6:first-child { margin-top: 0; }
#logmsg ul, #logmsg ol { padding: 0; list-style-position: inside; margin: 0 0 0 1em; }
#logmsg ul { text-indent: -1em; padding-left: 1em; }#logmsg ol { text-indent: -1.5em; padding-left: 1.5em; }
#logmsg > ul, #logmsg > ol { margin: 0 0 1em 0; }
#logmsg pre { background: #eee; padding: 1em; }
#logmsg blockquote { border: 1px solid #fa0; border-left-width: 10px; padding: 1em 1em 0 1em; background: white;}
#logmsg dl { margin: 0; }
#logmsg dt { font-weight: bold; }
#logmsg dd { margin: 0; padding: 0 0 0.5em 0; }
#logmsg dd:before { content:'\00bb';}
#logmsg table { border-spacing: 0px; border-collapse: collapse; border-top: 4px solid #fa0; border-bottom: 1px solid #fa0; background: #fff; }
#logmsg table th { text-align: left; font-weight: normal; padding: 0.2em 0.5em; border-top: 1px dotted #fa0; }
#logmsg table td { text-align: right; border-top: 1px dotted #fa0; padding: 0.2em 0.5em; }
#logmsg table thead th { text-align: center; border-bottom: 1px solid #fa0; }
#logmsg table th.Corner { text-align: left; }
#logmsg hr { border: none 0; border-top: 2px dashed #fa0; height: 1px; }
#header, #footer { color: #fff; background: #636; border: 1px #300 solid; padding: 6px; }
#patch { width: 100%; }
#patch h4 {font-family: verdana,arial,helvetica,sans-serif;font-size:10pt;padding:8px;background:#369;color:#fff;margin:0;}
#patch .propset h4, #patch .binary h4 {margin:0;}
#patch pre {padding:0;line-height:1.2em;margin:0;}
#patch .diff {width:100%;background:#eee;padding: 0 0 10px 0;overflow:auto;}
#patch .propset .diff, #patch .binary .diff {padding:10px 0;}
#patch span {display:block;padding:0 10px;}
#patch .modfile, #patch .addfile, #patch .delfile, #patch .propset, #patch .binary, #patch .copfile {border:1px solid #ccc;margin:10px 0;}
#patch ins {background:#dfd;text-decoration:none;display:block;padding:0 10px;}
#patch del {background:#fdd;text-decoration:none;display:block;padding:0 10px;}
#patch .lines, .info {color:#888;background:#fff;}
--></style>
<div id="msg">
<dl class="meta">
<dt>Revision</dt> <dd><a href="http://trac.webkit.org/projects/webkit/changeset/197861">197861</a></dd>
<dt>Author</dt> <dd>benjamin@webkit.org</dd>
<dt>Date</dt> <dd>2016-03-09 09:51:38 -0800 (Wed, 09 Mar 2016)</dd>
</dl>
<h3>Log Message</h3>
<pre>[JSC] Pick how to OSR Enter to FTL at runtime instead of compile time
https://bugs.webkit.org/show_bug.cgi?id=155217
Reviewed by Filip Pizlo.
This patch addresses 2 types of problems with tiering up to FTL
with OSR Entry in a loop:
-When there are nested loops, it is generally valuable to enter
an outer loop rather than an inner loop.
-When tiering up at a point that cannot OSR Enter, we are at
the mercy of the outer loop frequency to compile the right
entry point.
The first case is significant in the test "gaussian-blur".
That test has 4 nested loops. When we have an OSR Entry,
the analysis phases have to be pesimistic where we enter:
we do not really know what constraint can be proven from
the DFG code that was running.
In "gaussian-blur", integer-range analysis removes pretty
much all overflow checks in the inner loops of where we entered.
The more outside we enter, the better code we generate.
Since we spend the most iterations in the inner loop, we naturally
tend to OSR Enter into the 2 most inner loops, making the most
pessimistic assumptions.
To avoid such problems, I changed how we decide where to OSR Enter.
Previously, the last CheckTierUpAndOSREnter to cross the threshold
was where we take the entry point for FTL.
What happens now is that the entry point is not decied when
compiling the CheckTierUp variants. Instead, all the information
we need is gathered during compilation and keept on the JITCode
to be used at runtime.
When we try to tier up and decide to OSR Enter, we use the information
we have to pick a good outer loop for OSR Entry.
Now the problem is outer loop do not CheckTierUpAndOSREnter often,
wasting several miliseconds before entering the newly compiled FTL code.
To solve that, every CheckTierUpAndOSREnter has its own trigger that
bypass the counter. When the FTL Code is compiled, the trigger is set
and we enter through the right CheckTierUpAndOSREnter immediately.
---
This new mechanism also solves a problem of ai-astar.
When we try to tier up in ai-astar, we had nothing to compile until
the outer loop is reached.
To make sure we reached the CheckTierUpAndOSREnter in a reasonable time,
we had CheckTierUpWithNestedTriggerAndOSREnter with a special trigger.
With the new mechanism, we can do much better:
-When we keep hitting CheckTierUpInLoop, we now have all the information
we need to already start compiling the outer loop.
Instead of waiting for the outer loop to be reached a few times, we compile
it as soon as the inner loop is hammering CheckTierUpInLoop.
-With the new triggers, the very next time we hit the outer loop, we OSR Enter.
This allow us to compile what we need sooner and enter sooner.
* dfg/DFGAbstractInterpreterInlines.h:
(JSC::DFG::AbstractInterpreter<AbstractStateType>::executeEffects): Deleted.
* dfg/DFGClobberize.h:
(JSC::DFG::clobberize): Deleted.
* dfg/DFGDoesGC.cpp:
(JSC::DFG::doesGC): Deleted.
* dfg/DFGFixupPhase.cpp:
(JSC::DFG::FixupPhase::fixupNode): Deleted.
* dfg/DFGJITCode.h:
* dfg/DFGJITCompiler.cpp:
(JSC::DFG::JITCompiler::JITCompiler):
(JSC::DFG::JITCompiler::compileEntryExecutionFlag):
* dfg/DFGNodeType.h:
* dfg/DFGOperations.cpp:
* dfg/DFGOperations.h:
* dfg/DFGPlan.h:
(JSC::DFG::Plan::canTierUpAndOSREnter):
* dfg/DFGPredictionPropagationPhase.cpp:
(JSC::DFG::PredictionPropagationPhase::propagate): Deleted.
* dfg/DFGSafeToExecute.h:
(JSC::DFG::safeToExecute): Deleted.
* dfg/DFGSpeculativeJIT32_64.cpp:
(JSC::DFG::SpeculativeJIT::compile): Deleted.
* dfg/DFGSpeculativeJIT64.cpp:
(JSC::DFG::SpeculativeJIT::compile):
* dfg/DFGTierUpCheckInjectionPhase.cpp:
(JSC::DFG::TierUpCheckInjectionPhase::run):
(JSC::DFG::TierUpCheckInjectionPhase::buildNaturalLoopToLoopHintMap):
(JSC::DFG::TierUpCheckInjectionPhase::findLoopsContainingLoopHintWithoutOSREnter): Deleted.
* dfg/DFGToFTLForOSREntryDeferredCompilationCallback.cpp:
(JSC::DFG::ToFTLForOSREntryDeferredCompilationCallback::ToFTLForOSREntryDeferredCompilationCallback):
(JSC::DFG::Ref<ToFTLForOSREntryDeferredCompilationCallback>ToFTLForOSREntryDeferredCompilationCallback::create):
(JSC::DFG::ToFTLForOSREntryDeferredCompilationCallback::compilationDidBecomeReadyAsynchronously):
(JSC::DFG::ToFTLForOSREntryDeferredCompilationCallback::compilationDidComplete):
* dfg/DFGToFTLForOSREntryDeferredCompilationCallback.h:</pre>
<h3>Modified Paths</h3>
<ul>
<li><a href="#trunkSourceJavaScriptCoreChangeLog">trunk/Source/JavaScriptCore/ChangeLog</a></li>
<li><a href="#trunkSourceJavaScriptCoredfgDFGAbstractInterpreterInlinesh">trunk/Source/JavaScriptCore/dfg/DFGAbstractInterpreterInlines.h</a></li>
<li><a href="#trunkSourceJavaScriptCoredfgDFGClobberizeh">trunk/Source/JavaScriptCore/dfg/DFGClobberize.h</a></li>
<li><a href="#trunkSourceJavaScriptCoredfgDFGDoesGCcpp">trunk/Source/JavaScriptCore/dfg/DFGDoesGC.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoredfgDFGFixupPhasecpp">trunk/Source/JavaScriptCore/dfg/DFGFixupPhase.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoredfgDFGJITCodeh">trunk/Source/JavaScriptCore/dfg/DFGJITCode.h</a></li>
<li><a href="#trunkSourceJavaScriptCoredfgDFGJITCompilercpp">trunk/Source/JavaScriptCore/dfg/DFGJITCompiler.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoredfgDFGNodeTypeh">trunk/Source/JavaScriptCore/dfg/DFGNodeType.h</a></li>
<li><a href="#trunkSourceJavaScriptCoredfgDFGOperationscpp">trunk/Source/JavaScriptCore/dfg/DFGOperations.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoredfgDFGOperationsh">trunk/Source/JavaScriptCore/dfg/DFGOperations.h</a></li>
<li><a href="#trunkSourceJavaScriptCoredfgDFGPlanh">trunk/Source/JavaScriptCore/dfg/DFGPlan.h</a></li>
<li><a href="#trunkSourceJavaScriptCoredfgDFGPredictionPropagationPhasecpp">trunk/Source/JavaScriptCore/dfg/DFGPredictionPropagationPhase.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoredfgDFGSafeToExecuteh">trunk/Source/JavaScriptCore/dfg/DFGSafeToExecute.h</a></li>
<li><a href="#trunkSourceJavaScriptCoredfgDFGSpeculativeJIT32_64cpp">trunk/Source/JavaScriptCore/dfg/DFGSpeculativeJIT32_64.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoredfgDFGSpeculativeJIT64cpp">trunk/Source/JavaScriptCore/dfg/DFGSpeculativeJIT64.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoredfgDFGTierUpCheckInjectionPhasecpp">trunk/Source/JavaScriptCore/dfg/DFGTierUpCheckInjectionPhase.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoredfgDFGToFTLForOSREntryDeferredCompilationCallbackcpp">trunk/Source/JavaScriptCore/dfg/DFGToFTLForOSREntryDeferredCompilationCallback.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoredfgDFGToFTLForOSREntryDeferredCompilationCallbackh">trunk/Source/JavaScriptCore/dfg/DFGToFTLForOSREntryDeferredCompilationCallback.h</a></li>
</ul>
</div>
<div id="patch">
<h3>Diff</h3>
<a id="trunkSourceJavaScriptCoreChangeLog"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/ChangeLog (197860 => 197861)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/ChangeLog 2016-03-09 17:44:24 UTC (rev 197860)
+++ trunk/Source/JavaScriptCore/ChangeLog 2016-03-09 17:51:38 UTC (rev 197861)
</span><span class="lines">@@ -1,3 +1,105 @@
</span><ins>+2016-03-09 Benjamin Poulain <benjamin@webkit.org>
+
+ [JSC] Pick how to OSR Enter to FTL at runtime instead of compile time
+ https://bugs.webkit.org/show_bug.cgi?id=155217
+
+ Reviewed by Filip Pizlo.
+
+ This patch addresses 2 types of problems with tiering up to FTL
+ with OSR Entry in a loop:
+ -When there are nested loops, it is generally valuable to enter
+ an outer loop rather than an inner loop.
+ -When tiering up at a point that cannot OSR Enter, we are at
+ the mercy of the outer loop frequency to compile the right
+ entry point.
+
+ The first case is significant in the test "gaussian-blur".
+ That test has 4 nested loops. When we have an OSR Entry,
+ the analysis phases have to be pesimistic where we enter:
+ we do not really know what constraint can be proven from
+ the DFG code that was running.
+
+ In "gaussian-blur", integer-range analysis removes pretty
+ much all overflow checks in the inner loops of where we entered.
+ The more outside we enter, the better code we generate.
+
+ Since we spend the most iterations in the inner loop, we naturally
+ tend to OSR Enter into the 2 most inner loops, making the most
+ pessimistic assumptions.
+
+ To avoid such problems, I changed how we decide where to OSR Enter.
+ Previously, the last CheckTierUpAndOSREnter to cross the threshold
+ was where we take the entry point for FTL.
+
+ What happens now is that the entry point is not decied when
+ compiling the CheckTierUp variants. Instead, all the information
+ we need is gathered during compilation and keept on the JITCode
+ to be used at runtime.
+
+ When we try to tier up and decide to OSR Enter, we use the information
+ we have to pick a good outer loop for OSR Entry.
+
+ Now the problem is outer loop do not CheckTierUpAndOSREnter often,
+ wasting several miliseconds before entering the newly compiled FTL code.
+
+ To solve that, every CheckTierUpAndOSREnter has its own trigger that
+ bypass the counter. When the FTL Code is compiled, the trigger is set
+ and we enter through the right CheckTierUpAndOSREnter immediately.
+
+ ---
+
+ This new mechanism also solves a problem of ai-astar.
+ When we try to tier up in ai-astar, we had nothing to compile until
+ the outer loop is reached.
+
+ To make sure we reached the CheckTierUpAndOSREnter in a reasonable time,
+ we had CheckTierUpWithNestedTriggerAndOSREnter with a special trigger.
+
+ With the new mechanism, we can do much better:
+ -When we keep hitting CheckTierUpInLoop, we now have all the information
+ we need to already start compiling the outer loop.
+ Instead of waiting for the outer loop to be reached a few times, we compile
+ it as soon as the inner loop is hammering CheckTierUpInLoop.
+ -With the new triggers, the very next time we hit the outer loop, we OSR Enter.
+
+ This allow us to compile what we need sooner and enter sooner.
+
+ * dfg/DFGAbstractInterpreterInlines.h:
+ (JSC::DFG::AbstractInterpreter<AbstractStateType>::executeEffects): Deleted.
+ * dfg/DFGClobberize.h:
+ (JSC::DFG::clobberize): Deleted.
+ * dfg/DFGDoesGC.cpp:
+ (JSC::DFG::doesGC): Deleted.
+ * dfg/DFGFixupPhase.cpp:
+ (JSC::DFG::FixupPhase::fixupNode): Deleted.
+ * dfg/DFGJITCode.h:
+ * dfg/DFGJITCompiler.cpp:
+ (JSC::DFG::JITCompiler::JITCompiler):
+ (JSC::DFG::JITCompiler::compileEntryExecutionFlag):
+ * dfg/DFGNodeType.h:
+ * dfg/DFGOperations.cpp:
+ * dfg/DFGOperations.h:
+ * dfg/DFGPlan.h:
+ (JSC::DFG::Plan::canTierUpAndOSREnter):
+ * dfg/DFGPredictionPropagationPhase.cpp:
+ (JSC::DFG::PredictionPropagationPhase::propagate): Deleted.
+ * dfg/DFGSafeToExecute.h:
+ (JSC::DFG::safeToExecute): Deleted.
+ * dfg/DFGSpeculativeJIT32_64.cpp:
+ (JSC::DFG::SpeculativeJIT::compile): Deleted.
+ * dfg/DFGSpeculativeJIT64.cpp:
+ (JSC::DFG::SpeculativeJIT::compile):
+ * dfg/DFGTierUpCheckInjectionPhase.cpp:
+ (JSC::DFG::TierUpCheckInjectionPhase::run):
+ (JSC::DFG::TierUpCheckInjectionPhase::buildNaturalLoopToLoopHintMap):
+ (JSC::DFG::TierUpCheckInjectionPhase::findLoopsContainingLoopHintWithoutOSREnter): Deleted.
+ * dfg/DFGToFTLForOSREntryDeferredCompilationCallback.cpp:
+ (JSC::DFG::ToFTLForOSREntryDeferredCompilationCallback::ToFTLForOSREntryDeferredCompilationCallback):
+ (JSC::DFG::Ref<ToFTLForOSREntryDeferredCompilationCallback>ToFTLForOSREntryDeferredCompilationCallback::create):
+ (JSC::DFG::ToFTLForOSREntryDeferredCompilationCallback::compilationDidBecomeReadyAsynchronously):
+ (JSC::DFG::ToFTLForOSREntryDeferredCompilationCallback::compilationDidComplete):
+ * dfg/DFGToFTLForOSREntryDeferredCompilationCallback.h:
+
</ins><span class="cx"> 2016-03-08 Filip Pizlo <fpizlo@apple.com>
</span><span class="cx">
</span><span class="cx"> DFG should be able to constant-fold strings
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoredfgDFGAbstractInterpreterInlinesh"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/dfg/DFGAbstractInterpreterInlines.h (197860 => 197861)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/dfg/DFGAbstractInterpreterInlines.h 2016-03-09 17:44:24 UTC (rev 197860)
+++ trunk/Source/JavaScriptCore/dfg/DFGAbstractInterpreterInlines.h 2016-03-09 17:51:38 UTC (rev 197861)
</span><span class="lines">@@ -2660,7 +2660,6 @@
</span><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> case CheckTierUpAndOSREnter:
</span><del>- case CheckTierUpWithNestedTriggerAndOSREnter:
</del><span class="cx"> case LoopHint:
</span><span class="cx"> case ZombieHint:
</span><span class="cx"> case ExitOK:
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoredfgDFGClobberizeh"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/dfg/DFGClobberize.h (197860 => 197861)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/dfg/DFGClobberize.h 2016-03-09 17:44:24 UTC (rev 197860)
+++ trunk/Source/JavaScriptCore/dfg/DFGClobberize.h 2016-03-09 17:51:38 UTC (rev 197861)
</span><span class="lines">@@ -343,7 +343,6 @@
</span><span class="cx"> case CheckTierUpInLoop:
</span><span class="cx"> case CheckTierUpAtReturn:
</span><span class="cx"> case CheckTierUpAndOSREnter:
</span><del>- case CheckTierUpWithNestedTriggerAndOSREnter:
</del><span class="cx"> case LoopHint:
</span><span class="cx"> case Breakpoint:
</span><span class="cx"> case ProfileWillCall:
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoredfgDFGDoesGCcpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/dfg/DFGDoesGC.cpp (197860 => 197861)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/dfg/DFGDoesGC.cpp 2016-03-09 17:44:24 UTC (rev 197860)
+++ trunk/Source/JavaScriptCore/dfg/DFGDoesGC.cpp 2016-03-09 17:51:38 UTC (rev 197861)
</span><span class="lines">@@ -180,7 +180,6 @@
</span><span class="cx"> case CheckTierUpInLoop:
</span><span class="cx"> case CheckTierUpAtReturn:
</span><span class="cx"> case CheckTierUpAndOSREnter:
</span><del>- case CheckTierUpWithNestedTriggerAndOSREnter:
</del><span class="cx"> case LoopHint:
</span><span class="cx"> case StoreBarrier:
</span><span class="cx"> case InvalidationPoint:
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoredfgDFGFixupPhasecpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/dfg/DFGFixupPhase.cpp (197860 => 197861)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/dfg/DFGFixupPhase.cpp 2016-03-09 17:44:24 UTC (rev 197860)
+++ trunk/Source/JavaScriptCore/dfg/DFGFixupPhase.cpp 2016-03-09 17:51:38 UTC (rev 197861)
</span><span class="lines">@@ -1270,7 +1270,6 @@
</span><span class="cx"> case CheckTierUpInLoop:
</span><span class="cx"> case CheckTierUpAtReturn:
</span><span class="cx"> case CheckTierUpAndOSREnter:
</span><del>- case CheckTierUpWithNestedTriggerAndOSREnter:
</del><span class="cx"> case InvalidationPoint:
</span><span class="cx"> case CheckArray:
</span><span class="cx"> case CheckInBounds:
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoredfgDFGJITCodeh"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/dfg/DFGJITCode.h (197860 => 197861)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/dfg/DFGJITCode.h 2016-03-09 17:44:24 UTC (rev 197860)
+++ trunk/Source/JavaScriptCore/dfg/DFGJITCode.h 2016-03-09 17:51:38 UTC (rev 197861)
</span><span class="lines">@@ -138,9 +138,28 @@
</span><span class="cx"> DFG::VariableEventStream variableEventStream;
</span><span class="cx"> DFG::MinifiedGraph minifiedDFG;
</span><span class="cx"> #if ENABLE(FTL_JIT)
</span><del>- uint8_t nestedTriggerIsSet { 0 };
</del><span class="cx"> uint8_t neverExecutedEntry { 1 };
</span><ins>+
</ins><span class="cx"> UpperTierExecutionCounter tierUpCounter;
</span><ins>+
+ // For osrEntryPoint that are in inner loop, this maps their bytecode to the bytecode
+ // of the outerloop entry points in order (from innermost to outermost).
+ //
+ // The key may not always be a target for OSR Entry but the list in the value is guaranteed
+ // to be usable for OSR Entry.
+ HashMap<unsigned, Vector<unsigned>> tierUpInLoopHierarchy;
+
+ // Map each bytecode of CheckTierUpAndOSREnter to its stream index.
+ HashMap<unsigned, unsigned, WTF::IntHash<unsigned>, WTF::UnsignedWithZeroKeyHashTraits<unsigned>> bytecodeIndexToStreamIndex;
+
+ // Map each bytecode of CheckTierUpAndOSREnter to its trigger forcing OSR Entry.
+ // This can never be modified after it has been initialized since the addresses of the triggers
+ // are used by the JIT.
+ HashMap<unsigned, uint8_t> tierUpEntryTriggers;
+
+ // Set of bytecode that were the target of a TierUp operation.
+ HashSet<unsigned, WTF::IntHash<unsigned>, WTF::UnsignedWithZeroKeyHashTraits<unsigned>> tierUpEntrySeen;
+
</ins><span class="cx"> WriteBarrier<CodeBlock> m_osrEntryBlock;
</span><span class="cx"> unsigned osrEntryRetry;
</span><span class="cx"> bool abandonOSREntry;
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoredfgDFGJITCompilercpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/dfg/DFGJITCompiler.cpp (197860 => 197861)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/dfg/DFGJITCompiler.cpp 2016-03-09 17:44:24 UTC (rev 197860)
+++ trunk/Source/JavaScriptCore/dfg/DFGJITCompiler.cpp 2016-03-09 17:51:38 UTC (rev 197861)
</span><span class="lines">@@ -56,6 +56,11 @@
</span><span class="cx"> {
</span><span class="cx"> if (shouldDumpDisassembly() || m_graph.m_vm.m_perBytecodeProfiler)
</span><span class="cx"> m_disassembler = std::make_unique<Disassembler>(dfg);
</span><ins>+#if ENABLE(FTL_JIT)
+ m_jitCode->tierUpInLoopHierarchy = WTFMove(m_graph.m_plan.tierUpInLoopHierarchy);
+ for (unsigned tierUpBytecode : m_graph.m_plan.tierUpAndOSREnterBytecodes)
+ m_jitCode->tierUpEntryTriggers.add(tierUpBytecode, 0);
+#endif
</ins><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> JITCompiler::~JITCompiler()
</span><span class="lines">@@ -114,7 +119,7 @@
</span><span class="cx"> void JITCompiler::compileEntryExecutionFlag()
</span><span class="cx"> {
</span><span class="cx"> #if ENABLE(FTL_JIT)
</span><del>- if (m_graph.m_plan.canTierUpAndOSREnter)
</del><ins>+ if (m_graph.m_plan.canTierUpAndOSREnter())
</ins><span class="cx"> store8(TrustedImm32(0), &m_jitCode->neverExecutedEntry);
</span><span class="cx"> #endif // ENABLE(FTL_JIT)
</span><span class="cx"> }
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoredfgDFGNodeTypeh"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/dfg/DFGNodeType.h (197860 => 197861)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/dfg/DFGNodeType.h 2016-03-09 17:44:24 UTC (rev 197860)
+++ trunk/Source/JavaScriptCore/dfg/DFGNodeType.h 2016-03-09 17:51:38 UTC (rev 197861)
</span><span class="lines">@@ -95,7 +95,6 @@
</span><span class="cx"> /* Tier-up checks from the DFG to the FTL. */\
</span><span class="cx"> macro(CheckTierUpInLoop, NodeMustGenerate) \
</span><span class="cx"> macro(CheckTierUpAndOSREnter, NodeMustGenerate) \
</span><del>- macro(CheckTierUpWithNestedTriggerAndOSREnter, NodeMustGenerate) \
</del><span class="cx"> macro(CheckTierUpAtReturn, NodeMustGenerate) \
</span><span class="cx"> \
</span><span class="cx"> /* Get the value of a local variable, without linking into the VariableAccessData */\
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoredfgDFGOperationscpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/dfg/DFGOperations.cpp (197860 => 197861)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/dfg/DFGOperations.cpp 2016-03-09 17:44:24 UTC (rev 197860)
+++ trunk/Source/JavaScriptCore/dfg/DFGOperations.cpp 2016-03-09 17:51:38 UTC (rev 197861)
</span><span class="lines">@@ -1554,7 +1554,7 @@
</span><span class="cx"> codeBlock, CompilationDeferred);
</span><span class="cx"> }
</span><span class="cx">
</span><del>-static void triggerTierUpNowCommon(ExecState* exec, bool inLoop)
</del><ins>+void JIT_OPERATION triggerTierUpNow(ExecState* exec)
</ins><span class="cx"> {
</span><span class="cx"> VM* vm = &exec->vm();
</span><span class="cx"> NativeCallFrameTracer tracer(vm, exec);
</span><span class="lines">@@ -1573,45 +1573,64 @@
</span><span class="cx"> *codeBlock, ": Entered triggerTierUpNow with executeCounter = ",
</span><span class="cx"> jitCode->tierUpCounter, "\n");
</span><span class="cx"> }
</span><del>- if (inLoop)
- jitCode->nestedTriggerIsSet = 1;
</del><span class="cx">
</span><span class="cx"> if (shouldTriggerFTLCompile(codeBlock, jitCode))
</span><span class="cx"> triggerFTLReplacementCompile(vm, codeBlock, jitCode);
</span><del>-}
</del><span class="cx">
</span><del>-void JIT_OPERATION triggerTierUpNow(ExecState* exec)
-{
- triggerTierUpNowCommon(exec, false);
</del><ins>+ if (codeBlock->hasOptimizedReplacement()) {
+ if (jitCode->tierUpEntryTriggers.isEmpty()) {
+ // There is nothing more we can do, the only way this will be entered
+ // is through the function entry point.
+ jitCode->dontOptimizeAnytimeSoon(codeBlock);
+ return;
+ }
+ if (jitCode->osrEntryBlock() && jitCode->tierUpEntryTriggers.size() == 1) {
+ // There is only one outer loop and its trigger must have been set
+ // when the plan completed.
+ // Exiting the inner loop is useless, we can ignore the counter and leave
+ // the trigger do its job.
+ jitCode->dontOptimizeAnytimeSoon(codeBlock);
+ return;
+ }
+ }
</ins><span class="cx"> }
</span><span class="cx">
</span><del>-void JIT_OPERATION triggerTierUpNowInLoop(ExecState* exec)
</del><ins>+static char* tierUpCommon(ExecState* exec, unsigned originBytecodeIndex, unsigned osrEntryBytecodeIndex)
</ins><span class="cx"> {
</span><del>- triggerTierUpNowCommon(exec, true);
-}
-
-char* JIT_OPERATION triggerOSREntryNow(
- ExecState* exec, int32_t bytecodeIndex, int32_t streamIndex)
-{
</del><span class="cx"> VM* vm = &exec->vm();
</span><del>- NativeCallFrameTracer tracer(vm, exec);
- DeferGC deferGC(vm->heap);
</del><span class="cx"> CodeBlock* codeBlock = exec->codeBlock();
</span><del>-
- if (codeBlock->jitType() != JITCode::DFGJIT) {
- dataLog("Unexpected code block in DFG->FTL tier-up: ", *codeBlock, "\n");
- RELEASE_ASSERT_NOT_REACHED();
- }
-
</del><ins>+
+ // Resolve any pending plan for OSR Enter on this function.
+ Worklist::State worklistState;
+ if (Worklist* worklist = existingGlobalFTLWorklistOrNull()) {
+ worklistState = worklist->completeAllReadyPlansForVM(
+ *vm, CompilationKey(codeBlock->baselineVersion(), FTLForOSREntryMode));
+ } else
+ worklistState = Worklist::NotKnown;
+
</ins><span class="cx"> JITCode* jitCode = codeBlock->jitCode()->dfg();
</span><del>- jitCode->nestedTriggerIsSet = 0;
-
- if (Options::verboseOSR()) {
- dataLog(
- *codeBlock, ": Entered triggerOSREntryNow with executeCounter = ",
- jitCode->tierUpCounter, "\n");
</del><ins>+ if (worklistState == Worklist::Compiling) {
+ jitCode->setOptimizationThresholdBasedOnCompilationResult(
+ codeBlock, CompilationDeferred);
+ return nullptr;
</ins><span class="cx"> }
</span><del>-
</del><ins>+
+ if (worklistState == Worklist::Compiled) {
+ // This means that compilation failed and we already set the thresholds.
+ if (Options::verboseOSR())
+ dataLog("Code block ", *codeBlock, " was compiled but it doesn't have an optimized replacement.\n");
+ return nullptr;
+ }
+
+ // If we can OSR Enter, do it right away.
+ if (originBytecodeIndex == osrEntryBytecodeIndex) {
+ unsigned streamIndex = jitCode->bytecodeIndexToStreamIndex.get(originBytecodeIndex);
+ if (CodeBlock* entryBlock = jitCode->osrEntryBlock()) {
+ if (void* address = FTL::prepareOSREntry(exec, codeBlock, entryBlock, originBytecodeIndex, streamIndex))
+ return static_cast<char*>(address);
+ }
+ }
+
</ins><span class="cx"> // - If we don't have an FTL code block, then try to compile one.
</span><span class="cx"> // - If we do have an FTL code block, then try to enter for a while.
</span><span class="cx"> // - If we couldn't enter for a while, then trigger OSR entry.
</span><span class="lines">@@ -1630,29 +1649,13 @@
</span><span class="cx"> return nullptr;
</span><span class="cx"> }
</span><span class="cx"> }
</span><del>-
</del><ins>+
</ins><span class="cx"> // It's time to try to compile code for OSR entry.
</span><del>- Worklist::State worklistState;
- if (Worklist* worklist = existingGlobalFTLWorklistOrNull()) {
- worklistState = worklist->completeAllReadyPlansForVM(
- *vm, CompilationKey(codeBlock->baselineVersion(), FTLForOSREntryMode));
- } else
- worklistState = Worklist::NotKnown;
-
- if (worklistState == Worklist::Compiling) {
- jitCode->setOptimizationThresholdBasedOnCompilationResult(
- codeBlock, CompilationDeferred);
- return nullptr;
- }
-
</del><span class="cx"> if (CodeBlock* entryBlock = jitCode->osrEntryBlock()) {
</span><del>- void* address = FTL::prepareOSREntry(
- exec, codeBlock, entryBlock, bytecodeIndex, streamIndex);
- if (address)
- return static_cast<char*>(address);
-
</del><span class="cx"> if (jitCode->osrEntryRetry < Options::ftlOSREntryRetryThreshold()) {
</span><span class="cx"> jitCode->osrEntryRetry++;
</span><ins>+ jitCode->setOptimizationThresholdBasedOnCompilationResult(
+ codeBlock, CompilationDeferred);
</ins><span class="cx"> return nullptr;
</span><span class="cx"> }
</span><span class="cx">
</span><span class="lines">@@ -1660,33 +1663,47 @@
</span><span class="cx"> entryCode->countEntryFailure();
</span><span class="cx"> if (entryCode->entryFailureCount() <
</span><span class="cx"> Options::ftlOSREntryFailureCountForReoptimization()) {
</span><del>- jitCode->optimizeSoon(codeBlock);
</del><ins>+ jitCode->setOptimizationThresholdBasedOnCompilationResult(
+ codeBlock, CompilationDeferred);
</ins><span class="cx"> return nullptr;
</span><span class="cx"> }
</span><del>-
</del><ins>+
</ins><span class="cx"> // OSR entry failed. Oh no! This implies that we need to retry. We retry
</span><span class="cx"> // without exponential backoff and we only do this for the entry code block.
</span><ins>+ unsigned osrEntryBytecode = entryBlock->jitCode()->ftlForOSREntry()->bytecodeIndex();
</ins><span class="cx"> jitCode->clearOSREntryBlock();
</span><span class="cx"> jitCode->osrEntryRetry = 0;
</span><ins>+ jitCode->tierUpEntryTriggers.set(osrEntryBytecode, 0);
+ jitCode->setOptimizationThresholdBasedOnCompilationResult(
+ codeBlock, CompilationDeferred);
</ins><span class="cx"> return nullptr;
</span><span class="cx"> }
</span><del>-
- if (worklistState == Worklist::Compiled) {
- // This means that compilation failed and we already set the thresholds.
- if (Options::verboseOSR())
- dataLog("Code block ", *codeBlock, " was compiled but it doesn't have an optimized replacement.\n");
- return nullptr;
</del><ins>+
+ unsigned streamIndex = jitCode->bytecodeIndexToStreamIndex.get(osrEntryBytecodeIndex);
+ auto tierUpHierarchyEntry = jitCode->tierUpInLoopHierarchy.find(osrEntryBytecodeIndex);
+ if (tierUpHierarchyEntry != jitCode->tierUpInLoopHierarchy.end()) {
+ for (unsigned osrEntryCandidate : tierUpHierarchyEntry->value) {
+ if (jitCode->tierUpEntrySeen.contains(osrEntryCandidate)) {
+ osrEntryBytecodeIndex = osrEntryCandidate;
+ streamIndex = jitCode->bytecodeIndexToStreamIndex.get(osrEntryBytecodeIndex);
+ }
+ }
</ins><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> // We aren't compiling and haven't compiled anything for OSR entry. So, try to compile
</span><span class="cx"> // something.
</span><ins>+ auto triggerIterator = jitCode->tierUpEntryTriggers.find(osrEntryBytecodeIndex);
+ RELEASE_ASSERT(triggerIterator != jitCode->tierUpEntryTriggers.end());
+ uint8_t* triggerAddress = &(triggerIterator->value);
+
</ins><span class="cx"> Operands<JSValue> mustHandleValues;
</span><span class="cx"> jitCode->reconstruct(
</span><del>- exec, codeBlock, CodeOrigin(bytecodeIndex), streamIndex, mustHandleValues);
</del><ins>+ exec, codeBlock, CodeOrigin(osrEntryBytecodeIndex), streamIndex, mustHandleValues);
</ins><span class="cx"> CodeBlock* replacementCodeBlock = codeBlock->newReplacement();
</span><ins>+
</ins><span class="cx"> CompilationResult forEntryResult = compile(
</span><del>- *vm, replacementCodeBlock, codeBlock, FTLForOSREntryMode, bytecodeIndex,
- mustHandleValues, ToFTLForOSREntryDeferredCompilationCallback::create());
</del><ins>+ *vm, replacementCodeBlock, codeBlock, FTLForOSREntryMode, osrEntryBytecodeIndex,
+ mustHandleValues, ToFTLForOSREntryDeferredCompilationCallback::create(triggerAddress));
</ins><span class="cx">
</span><span class="cx"> if (jitCode->neverExecutedEntry)
</span><span class="cx"> triggerFTLReplacementCompile(vm, codeBlock, jitCode);
</span><span class="lines">@@ -1701,10 +1718,66 @@
</span><span class="cx"> // entry will succeed unless we ran out of stack. It's not clear what we should do.
</span><span class="cx"> // We signal to try again after a while if that happens.
</span><span class="cx"> void* address = FTL::prepareOSREntry(
</span><del>- exec, codeBlock, jitCode->osrEntryBlock(), bytecodeIndex, streamIndex);
</del><ins>+ exec, codeBlock, jitCode->osrEntryBlock(), originBytecodeIndex, streamIndex);
</ins><span class="cx"> return static_cast<char*>(address);
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+void JIT_OPERATION triggerTierUpNowInLoop(ExecState* exec, unsigned bytecodeIndex)
+{
+ VM* vm = &exec->vm();
+ NativeCallFrameTracer tracer(vm, exec);
+ DeferGC deferGC(vm->heap);
+ CodeBlock* codeBlock = exec->codeBlock();
+
+ if (codeBlock->jitType() != JITCode::DFGJIT) {
+ dataLog("Unexpected code block in DFG->FTL tier-up: ", *codeBlock, "\n");
+ RELEASE_ASSERT_NOT_REACHED();
+ }
+
+ JITCode* jitCode = codeBlock->jitCode()->dfg();
+
+ if (Options::verboseOSR()) {
+ dataLog(
+ *codeBlock, ": Entered triggerTierUpNowInLoop with executeCounter = ",
+ jitCode->tierUpCounter, "\n");
+ }
+
+ auto tierUpHierarchyEntry = jitCode->tierUpInLoopHierarchy.find(bytecodeIndex);
+ if (tierUpHierarchyEntry != jitCode->tierUpInLoopHierarchy.end()
+ && !tierUpHierarchyEntry->value.isEmpty()) {
+ tierUpCommon(exec, bytecodeIndex, tierUpHierarchyEntry->value.first());
+ } else if (shouldTriggerFTLCompile(codeBlock, jitCode))
+ triggerFTLReplacementCompile(vm, codeBlock, jitCode);
+
+ // Since we cannot OSR Enter here, the default "optimizeSoon()" is not useful.
+ if (codeBlock->hasOptimizedReplacement())
+ jitCode->setOptimizationThresholdBasedOnCompilationResult(codeBlock, CompilationDeferred);
+}
+
+char* JIT_OPERATION triggerOSREntryNow(ExecState* exec, unsigned bytecodeIndex)
+{
+ VM* vm = &exec->vm();
+ NativeCallFrameTracer tracer(vm, exec);
+ DeferGC deferGC(vm->heap);
+ CodeBlock* codeBlock = exec->codeBlock();
+
+ if (codeBlock->jitType() != JITCode::DFGJIT) {
+ dataLog("Unexpected code block in DFG->FTL tier-up: ", *codeBlock, "\n");
+ RELEASE_ASSERT_NOT_REACHED();
+ }
+
+ JITCode* jitCode = codeBlock->jitCode()->dfg();
+ jitCode->tierUpEntrySeen.add(bytecodeIndex);
+
+ if (Options::verboseOSR()) {
+ dataLog(
+ *codeBlock, ": Entered triggerOSREntryNow with executeCounter = ",
+ jitCode->tierUpCounter, "\n");
+ }
+
+ return tierUpCommon(exec, bytecodeIndex, bytecodeIndex);
+}
+
</ins><span class="cx"> #endif // ENABLE(FTL_JIT)
</span><span class="cx">
</span><span class="cx"> } // extern "C"
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoredfgDFGOperationsh"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/dfg/DFGOperations.h (197860 => 197861)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/dfg/DFGOperations.h 2016-03-09 17:44:24 UTC (rev 197860)
+++ trunk/Source/JavaScriptCore/dfg/DFGOperations.h 2016-03-09 17:51:38 UTC (rev 197861)
</span><span class="lines">@@ -169,8 +169,8 @@
</span><span class="cx">
</span><span class="cx"> #if ENABLE(FTL_JIT)
</span><span class="cx"> void JIT_OPERATION triggerTierUpNow(ExecState*) WTF_INTERNAL;
</span><del>-void JIT_OPERATION triggerTierUpNowInLoop(ExecState*) WTF_INTERNAL;
-char* JIT_OPERATION triggerOSREntryNow(ExecState*, int32_t bytecodeIndex, int32_t streamIndex) WTF_INTERNAL;
</del><ins>+void JIT_OPERATION triggerTierUpNowInLoop(ExecState*, unsigned bytecodeIndex) WTF_INTERNAL;
+char* JIT_OPERATION triggerOSREntryNow(ExecState*, unsigned bytecodeIndex) WTF_INTERNAL;
</ins><span class="cx"> #endif // ENABLE(FTL_JIT)
</span><span class="cx">
</span><span class="cx"> } // extern "C"
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoredfgDFGPlanh"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/dfg/DFGPlan.h (197860 => 197861)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/dfg/DFGPlan.h 2016-03-09 17:44:24 UTC (rev 197860)
+++ trunk/Source/JavaScriptCore/dfg/DFGPlan.h 2016-03-09 17:51:38 UTC (rev 197861)
</span><span class="lines">@@ -75,6 +75,8 @@
</span><span class="cx"> void checkLivenessAndVisitChildren(SlotVisitor&);
</span><span class="cx"> bool isKnownToBeLiveDuringGC();
</span><span class="cx"> void cancel();
</span><ins>+
+ bool canTierUpAndOSREnter() const { return !tierUpAndOSREnterBytecodes.isEmpty(); }
</ins><span class="cx">
</span><span class="cx"> VM& vm;
</span><span class="cx">
</span><span class="lines">@@ -99,8 +101,10 @@
</span><span class="cx"> DesiredTransitions transitions;
</span><span class="cx">
</span><span class="cx"> bool willTryToTierUp { false };
</span><del>- bool canTierUpAndOSREnter { false };
</del><span class="cx">
</span><ins>+ HashMap<unsigned, Vector<unsigned>> tierUpInLoopHierarchy;
+ Vector<unsigned> tierUpAndOSREnterBytecodes;
+
</ins><span class="cx"> enum Stage { Preparing, Compiling, Compiled, Ready, Cancelled };
</span><span class="cx"> Stage stage;
</span><span class="cx">
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoredfgDFGPredictionPropagationPhasecpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/dfg/DFGPredictionPropagationPhase.cpp (197860 => 197861)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/dfg/DFGPredictionPropagationPhase.cpp 2016-03-09 17:44:24 UTC (rev 197860)
+++ trunk/Source/JavaScriptCore/dfg/DFGPredictionPropagationPhase.cpp 2016-03-09 17:51:38 UTC (rev 197861)
</span><span class="lines">@@ -641,7 +641,6 @@
</span><span class="cx"> case CheckTierUpInLoop:
</span><span class="cx"> case CheckTierUpAtReturn:
</span><span class="cx"> case CheckTierUpAndOSREnter:
</span><del>- case CheckTierUpWithNestedTriggerAndOSREnter:
</del><span class="cx"> case InvalidationPoint:
</span><span class="cx"> case CheckInBounds:
</span><span class="cx"> case ValueToInt32:
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoredfgDFGSafeToExecuteh"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/dfg/DFGSafeToExecute.h (197860 => 197861)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/dfg/DFGSafeToExecute.h 2016-03-09 17:44:24 UTC (rev 197860)
+++ trunk/Source/JavaScriptCore/dfg/DFGSafeToExecute.h 2016-03-09 17:51:38 UTC (rev 197861)
</span><span class="lines">@@ -296,7 +296,6 @@
</span><span class="cx"> case CheckTierUpInLoop:
</span><span class="cx"> case CheckTierUpAtReturn:
</span><span class="cx"> case CheckTierUpAndOSREnter:
</span><del>- case CheckTierUpWithNestedTriggerAndOSREnter:
</del><span class="cx"> case LoopHint:
</span><span class="cx"> case StoreBarrier:
</span><span class="cx"> case InvalidationPoint:
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoredfgDFGSpeculativeJIT32_64cpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/dfg/DFGSpeculativeJIT32_64.cpp (197860 => 197861)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/dfg/DFGSpeculativeJIT32_64.cpp 2016-03-09 17:44:24 UTC (rev 197860)
+++ trunk/Source/JavaScriptCore/dfg/DFGSpeculativeJIT32_64.cpp 2016-03-09 17:51:38 UTC (rev 197861)
</span><span class="lines">@@ -4990,7 +4990,6 @@
</span><span class="cx"> case CheckTierUpInLoop:
</span><span class="cx"> case CheckTierUpAtReturn:
</span><span class="cx"> case CheckTierUpAndOSREnter:
</span><del>- case CheckTierUpWithNestedTriggerAndOSREnter:
</del><span class="cx"> case Int52Rep:
</span><span class="cx"> case FiatInt52:
</span><span class="cx"> case Int52Constant:
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoredfgDFGSpeculativeJIT64cpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/dfg/DFGSpeculativeJIT64.cpp (197860 => 197861)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/dfg/DFGSpeculativeJIT64.cpp 2016-03-09 17:44:24 UTC (rev 197860)
+++ trunk/Source/JavaScriptCore/dfg/DFGSpeculativeJIT64.cpp 2016-03-09 17:51:38 UTC (rev 197861)
</span><span class="lines">@@ -4979,7 +4979,8 @@
</span><span class="cx"> MacroAssembler::AbsoluteAddress(&m_jit.jitCode()->tierUpCounter.m_counter));
</span><span class="cx">
</span><span class="cx"> silentSpillAllRegisters(InvalidGPRReg);
</span><del>- m_jit.setupArgumentsExecState();
</del><ins>+ m_jit.setupArgumentsWithExecState(
+ TrustedImm32(node->origin.semantic.bytecodeIndex));
</ins><span class="cx"> appendCall(triggerTierUpNowInLoop);
</span><span class="cx"> silentFillAllRegisters(InvalidGPRReg);
</span><span class="cx">
</span><span class="lines">@@ -5002,28 +5003,28 @@
</span><span class="cx"> break;
</span><span class="cx"> }
</span><span class="cx">
</span><del>- case CheckTierUpAndOSREnter:
- case CheckTierUpWithNestedTriggerAndOSREnter: {
</del><ins>+ case CheckTierUpAndOSREnter: {
</ins><span class="cx"> ASSERT(!node->origin.semantic.inlineCallFrame);
</span><span class="cx">
</span><span class="cx"> GPRTemporary temp(this);
</span><span class="cx"> GPRReg tempGPR = temp.gpr();
</span><span class="cx">
</span><del>- MacroAssembler::Jump forceOSREntry;
- if (op == CheckTierUpWithNestedTriggerAndOSREnter)
- forceOSREntry = m_jit.branchTest8(MacroAssembler::NonZero, MacroAssembler::AbsoluteAddress(&m_jit.jitCode()->nestedTriggerIsSet));
</del><ins>+ unsigned bytecodeIndex = node->origin.semantic.bytecodeIndex;
+ auto triggerIterator = m_jit.jitCode()->tierUpEntryTriggers.find(bytecodeIndex);
+ RELEASE_ASSERT(triggerIterator != m_jit.jitCode()->tierUpEntryTriggers.end());
+ uint8_t* forceEntryTrigger = &(m_jit.jitCode()->tierUpEntryTriggers.find(bytecodeIndex)->value);
+ MacroAssembler::Jump forceOSREntry = m_jit.branchTest8(MacroAssembler::NonZero, MacroAssembler::AbsoluteAddress(forceEntryTrigger));
</ins><span class="cx">
</span><span class="cx"> MacroAssembler::Jump done = m_jit.branchAdd32(
</span><span class="cx"> MacroAssembler::Signed,
</span><span class="cx"> TrustedImm32(Options::ftlTierUpCounterIncrementForLoop()),
</span><span class="cx"> MacroAssembler::AbsoluteAddress(&m_jit.jitCode()->tierUpCounter.m_counter));
</span><span class="cx">
</span><del>- if (forceOSREntry.isSet())
- forceOSREntry.link(&m_jit);
</del><ins>+ forceOSREntry.link(&m_jit);
</ins><span class="cx"> silentSpillAllRegisters(tempGPR);
</span><del>- m_jit.setupArgumentsWithExecState(
- TrustedImm32(node->origin.semantic.bytecodeIndex),
- TrustedImm32(m_stream->size()));
</del><ins>+ unsigned streamIndex = m_stream->size();
+ m_jit.jitCode()->bytecodeIndexToStreamIndex.add(bytecodeIndex, streamIndex);
+ m_jit.setupArgumentsWithExecState(TrustedImm32(bytecodeIndex));
</ins><span class="cx"> appendCallSetResult(triggerOSREntryNow, tempGPR);
</span><span class="cx"> MacroAssembler::Jump dontEnter = m_jit.branchTestPtr(MacroAssembler::Zero, tempGPR);
</span><span class="cx"> m_jit.emitRestoreCalleeSaves();
</span><span class="lines">@@ -5038,7 +5039,6 @@
</span><span class="cx"> case CheckTierUpInLoop:
</span><span class="cx"> case CheckTierUpAtReturn:
</span><span class="cx"> case CheckTierUpAndOSREnter:
</span><del>- case CheckTierUpWithNestedTriggerAndOSREnter:
</del><span class="cx"> DFG_CRASH(m_jit.graph(), node, "Unexpected tier-up node");
</span><span class="cx"> break;
</span><span class="cx"> #endif // ENABLE(FTL_JIT)
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoredfgDFGTierUpCheckInjectionPhasecpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/dfg/DFGTierUpCheckInjectionPhase.cpp (197860 => 197861)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/dfg/DFGTierUpCheckInjectionPhase.cpp 2016-03-09 17:44:24 UTC (rev 197860)
+++ trunk/Source/JavaScriptCore/dfg/DFGTierUpCheckInjectionPhase.cpp 2016-03-09 17:51:38 UTC (rev 197861)
</span><span class="lines">@@ -65,49 +65,73 @@
</span><span class="cx"> if (!Options::useOSREntryToFTL())
</span><span class="cx"> level = FTL::CanCompile;
</span><span class="cx">
</span><del>- // First we find all the loops that contain a LoopHint for which we cannot OSR enter.
- // We use that information to decide if we need CheckTierUpAndOSREnter or CheckTierUpWithNestedTriggerAndOSREnter.
</del><span class="cx"> m_graph.ensureNaturalLoops();
</span><span class="cx"> NaturalLoops& naturalLoops = *m_graph.m_naturalLoops;
</span><ins>+ HashMap<const NaturalLoop*, unsigned> naturalLoopToLoopHint = buildNaturalLoopToLoopHintMap(naturalLoops);
</ins><span class="cx">
</span><del>- HashSet<const NaturalLoop*> loopsContainingLoopHintWithoutOSREnter = findLoopsContainingLoopHintWithoutOSREnter(naturalLoops, level);
</del><ins>+ HashMap<unsigned, LoopHintDescriptor> tierUpHierarchy;
</ins><span class="cx">
</span><del>- bool canTierUpAndOSREnter = false;
-
</del><span class="cx"> InsertionSet insertionSet(m_graph);
</span><span class="cx"> for (BlockIndex blockIndex = m_graph.numBlocks(); blockIndex--;) {
</span><span class="cx"> BasicBlock* block = m_graph.block(blockIndex);
</span><span class="cx"> if (!block)
</span><span class="cx"> continue;
</span><del>-
</del><ins>+
</ins><span class="cx"> for (unsigned nodeIndex = 0; nodeIndex < block->size(); ++nodeIndex) {
</span><span class="cx"> Node* node = block->at(nodeIndex);
</span><span class="cx"> if (node->op() != LoopHint)
</span><span class="cx"> continue;
</span><span class="cx">
</span><span class="cx"> NodeOrigin origin = node->origin;
</span><del>- if (canOSREnterAtLoopHint(level, block, nodeIndex)) {
- canTierUpAndOSREnter = true;
- const NaturalLoop* loop = naturalLoops.innerMostLoopOf(block);
- if (loop && loopsContainingLoopHintWithoutOSREnter.contains(loop))
- insertionSet.insertNode(nodeIndex + 1, SpecNone, CheckTierUpWithNestedTriggerAndOSREnter, origin);
- else
- insertionSet.insertNode(nodeIndex + 1, SpecNone, CheckTierUpAndOSREnter, origin);
- } else
- insertionSet.insertNode(nodeIndex + 1, SpecNone, CheckTierUpInLoop, origin);
</del><ins>+ bool canOSREnter = canOSREnterAtLoopHint(level, block, nodeIndex);
+
+ NodeType tierUpType = CheckTierUpAndOSREnter;
+ if (!canOSREnter)
+ tierUpType = CheckTierUpInLoop;
+ insertionSet.insertNode(nodeIndex + 1, SpecNone, tierUpType, origin);
+
+ unsigned bytecodeIndex = origin.semantic.bytecodeIndex;
+ if (canOSREnter)
+ m_graph.m_plan.tierUpAndOSREnterBytecodes.append(bytecodeIndex);
+
+ if (const NaturalLoop* loop = naturalLoops.innerMostLoopOf(block)) {
+ LoopHintDescriptor descriptor;
+ descriptor.canOSREnter = canOSREnter;
+
+ const NaturalLoop* outerLoop = loop;
+ while ((outerLoop = naturalLoops.innerMostOuterLoop(*outerLoop))) {
+ auto it = naturalLoopToLoopHint.find(outerLoop);
+ if (it != naturalLoopToLoopHint.end())
+ descriptor.osrEntryCandidates.append(it->value);
+ }
+ if (!descriptor.osrEntryCandidates.isEmpty())
+ tierUpHierarchy.add(bytecodeIndex, WTFMove(descriptor));
+ }
</ins><span class="cx"> break;
</span><span class="cx"> }
</span><del>-
</del><ins>+
</ins><span class="cx"> NodeAndIndex terminal = block->findTerminal();
</span><span class="cx"> if (terminal.node->isFunctionTerminal()) {
</span><span class="cx"> insertionSet.insertNode(
</span><span class="cx"> terminal.index, SpecNone, CheckTierUpAtReturn, terminal.node->origin);
</span><span class="cx"> }
</span><del>-
</del><ins>+
</ins><span class="cx"> insertionSet.execute(block);
</span><span class="cx"> }
</span><span class="cx">
</span><del>- m_graph.m_plan.canTierUpAndOSREnter = canTierUpAndOSREnter;
</del><ins>+ // Add all the candidates that can be OSR Entered.
+ for (auto entry : tierUpHierarchy) {
+ Vector<unsigned> tierUpCandidates;
+ for (unsigned bytecodeIndex : entry.value.osrEntryCandidates) {
+ auto descriptorIt = tierUpHierarchy.find(bytecodeIndex);
+ if (descriptorIt != tierUpHierarchy.end()
+ && descriptorIt->value.canOSREnter)
+ tierUpCandidates.append(bytecodeIndex);
+ }
+
+ if (!tierUpCandidates.isEmpty())
+ m_graph.m_plan.tierUpInLoopHierarchy.add(entry.key, WTFMove(tierUpCandidates));
+ }
</ins><span class="cx"> m_graph.m_plan.willTryToTierUp = true;
</span><span class="cx"> return true;
</span><span class="cx"> #else // ENABLE(FTL_JIT)
</span><span class="lines">@@ -118,6 +142,11 @@
</span><span class="cx">
</span><span class="cx"> private:
</span><span class="cx"> #if ENABLE(FTL_JIT)
</span><ins>+ struct LoopHintDescriptor {
+ Vector<unsigned> osrEntryCandidates;
+ bool canOSREnter;
+ };
+
</ins><span class="cx"> bool canOSREnterAtLoopHint(FTL::CapabilityLevel level, const BasicBlock* block, unsigned nodeIndex)
</span><span class="cx"> {
</span><span class="cx"> Node* node = block->at(nodeIndex);
</span><span class="lines">@@ -137,25 +166,24 @@
</span><span class="cx"> return true;
</span><span class="cx"> }
</span><span class="cx">
</span><del>- HashSet<const NaturalLoop*> findLoopsContainingLoopHintWithoutOSREnter(const NaturalLoops& naturalLoops, FTL::CapabilityLevel level)
</del><ins>+ HashMap<const NaturalLoop*, unsigned> buildNaturalLoopToLoopHintMap(const NaturalLoops& naturalLoops)
</ins><span class="cx"> {
</span><del>- HashSet<const NaturalLoop*> loopsContainingLoopHintWithoutOSREnter;
</del><ins>+ HashMap<const NaturalLoop*, unsigned> naturalLoopsToLoopHint;
+
</ins><span class="cx"> for (BasicBlock* block : m_graph.blocksInNaturalOrder()) {
</span><span class="cx"> for (unsigned nodeIndex = 0; nodeIndex < block->size(); ++nodeIndex) {
</span><span class="cx"> Node* node = block->at(nodeIndex);
</span><span class="cx"> if (node->op() != LoopHint)
</span><span class="cx"> continue;
</span><span class="cx">
</span><del>- if (!canOSREnterAtLoopHint(level, block, nodeIndex)) {
- const NaturalLoop* loop = naturalLoops.innerMostLoopOf(block);
- while (loop) {
- loopsContainingLoopHintWithoutOSREnter.add(loop);
- loop = naturalLoops.innerMostOuterLoop(*loop);
- }
</del><ins>+ if (const NaturalLoop* loop = naturalLoops.innerMostLoopOf(block)) {
+ unsigned bytecodeIndex = node->origin.semantic.bytecodeIndex;
+ naturalLoopsToLoopHint.add(loop, bytecodeIndex);
</ins><span class="cx"> }
</span><ins>+ break;
</ins><span class="cx"> }
</span><span class="cx"> }
</span><del>- return loopsContainingLoopHintWithoutOSREnter;
</del><ins>+ return naturalLoopsToLoopHint;
</ins><span class="cx"> }
</span><span class="cx"> #endif
</span><span class="cx"> };
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoredfgDFGToFTLForOSREntryDeferredCompilationCallbackcpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/dfg/DFGToFTLForOSREntryDeferredCompilationCallback.cpp (197860 => 197861)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/dfg/DFGToFTLForOSREntryDeferredCompilationCallback.cpp 2016-03-09 17:44:24 UTC (rev 197860)
+++ trunk/Source/JavaScriptCore/dfg/DFGToFTLForOSREntryDeferredCompilationCallback.cpp 2016-03-09 17:51:38 UTC (rev 197861)
</span><span class="lines">@@ -31,11 +31,13 @@
</span><span class="cx"> #include "CodeBlock.h"
</span><span class="cx"> #include "DFGJITCode.h"
</span><span class="cx"> #include "Executable.h"
</span><ins>+#include "FTLForOSREntryJITCode.h"
</ins><span class="cx"> #include "JSCInlines.h"
</span><span class="cx">
</span><span class="cx"> namespace JSC { namespace DFG {
</span><span class="cx">
</span><del>-ToFTLForOSREntryDeferredCompilationCallback::ToFTLForOSREntryDeferredCompilationCallback()
</del><ins>+ToFTLForOSREntryDeferredCompilationCallback::ToFTLForOSREntryDeferredCompilationCallback(uint8_t* forcedOSREntryTrigger)
+ : m_forcedOSREntryTrigger(forcedOSREntryTrigger)
</ins><span class="cx"> {
</span><span class="cx"> }
</span><span class="cx">
</span><span class="lines">@@ -43,9 +45,9 @@
</span><span class="cx"> {
</span><span class="cx"> }
</span><span class="cx">
</span><del>-Ref<ToFTLForOSREntryDeferredCompilationCallback>ToFTLForOSREntryDeferredCompilationCallback::create()
</del><ins>+Ref<ToFTLForOSREntryDeferredCompilationCallback>ToFTLForOSREntryDeferredCompilationCallback::create(uint8_t* forcedOSREntryTrigger)
</ins><span class="cx"> {
</span><del>- return adoptRef(*new ToFTLForOSREntryDeferredCompilationCallback());
</del><ins>+ return adoptRef(*new ToFTLForOSREntryDeferredCompilationCallback(forcedOSREntryTrigger));
</ins><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> void ToFTLForOSREntryDeferredCompilationCallback::compilationDidBecomeReadyAsynchronously(
</span><span class="lines">@@ -56,9 +58,8 @@
</span><span class="cx"> "Optimizing compilation of ", *codeBlock, " (for ", *profiledDFGCodeBlock,
</span><span class="cx"> ") did become ready.\n");
</span><span class="cx"> }
</span><del>-
- profiledDFGCodeBlock->jitCode()->dfg()->forceOptimizationSlowPathConcurrently(
- profiledDFGCodeBlock);
</del><ins>+
+ *m_forcedOSREntryTrigger = 1;
</ins><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> void ToFTLForOSREntryDeferredCompilationCallback::compilationDidComplete(
</span><span class="lines">@@ -73,12 +74,17 @@
</span><span class="cx"> JITCode* jitCode = profiledDFGCodeBlock->jitCode()->dfg();
</span><span class="cx">
</span><span class="cx"> switch (result) {
</span><del>- case CompilationSuccessful:
</del><ins>+ case CompilationSuccessful: {
</ins><span class="cx"> jitCode->setOSREntryBlock(*codeBlock->vm(), profiledDFGCodeBlock, codeBlock);
</span><ins>+ unsigned osrEntryBytecode = codeBlock->jitCode()->ftlForOSREntry()->bytecodeIndex();
+ jitCode->tierUpEntryTriggers.set(osrEntryBytecode, 1);
</ins><span class="cx"> break;
</span><ins>+ }
</ins><span class="cx"> case CompilationFailed:
</span><span class="cx"> jitCode->osrEntryRetry = 0;
</span><span class="cx"> jitCode->abandonOSREntry = true;
</span><ins>+ profiledDFGCodeBlock->jitCode()->dfg()->setOptimizationThresholdBasedOnCompilationResult(
+ profiledDFGCodeBlock, result);
</ins><span class="cx"> break;
</span><span class="cx"> case CompilationDeferred:
</span><span class="cx"> RELEASE_ASSERT_NOT_REACHED();
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoredfgDFGToFTLForOSREntryDeferredCompilationCallbackh"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/dfg/DFGToFTLForOSREntryDeferredCompilationCallback.h (197860 => 197861)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/dfg/DFGToFTLForOSREntryDeferredCompilationCallback.h 2016-03-09 17:44:24 UTC (rev 197860)
+++ trunk/Source/JavaScriptCore/dfg/DFGToFTLForOSREntryDeferredCompilationCallback.h 2016-03-09 17:51:38 UTC (rev 197861)
</span><span class="lines">@@ -40,15 +40,18 @@
</span><span class="cx">
</span><span class="cx"> class ToFTLForOSREntryDeferredCompilationCallback : public DeferredCompilationCallback {
</span><span class="cx"> protected:
</span><del>- ToFTLForOSREntryDeferredCompilationCallback();
</del><ins>+ ToFTLForOSREntryDeferredCompilationCallback(uint8_t* forcedOSREntryTrigger);
</ins><span class="cx">
</span><span class="cx"> public:
</span><span class="cx"> virtual ~ToFTLForOSREntryDeferredCompilationCallback();
</span><span class="cx">
</span><del>- static Ref<ToFTLForOSREntryDeferredCompilationCallback> create();
</del><ins>+ static Ref<ToFTLForOSREntryDeferredCompilationCallback> create(uint8_t* forcedOSREntryTrigger);
</ins><span class="cx">
</span><span class="cx"> virtual void compilationDidBecomeReadyAsynchronously(CodeBlock*, CodeBlock* profiledDFGCodeBlock);
</span><span class="cx"> virtual void compilationDidComplete(CodeBlock*, CodeBlock* profiledDFGCodeBlock, CompilationResult);
</span><ins>+
+private:
+ uint8_t* m_forcedOSREntryTrigger;
</ins><span class="cx"> };
</span><span class="cx">
</span><span class="cx"> } } // namespace JSC::DFG
</span></span></pre>
</div>
</div>
</body>
</html>