<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
"http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head><meta http-equiv="content-type" content="text/html; charset=utf-8" />
<title>[190220] trunk/Source/JavaScriptCore</title>
</head>
<body>
<style type="text/css"><!--
#msg dl.meta { border: 1px #006 solid; background: #369; padding: 6px; color: #fff; }
#msg dl.meta dt { float: left; width: 6em; font-weight: bold; }
#msg dt:after { content:':';}
#msg dl, #msg dt, #msg ul, #msg li, #header, #footer, #logmsg { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt; }
#msg dl a { font-weight: bold}
#msg dl a:link { color:#fc3; }
#msg dl a:active { color:#ff0; }
#msg dl a:visited { color:#cc6; }
h3 { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt; font-weight: bold; }
#msg pre { overflow: auto; background: #ffc; border: 1px #fa0 solid; padding: 6px; }
#logmsg { background: #ffc; border: 1px #fa0 solid; padding: 1em 1em 0 1em; }
#logmsg p, #logmsg pre, #logmsg blockquote { margin: 0 0 1em 0; }
#logmsg p, #logmsg li, #logmsg dt, #logmsg dd { line-height: 14pt; }
#logmsg h1, #logmsg h2, #logmsg h3, #logmsg h4, #logmsg h5, #logmsg h6 { margin: .5em 0; }
#logmsg h1:first-child, #logmsg h2:first-child, #logmsg h3:first-child, #logmsg h4:first-child, #logmsg h5:first-child, #logmsg h6:first-child { margin-top: 0; }
#logmsg ul, #logmsg ol { padding: 0; list-style-position: inside; margin: 0 0 0 1em; }
#logmsg ul { text-indent: -1em; padding-left: 1em; }#logmsg ol { text-indent: -1.5em; padding-left: 1.5em; }
#logmsg > ul, #logmsg > ol { margin: 0 0 1em 0; }
#logmsg pre { background: #eee; padding: 1em; }
#logmsg blockquote { border: 1px solid #fa0; border-left-width: 10px; padding: 1em 1em 0 1em; background: white;}
#logmsg dl { margin: 0; }
#logmsg dt { font-weight: bold; }
#logmsg dd { margin: 0; padding: 0 0 0.5em 0; }
#logmsg dd:before { content:'\00bb';}
#logmsg table { border-spacing: 0px; border-collapse: collapse; border-top: 4px solid #fa0; border-bottom: 1px solid #fa0; background: #fff; }
#logmsg table th { text-align: left; font-weight: normal; padding: 0.2em 0.5em; border-top: 1px dotted #fa0; }
#logmsg table td { text-align: right; border-top: 1px dotted #fa0; padding: 0.2em 0.5em; }
#logmsg table thead th { text-align: center; border-bottom: 1px solid #fa0; }
#logmsg table th.Corner { text-align: left; }
#logmsg hr { border: none 0; border-top: 2px dashed #fa0; height: 1px; }
#header, #footer { color: #fff; background: #636; border: 1px #300 solid; padding: 6px; }
#patch { width: 100%; }
#patch h4 {font-family: verdana,arial,helvetica,sans-serif;font-size:10pt;padding:8px;background:#369;color:#fff;margin:0;}
#patch .propset h4, #patch .binary h4 {margin:0;}
#patch pre {padding:0;line-height:1.2em;margin:0;}
#patch .diff {width:100%;background:#eee;padding: 0 0 10px 0;overflow:auto;}
#patch .propset .diff, #patch .binary .diff {padding:10px 0;}
#patch span {display:block;padding:0 10px;}
#patch .modfile, #patch .addfile, #patch .delfile, #patch .propset, #patch .binary, #patch .copfile {border:1px solid #ccc;margin:10px 0;}
#patch ins {background:#dfd;text-decoration:none;display:block;padding:0 10px;}
#patch del {background:#fdd;text-decoration:none;display:block;padding:0 10px;}
#patch .lines, .info {color:#888;background:#fff;}
--></style>
<div id="msg">
<dl class="meta">
<dt>Revision</dt> <dd><a href="http://trac.webkit.org/projects/webkit/changeset/190220">190220</a></dd>
<dt>Author</dt> <dd>msaboff@apple.com</dd>
<dt>Date</dt> <dd>2015-09-24 14:42:59 -0700 (Thu, 24 Sep 2015)</dd>
</dl>
<h3>Log Message</h3>
<pre>[ES6] Implement tail calls in the DFG
https://bugs.webkit.org/show_bug.cgi?id=148663
Reviewed by Filip Pizlo.
jsc-tailcall: Implement the tail call opcodes in the DFG
https://bugs.webkit.org/show_bug.cgi?id=146850
This patch adds support for tail calls in the DFG. This requires a slightly high number of nodes:
- TailCall and TailCallVarargs are straightforward. They are terminal
nodes and have the semantics of an actual tail call.
- TailCallInlinedCaller and TailCallVarargsInlinedCaller are here to perform a
tail call inside an inlined function. They are non terminal nodes,
and are performing the call as a regular call after popping an
appropriate number of inlined tail call frames.
- TailCallForwardVarargs and TailCallForwardVarargsInlinedCaller are the
extension of TailCallVarargs and TailCallVarargsInlinedCaller to enable
the varargs forwarding optimization so that we don't lose
performance with a tail call instead of a regular call.
This also required two broad kind of changes:
- Changes in the JIT itself (DFGSpeculativeJIT) are pretty
straightforward since they are just an extension of the baseline JIT
changes introduced previously.
- Changes in the runtime are mostly related with handling inline call
frames. The idea here is that we have a special TailCall type for
call frames that indicates to the various pieces of code walking the
inline call frame that they should (recursively) skip the caller in
their analysis.
* bytecode/CallMode.h:
(JSC::specializationKindFor):
* bytecode/CodeOrigin.cpp:
(JSC::CodeOrigin::inlineDepthForCallFrame):
(JSC::CodeOrigin::isApproximatelyEqualTo):
(JSC::CodeOrigin::approximateHash):
(JSC::CodeOrigin::inlineStack):
* bytecode/CodeOrigin.h:
* bytecode/InlineCallFrame.cpp:
(JSC::InlineCallFrame::dumpInContext):
(WTF::printInternal):
* bytecode/InlineCallFrame.h:
(JSC::InlineCallFrame::callModeFor):
(JSC::InlineCallFrame::kindFor):
(JSC::InlineCallFrame::varargsKindFor):
(JSC::InlineCallFrame::specializationKindFor):
(JSC::InlineCallFrame::isVarargs):
(JSC::InlineCallFrame::isTail):
(JSC::InlineCallFrame::computeCallerSkippingDeadFrames):
(JSC::InlineCallFrame::getCallerSkippingDeadFrames):
(JSC::InlineCallFrame::getCallerInlineFrameSkippingDeadFrames):
* dfg/DFGAbstractInterpreterInlines.h:
(JSC::DFG::AbstractInterpreter<AbstractStateType>::executeEffects):
* dfg/DFGArgumentsEliminationPhase.cpp:
* dfg/DFGBasicBlock.h:
(JSC::DFG::BasicBlock::findTerminal):
* dfg/DFGByteCodeParser.cpp:
(JSC::DFG::ByteCodeParser::inlineCallFrame):
(JSC::DFG::ByteCodeParser::allInlineFramesAreTailCalls):
(JSC::DFG::ByteCodeParser::currentCodeOrigin):
(JSC::DFG::ByteCodeParser::addCallWithoutSettingResult):
(JSC::DFG::ByteCodeParser::addCall):
(JSC::DFG::ByteCodeParser::getPredictionWithoutOSRExit):
(JSC::DFG::ByteCodeParser::getPrediction):
(JSC::DFG::ByteCodeParser::handleCall):
(JSC::DFG::ByteCodeParser::handleVarargsCall):
(JSC::DFG::ByteCodeParser::emitArgumentPhantoms):
(JSC::DFG::ByteCodeParser::inliningCost):
(JSC::DFG::ByteCodeParser::inlineCall):
(JSC::DFG::ByteCodeParser::attemptToInlineCall):
(JSC::DFG::ByteCodeParser::parseBlock):
(JSC::DFG::ByteCodeParser::InlineStackEntry::InlineStackEntry):
(JSC::DFG::ByteCodeParser::parseCodeBlock):
* dfg/DFGCapabilities.cpp:
(JSC::DFG::capabilityLevel):
* dfg/DFGClobberize.h:
(JSC::DFG::clobberize):
* dfg/DFGDoesGC.cpp:
(JSC::DFG::doesGC):
* dfg/DFGFixupPhase.cpp:
(JSC::DFG::FixupPhase::fixupNode):
* dfg/DFGGraph.cpp:
(JSC::DFG::Graph::isLiveInBytecode):
* dfg/DFGGraph.h:
(JSC::DFG::Graph::forAllLocalsLiveInBytecode):
* dfg/DFGInPlaceAbstractState.cpp:
(JSC::DFG::InPlaceAbstractState::mergeToSuccessors):
* dfg/DFGJITCompiler.cpp:
(JSC::DFG::JITCompiler::willCatchExceptionInMachineFrame):
* dfg/DFGLiveCatchVariablePreservationPhase.cpp:
(JSC::DFG::FlushLiveCatchVariablesInsertionPhase::willCatchException):
* dfg/DFGNode.h:
(JSC::DFG::Node::hasCallVarargsData):
(JSC::DFG::Node::isTerminal):
(JSC::DFG::Node::hasHeapPrediction):
* dfg/DFGNodeType.h:
* dfg/DFGOSRExitCompilerCommon.cpp:
(JSC::DFG::handleExitCounts):
(JSC::DFG::reifyInlinedCallFrames):
(JSC::DFG::osrWriteBarrier):
* dfg/DFGOSRExitPreparation.cpp:
(JSC::DFG::prepareCodeOriginForOSRExit):
* dfg/DFGOperations.cpp:
* dfg/DFGPreciseLocalClobberize.h:
(JSC::DFG::PreciseLocalClobberizeAdaptor::readTop):
* dfg/DFGPredictionPropagationPhase.cpp:
(JSC::DFG::PredictionPropagationPhase::propagate):
* dfg/DFGSafeToExecute.h:
(JSC::DFG::safeToExecute):
* dfg/DFGSpeculativeJIT32_64.cpp:
(JSC::DFG::SpeculativeJIT::emitCall):
(JSC::DFG::SpeculativeJIT::compile):
* dfg/DFGSpeculativeJIT64.cpp:
(JSC::DFG::SpeculativeJIT::emitCall):
(JSC::DFG::SpeculativeJIT::compile):
* dfg/DFGValidate.cpp:
(JSC::DFG::Validate::validateSSA):
* dfg/DFGVarargsForwardingPhase.cpp:
* interpreter/CallFrame.cpp:
(JSC::CallFrame::bytecodeOffset):
* interpreter/StackVisitor.cpp:
(JSC::StackVisitor::gotoNextFrame):</pre>
<h3>Modified Paths</h3>
<ul>
<li><a href="#trunkSourceJavaScriptCoreChangeLog">trunk/Source/JavaScriptCore/ChangeLog</a></li>
<li><a href="#trunkSourceJavaScriptCorebytecodeCallModeh">trunk/Source/JavaScriptCore/bytecode/CallMode.h</a></li>
<li><a href="#trunkSourceJavaScriptCorebytecodeCodeOrigincpp">trunk/Source/JavaScriptCore/bytecode/CodeOrigin.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCorebytecodeCodeOriginh">trunk/Source/JavaScriptCore/bytecode/CodeOrigin.h</a></li>
<li><a href="#trunkSourceJavaScriptCorebytecodeInlineCallFramecpp">trunk/Source/JavaScriptCore/bytecode/InlineCallFrame.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCorebytecodeInlineCallFrameh">trunk/Source/JavaScriptCore/bytecode/InlineCallFrame.h</a></li>
<li><a href="#trunkSourceJavaScriptCoredfgDFGAbstractInterpreterInlinesh">trunk/Source/JavaScriptCore/dfg/DFGAbstractInterpreterInlines.h</a></li>
<li><a href="#trunkSourceJavaScriptCoredfgDFGArgumentsEliminationPhasecpp">trunk/Source/JavaScriptCore/dfg/DFGArgumentsEliminationPhase.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoredfgDFGBasicBlockh">trunk/Source/JavaScriptCore/dfg/DFGBasicBlock.h</a></li>
<li><a href="#trunkSourceJavaScriptCoredfgDFGByteCodeParsercpp">trunk/Source/JavaScriptCore/dfg/DFGByteCodeParser.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoredfgDFGCapabilitiescpp">trunk/Source/JavaScriptCore/dfg/DFGCapabilities.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoredfgDFGClobberizeh">trunk/Source/JavaScriptCore/dfg/DFGClobberize.h</a></li>
<li><a href="#trunkSourceJavaScriptCoredfgDFGDoesGCcpp">trunk/Source/JavaScriptCore/dfg/DFGDoesGC.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoredfgDFGFixupPhasecpp">trunk/Source/JavaScriptCore/dfg/DFGFixupPhase.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoredfgDFGGraphcpp">trunk/Source/JavaScriptCore/dfg/DFGGraph.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoredfgDFGGraphh">trunk/Source/JavaScriptCore/dfg/DFGGraph.h</a></li>
<li><a href="#trunkSourceJavaScriptCoredfgDFGInPlaceAbstractStatecpp">trunk/Source/JavaScriptCore/dfg/DFGInPlaceAbstractState.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoredfgDFGJITCompilercpp">trunk/Source/JavaScriptCore/dfg/DFGJITCompiler.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoredfgDFGLiveCatchVariablePreservationPhasecpp">trunk/Source/JavaScriptCore/dfg/DFGLiveCatchVariablePreservationPhase.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoredfgDFGNodeh">trunk/Source/JavaScriptCore/dfg/DFGNode.h</a></li>
<li><a href="#trunkSourceJavaScriptCoredfgDFGNodeTypeh">trunk/Source/JavaScriptCore/dfg/DFGNodeType.h</a></li>
<li><a href="#trunkSourceJavaScriptCoredfgDFGOSRExitCompilerCommoncpp">trunk/Source/JavaScriptCore/dfg/DFGOSRExitCompilerCommon.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoredfgDFGOSRExitPreparationcpp">trunk/Source/JavaScriptCore/dfg/DFGOSRExitPreparation.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoredfgDFGOperationscpp">trunk/Source/JavaScriptCore/dfg/DFGOperations.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoredfgDFGPreciseLocalClobberizeh">trunk/Source/JavaScriptCore/dfg/DFGPreciseLocalClobberize.h</a></li>
<li><a href="#trunkSourceJavaScriptCoredfgDFGPredictionPropagationPhasecpp">trunk/Source/JavaScriptCore/dfg/DFGPredictionPropagationPhase.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoredfgDFGSafeToExecuteh">trunk/Source/JavaScriptCore/dfg/DFGSafeToExecute.h</a></li>
<li><a href="#trunkSourceJavaScriptCoredfgDFGSpeculativeJIT32_64cpp">trunk/Source/JavaScriptCore/dfg/DFGSpeculativeJIT32_64.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoredfgDFGSpeculativeJIT64cpp">trunk/Source/JavaScriptCore/dfg/DFGSpeculativeJIT64.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoredfgDFGValidatecpp">trunk/Source/JavaScriptCore/dfg/DFGValidate.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoredfgDFGVarargsForwardingPhasecpp">trunk/Source/JavaScriptCore/dfg/DFGVarargsForwardingPhase.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoreinterpreterCallFramecpp">trunk/Source/JavaScriptCore/interpreter/CallFrame.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoreinterpreterStackVisitorcpp">trunk/Source/JavaScriptCore/interpreter/StackVisitor.cpp</a></li>
</ul>
</div>
<div id="patch">
<h3>Diff</h3>
<a id="trunkSourceJavaScriptCoreChangeLog"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/ChangeLog (190219 => 190220)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/ChangeLog        2015-09-24 21:39:36 UTC (rev 190219)
+++ trunk/Source/JavaScriptCore/ChangeLog        2015-09-24 21:42:59 UTC (rev 190220)
</span><span class="lines">@@ -1,3 +1,133 @@
</span><ins>+2015-09-24 Michael Saboff <msaboff@apple.com>
+
+ [ES6] Implement tail calls in the DFG
+ https://bugs.webkit.org/show_bug.cgi?id=148663
+
+ Reviewed by Filip Pizlo.
+
+ jsc-tailcall: Implement the tail call opcodes in the DFG
+ https://bugs.webkit.org/show_bug.cgi?id=146850
+
+ This patch adds support for tail calls in the DFG. This requires a slightly high number of nodes:
+
+ - TailCall and TailCallVarargs are straightforward. They are terminal
+ nodes and have the semantics of an actual tail call.
+
+ - TailCallInlinedCaller and TailCallVarargsInlinedCaller are here to perform a
+ tail call inside an inlined function. They are non terminal nodes,
+ and are performing the call as a regular call after popping an
+ appropriate number of inlined tail call frames.
+
+ - TailCallForwardVarargs and TailCallForwardVarargsInlinedCaller are the
+ extension of TailCallVarargs and TailCallVarargsInlinedCaller to enable
+ the varargs forwarding optimization so that we don't lose
+ performance with a tail call instead of a regular call.
+
+ This also required two broad kind of changes:
+
+ - Changes in the JIT itself (DFGSpeculativeJIT) are pretty
+ straightforward since they are just an extension of the baseline JIT
+ changes introduced previously.
+
+ - Changes in the runtime are mostly related with handling inline call
+ frames. The idea here is that we have a special TailCall type for
+ call frames that indicates to the various pieces of code walking the
+ inline call frame that they should (recursively) skip the caller in
+ their analysis.
+
+ * bytecode/CallMode.h:
+ (JSC::specializationKindFor):
+ * bytecode/CodeOrigin.cpp:
+ (JSC::CodeOrigin::inlineDepthForCallFrame):
+ (JSC::CodeOrigin::isApproximatelyEqualTo):
+ (JSC::CodeOrigin::approximateHash):
+ (JSC::CodeOrigin::inlineStack):
+ * bytecode/CodeOrigin.h:
+ * bytecode/InlineCallFrame.cpp:
+ (JSC::InlineCallFrame::dumpInContext):
+ (WTF::printInternal):
+ * bytecode/InlineCallFrame.h:
+ (JSC::InlineCallFrame::callModeFor):
+ (JSC::InlineCallFrame::kindFor):
+ (JSC::InlineCallFrame::varargsKindFor):
+ (JSC::InlineCallFrame::specializationKindFor):
+ (JSC::InlineCallFrame::isVarargs):
+ (JSC::InlineCallFrame::isTail):
+ (JSC::InlineCallFrame::computeCallerSkippingDeadFrames):
+ (JSC::InlineCallFrame::getCallerSkippingDeadFrames):
+ (JSC::InlineCallFrame::getCallerInlineFrameSkippingDeadFrames):
+ * dfg/DFGAbstractInterpreterInlines.h:
+ (JSC::DFG::AbstractInterpreter<AbstractStateType>::executeEffects):
+ * dfg/DFGArgumentsEliminationPhase.cpp:
+ * dfg/DFGBasicBlock.h:
+ (JSC::DFG::BasicBlock::findTerminal):
+ * dfg/DFGByteCodeParser.cpp:
+ (JSC::DFG::ByteCodeParser::inlineCallFrame):
+ (JSC::DFG::ByteCodeParser::allInlineFramesAreTailCalls):
+ (JSC::DFG::ByteCodeParser::currentCodeOrigin):
+ (JSC::DFG::ByteCodeParser::addCallWithoutSettingResult):
+ (JSC::DFG::ByteCodeParser::addCall):
+ (JSC::DFG::ByteCodeParser::getPredictionWithoutOSRExit):
+ (JSC::DFG::ByteCodeParser::getPrediction):
+ (JSC::DFG::ByteCodeParser::handleCall):
+ (JSC::DFG::ByteCodeParser::handleVarargsCall):
+ (JSC::DFG::ByteCodeParser::emitArgumentPhantoms):
+ (JSC::DFG::ByteCodeParser::inliningCost):
+ (JSC::DFG::ByteCodeParser::inlineCall):
+ (JSC::DFG::ByteCodeParser::attemptToInlineCall):
+ (JSC::DFG::ByteCodeParser::parseBlock):
+ (JSC::DFG::ByteCodeParser::InlineStackEntry::InlineStackEntry):
+ (JSC::DFG::ByteCodeParser::parseCodeBlock):
+ * dfg/DFGCapabilities.cpp:
+ (JSC::DFG::capabilityLevel):
+ * dfg/DFGClobberize.h:
+ (JSC::DFG::clobberize):
+ * dfg/DFGDoesGC.cpp:
+ (JSC::DFG::doesGC):
+ * dfg/DFGFixupPhase.cpp:
+ (JSC::DFG::FixupPhase::fixupNode):
+ * dfg/DFGGraph.cpp:
+ (JSC::DFG::Graph::isLiveInBytecode):
+ * dfg/DFGGraph.h:
+ (JSC::DFG::Graph::forAllLocalsLiveInBytecode):
+ * dfg/DFGInPlaceAbstractState.cpp:
+ (JSC::DFG::InPlaceAbstractState::mergeToSuccessors):
+ * dfg/DFGJITCompiler.cpp:
+ (JSC::DFG::JITCompiler::willCatchExceptionInMachineFrame):
+ * dfg/DFGLiveCatchVariablePreservationPhase.cpp:
+ (JSC::DFG::FlushLiveCatchVariablesInsertionPhase::willCatchException):
+ * dfg/DFGNode.h:
+ (JSC::DFG::Node::hasCallVarargsData):
+ (JSC::DFG::Node::isTerminal):
+ (JSC::DFG::Node::hasHeapPrediction):
+ * dfg/DFGNodeType.h:
+ * dfg/DFGOSRExitCompilerCommon.cpp:
+ (JSC::DFG::handleExitCounts):
+ (JSC::DFG::reifyInlinedCallFrames):
+ (JSC::DFG::osrWriteBarrier):
+ * dfg/DFGOSRExitPreparation.cpp:
+ (JSC::DFG::prepareCodeOriginForOSRExit):
+ * dfg/DFGOperations.cpp:
+ * dfg/DFGPreciseLocalClobberize.h:
+ (JSC::DFG::PreciseLocalClobberizeAdaptor::readTop):
+ * dfg/DFGPredictionPropagationPhase.cpp:
+ (JSC::DFG::PredictionPropagationPhase::propagate):
+ * dfg/DFGSafeToExecute.h:
+ (JSC::DFG::safeToExecute):
+ * dfg/DFGSpeculativeJIT32_64.cpp:
+ (JSC::DFG::SpeculativeJIT::emitCall):
+ (JSC::DFG::SpeculativeJIT::compile):
+ * dfg/DFGSpeculativeJIT64.cpp:
+ (JSC::DFG::SpeculativeJIT::emitCall):
+ (JSC::DFG::SpeculativeJIT::compile):
+ * dfg/DFGValidate.cpp:
+ (JSC::DFG::Validate::validateSSA):
+ * dfg/DFGVarargsForwardingPhase.cpp:
+ * interpreter/CallFrame.cpp:
+ (JSC::CallFrame::bytecodeOffset):
+ * interpreter/StackVisitor.cpp:
+ (JSC::StackVisitor::gotoNextFrame):
+
</ins><span class="cx"> 2015-09-23 Filip Pizlo <fpizlo@apple.com>
</span><span class="cx">
</span><span class="cx"> Remove special case code for the no-parallel-GC case
</span></span></pre></div>
<a id="trunkSourceJavaScriptCorebytecodeCallModeh"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/bytecode/CallMode.h (190219 => 190220)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/bytecode/CallMode.h        2015-09-24 21:39:36 UTC (rev 190219)
+++ trunk/Source/JavaScriptCore/bytecode/CallMode.h        2015-09-24 21:42:59 UTC (rev 190220)
</span><span class="lines">@@ -26,12 +26,22 @@
</span><span class="cx"> #ifndef CallMode_h
</span><span class="cx"> #define CallMode_h
</span><span class="cx">
</span><ins>+#include "CodeSpecializationKind.h"
+
</ins><span class="cx"> namespace JSC {
</span><span class="cx">
</span><span class="cx"> enum class CallMode { Regular, Tail, Construct };
</span><span class="cx">
</span><span class="cx"> enum FrameAction { KeepTheFrame = 0, ReuseTheFrame };
</span><span class="cx">
</span><ins>+inline CodeSpecializationKind specializationKindFor(CallMode callMode)
+{
+ if (callMode == CallMode::Construct)
+ return CodeForConstruct;
+
+ return CodeForCall;
+}
+
</ins><span class="cx"> } // namespace JSC
</span><span class="cx">
</span><span class="cx"> namespace WTF {
</span></span></pre></div>
<a id="trunkSourceJavaScriptCorebytecodeCodeOrigincpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/bytecode/CodeOrigin.cpp (190219 => 190220)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/bytecode/CodeOrigin.cpp        2015-09-24 21:39:36 UTC (rev 190219)
+++ trunk/Source/JavaScriptCore/bytecode/CodeOrigin.cpp        2015-09-24 21:42:59 UTC (rev 190220)
</span><span class="lines">@@ -37,7 +37,7 @@
</span><span class="cx"> unsigned CodeOrigin::inlineDepthForCallFrame(InlineCallFrame* inlineCallFrame)
</span><span class="cx"> {
</span><span class="cx"> unsigned result = 1;
</span><del>- for (InlineCallFrame* current = inlineCallFrame; current; current = current->caller.inlineCallFrame)
</del><ins>+ for (InlineCallFrame* current = inlineCallFrame; current; current = current->directCaller.inlineCallFrame)
</ins><span class="cx"> result++;
</span><span class="cx"> return result;
</span><span class="cx"> }
</span><span class="lines">@@ -78,8 +78,8 @@
</span><span class="cx"> if (a.inlineCallFrame->executable.get() != b.inlineCallFrame->executable.get())
</span><span class="cx"> return false;
</span><span class="cx">
</span><del>- a = a.inlineCallFrame->caller;
- b = b.inlineCallFrame->caller;
</del><ins>+ a = a.inlineCallFrame->directCaller;
+ b = b.inlineCallFrame->directCaller;
</ins><span class="cx"> }
</span><span class="cx"> }
</span><span class="cx">
</span><span class="lines">@@ -100,7 +100,7 @@
</span><span class="cx">
</span><span class="cx"> result += WTF::PtrHash<JSCell*>::hash(codeOrigin.inlineCallFrame->executable.get());
</span><span class="cx">
</span><del>- codeOrigin = codeOrigin.inlineCallFrame->caller;
</del><ins>+ codeOrigin = codeOrigin.inlineCallFrame->directCaller;
</ins><span class="cx"> }
</span><span class="cx"> }
</span><span class="cx">
</span><span class="lines">@@ -109,8 +109,8 @@
</span><span class="cx"> Vector<CodeOrigin> result(inlineDepth());
</span><span class="cx"> result.last() = *this;
</span><span class="cx"> unsigned index = result.size() - 2;
</span><del>- for (InlineCallFrame* current = inlineCallFrame; current; current = current->caller.inlineCallFrame)
- result[index--] = current->caller;
</del><ins>+ for (InlineCallFrame* current = inlineCallFrame; current; current = current->directCaller.inlineCallFrame)
+ result[index--] = current->directCaller;
</ins><span class="cx"> RELEASE_ASSERT(!result[0].inlineCallFrame);
</span><span class="cx"> return result;
</span><span class="cx"> }
</span></span></pre></div>
<a id="trunkSourceJavaScriptCorebytecodeCodeOriginh"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/bytecode/CodeOrigin.h (190219 => 190220)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/bytecode/CodeOrigin.h        2015-09-24 21:39:36 UTC (rev 190219)
+++ trunk/Source/JavaScriptCore/bytecode/CodeOrigin.h        2015-09-24 21:42:59 UTC (rev 190220)
</span><span class="lines">@@ -26,6 +26,7 @@
</span><span class="cx"> #ifndef CodeOrigin_h
</span><span class="cx"> #define CodeOrigin_h
</span><span class="cx">
</span><ins>+#include "CallMode.h"
</ins><span class="cx"> #include "CodeBlockHash.h"
</span><span class="cx"> #include "CodeSpecializationKind.h"
</span><span class="cx"> #include "WriteBarrier.h"
</span></span></pre></div>
<a id="trunkSourceJavaScriptCorebytecodeInlineCallFramecpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/bytecode/InlineCallFrame.cpp (190219 => 190220)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/bytecode/InlineCallFrame.cpp        2015-09-24 21:39:36 UTC (rev 190219)
+++ trunk/Source/JavaScriptCore/bytecode/InlineCallFrame.cpp        2015-09-24 21:42:59 UTC (rev 190220)
</span><span class="lines">@@ -77,7 +77,7 @@
</span><span class="cx"> out.print(briefFunctionInformation(), ":<", RawPointer(executable.get()));
</span><span class="cx"> if (executable->isStrictMode())
</span><span class="cx"> out.print(" (StrictMode)");
</span><del>- out.print(", bc#", caller.bytecodeIndex, ", ", kind);
</del><ins>+ out.print(", bc#", directCaller.bytecodeIndex, ", ", static_cast<Kind>(kind));
</ins><span class="cx"> if (isClosureCall)
</span><span class="cx"> out.print(", closure call");
</span><span class="cx"> else
</span><span class="lines">@@ -105,12 +105,18 @@
</span><span class="cx"> case JSC::InlineCallFrame::Construct:
</span><span class="cx"> out.print("Construct");
</span><span class="cx"> return;
</span><ins>+ case JSC::InlineCallFrame::TailCall:
+ out.print("TailCall");
+ return;
</ins><span class="cx"> case JSC::InlineCallFrame::CallVarargs:
</span><span class="cx"> out.print("CallVarargs");
</span><span class="cx"> return;
</span><span class="cx"> case JSC::InlineCallFrame::ConstructVarargs:
</span><span class="cx"> out.print("ConstructVarargs");
</span><span class="cx"> return;
</span><ins>+ case JSC::InlineCallFrame::TailCallVarargs:
+ out.print("TailCallVarargs");
+ return;
</ins><span class="cx"> case JSC::InlineCallFrame::GetterCall:
</span><span class="cx"> out.print("GetterCall");
</span><span class="cx"> return;
</span></span></pre></div>
<a id="trunkSourceJavaScriptCorebytecodeInlineCallFrameh"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/bytecode/InlineCallFrame.h (190219 => 190220)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/bytecode/InlineCallFrame.h        2015-09-24 21:39:36 UTC (rev 190219)
+++ trunk/Source/JavaScriptCore/bytecode/InlineCallFrame.h        2015-09-24 21:42:59 UTC (rev 190220)
</span><span class="lines">@@ -49,37 +49,59 @@
</span><span class="cx"> enum Kind {
</span><span class="cx"> Call,
</span><span class="cx"> Construct,
</span><ins>+ TailCall,
</ins><span class="cx"> CallVarargs,
</span><span class="cx"> ConstructVarargs,
</span><ins>+ TailCallVarargs,
</ins><span class="cx">
</span><span class="cx"> // For these, the stackOffset incorporates the argument count plus the true return PC
</span><span class="cx"> // slot.
</span><span class="cx"> GetterCall,
</span><span class="cx"> SetterCall
</span><span class="cx"> };
</span><del>-
- static Kind kindFor(CodeSpecializationKind kind)
</del><ins>+
+ static CallMode callModeFor(Kind kind)
</ins><span class="cx"> {
</span><span class="cx"> switch (kind) {
</span><del>- case CodeForCall:
</del><ins>+ case Call:
+ case CallVarargs:
+ case GetterCall:
+ case SetterCall:
+ return CallMode::Regular;
+ case TailCall:
+ case TailCallVarargs:
+ return CallMode::Tail;
+ case Construct:
+ case ConstructVarargs:
+ return CallMode::Construct;
+ }
+ RELEASE_ASSERT_NOT_REACHED();
+ }
+
+ static Kind kindFor(CallMode callMode)
+ {
+ switch (callMode) {
+ case CallMode::Regular:
</ins><span class="cx"> return Call;
</span><del>- case CodeForConstruct:
</del><ins>+ case CallMode::Construct:
</ins><span class="cx"> return Construct;
</span><ins>+ case CallMode::Tail:
+ return TailCall;
</ins><span class="cx"> }
</span><span class="cx"> RELEASE_ASSERT_NOT_REACHED();
</span><del>- return Call;
</del><span class="cx"> }
</span><span class="cx">
</span><del>- static Kind varargsKindFor(CodeSpecializationKind kind)
</del><ins>+ static Kind varargsKindFor(CallMode callMode)
</ins><span class="cx"> {
</span><del>- switch (kind) {
- case CodeForCall:
</del><ins>+ switch (callMode) {
+ case CallMode::Regular:
</ins><span class="cx"> return CallVarargs;
</span><del>- case CodeForConstruct:
</del><ins>+ case CallMode::Construct:
</ins><span class="cx"> return ConstructVarargs;
</span><ins>+ case CallMode::Tail:
+ return TailCallVarargs;
</ins><span class="cx"> }
</span><span class="cx"> RELEASE_ASSERT_NOT_REACHED();
</span><del>- return Call;
</del><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> static CodeSpecializationKind specializationKindFor(Kind kind)
</span><span class="lines">@@ -87,6 +109,8 @@
</span><span class="cx"> switch (kind) {
</span><span class="cx"> case Call:
</span><span class="cx"> case CallVarargs:
</span><ins>+ case TailCall:
+ case TailCallVarargs:
</ins><span class="cx"> case GetterCall:
</span><span class="cx"> case SetterCall:
</span><span class="cx"> return CodeForCall;
</span><span class="lines">@@ -95,24 +119,64 @@
</span><span class="cx"> return CodeForConstruct;
</span><span class="cx"> }
</span><span class="cx"> RELEASE_ASSERT_NOT_REACHED();
</span><del>- return CodeForCall;
</del><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> static bool isVarargs(Kind kind)
</span><span class="cx"> {
</span><span class="cx"> switch (kind) {
</span><span class="cx"> case CallVarargs:
</span><ins>+ case TailCallVarargs:
</ins><span class="cx"> case ConstructVarargs:
</span><span class="cx"> return true;
</span><span class="cx"> default:
</span><span class="cx"> return false;
</span><span class="cx"> }
</span><span class="cx"> }
</span><ins>+
+ static bool isTail(Kind kind)
+ {
+ switch (kind) {
+ case TailCall:
+ case TailCallVarargs:
+ return true;
+ default:
+ return false;
+ }
+ }
+ bool isTail() const
+ {
+ return isTail(static_cast<Kind>(kind));
+ }
+
+ static CodeOrigin* computeCallerSkippingDeadFrames(InlineCallFrame* inlineCallFrame)
+ {
+ CodeOrigin* codeOrigin;
+ bool tailCallee;
+ do {
+ tailCallee = inlineCallFrame->isTail();
+ codeOrigin = &inlineCallFrame->directCaller;
+ inlineCallFrame = codeOrigin->inlineCallFrame;
+ } while (inlineCallFrame && tailCallee);
+ if (tailCallee)
+ return nullptr;
+ return codeOrigin;
+ }
+
+ CodeOrigin* getCallerSkippingDeadFrames()
+ {
+ return computeCallerSkippingDeadFrames(this);
+ }
+
+ InlineCallFrame* getCallerInlineFrameSkippingDeadFrames()
+ {
+ CodeOrigin* caller = getCallerSkippingDeadFrames();
+ return caller ? caller->inlineCallFrame : nullptr;
+ }
</ins><span class="cx">
</span><span class="cx"> Vector<ValueRecovery> arguments; // Includes 'this'.
</span><span class="cx"> WriteBarrier<ScriptExecutable> executable;
</span><span class="cx"> ValueRecovery calleeRecovery;
</span><del>- CodeOrigin caller;
</del><ins>+ CodeOrigin directCaller;
</ins><span class="cx">
</span><span class="cx"> signed stackOffset : 28;
</span><span class="cx"> unsigned kind : 3; // real type is Kind
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoredfgDFGAbstractInterpreterInlinesh"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/dfg/DFGAbstractInterpreterInlines.h (190219 => 190220)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/dfg/DFGAbstractInterpreterInlines.h        2015-09-24 21:39:36 UTC (rev 190219)
+++ trunk/Source/JavaScriptCore/dfg/DFGAbstractInterpreterInlines.h        2015-09-24 21:42:59 UTC (rev 190220)
</span><span class="lines">@@ -1552,6 +1552,13 @@
</span><span class="cx"> case Return:
</span><span class="cx"> m_state.setIsValid(false);
</span><span class="cx"> break;
</span><ins>+
+ case TailCall:
+ case TailCallVarargs:
+ case TailCallForwardVarargs:
+ clobberWorld(node->origin.semantic, clobberLimit);
+ m_state.setIsValid(false);
+ break;
</ins><span class="cx">
</span><span class="cx"> case Throw:
</span><span class="cx"> case ThrowReferenceError:
</span><span class="lines">@@ -2442,11 +2449,14 @@
</span><span class="cx"> break;
</span><span class="cx">
</span><span class="cx"> case Call:
</span><ins>+ case TailCallInlinedCaller:
</ins><span class="cx"> case Construct:
</span><span class="cx"> case CallVarargs:
</span><span class="cx"> case CallForwardVarargs:
</span><ins>+ case TailCallVarargsInlinedCaller:
</ins><span class="cx"> case ConstructVarargs:
</span><span class="cx"> case ConstructForwardVarargs:
</span><ins>+ case TailCallForwardVarargsInlinedCaller:
</ins><span class="cx"> clobberWorld(node->origin.semantic, clobberLimit);
</span><span class="cx"> forNode(node).makeHeapTop();
</span><span class="cx"> break;
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoredfgDFGArgumentsEliminationPhasecpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/dfg/DFGArgumentsEliminationPhase.cpp (190219 => 190220)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/dfg/DFGArgumentsEliminationPhase.cpp        2015-09-24 21:39:36 UTC (rev 190219)
+++ trunk/Source/JavaScriptCore/dfg/DFGArgumentsEliminationPhase.cpp        2015-09-24 21:42:59 UTC (rev 190220)
</span><span class="lines">@@ -167,6 +167,8 @@
</span><span class="cx">
</span><span class="cx"> case CallVarargs:
</span><span class="cx"> case ConstructVarargs:
</span><ins>+ case TailCallVarargs:
+ case TailCallVarargsInlinedCaller:
</ins><span class="cx"> escape(node->child1());
</span><span class="cx"> escape(node->child3());
</span><span class="cx"> break;
</span><span class="lines">@@ -560,7 +562,9 @@
</span><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> case CallVarargs:
</span><del>- case ConstructVarargs: {
</del><ins>+ case ConstructVarargs:
+ case TailCallVarargs:
+ case TailCallVarargsInlinedCaller: {
</ins><span class="cx"> Node* candidate = node->child2().node();
</span><span class="cx"> if (!m_candidates.contains(candidate))
</span><span class="cx"> break;
</span><span class="lines">@@ -585,16 +589,44 @@
</span><span class="cx"> m_graph.m_varArgChildren.append(node->child3());
</span><span class="cx"> for (Node* argument : arguments)
</span><span class="cx"> m_graph.m_varArgChildren.append(Edge(argument));
</span><del>- node->setOpAndDefaultFlags(
- node->op() == CallVarargs ? Call : Construct);
</del><ins>+ switch (node->op()) {
+ case CallVarargs:
+ node->setOpAndDefaultFlags(Call);
+ break;
+ case ConstructVarargs:
+ node->setOpAndDefaultFlags(Construct);
+ break;
+ case TailCallVarargs:
+ node->setOpAndDefaultFlags(TailCall);
+ break;
+ case TailCallVarargsInlinedCaller:
+ node->setOpAndDefaultFlags(TailCallInlinedCaller);
+ break;
+ default:
+ RELEASE_ASSERT_NOT_REACHED();
+ }
</ins><span class="cx"> node->children = AdjacencyList(
</span><span class="cx"> AdjacencyList::Variable,
</span><span class="cx"> firstChild, m_graph.m_varArgChildren.size() - firstChild);
</span><span class="cx"> break;
</span><span class="cx"> }
</span><span class="cx">
</span><del>- node->setOpAndDefaultFlags(
- node->op() == CallVarargs ? CallForwardVarargs : ConstructForwardVarargs);
</del><ins>+ switch (node->op()) {
+ case CallVarargs:
+ node->setOpAndDefaultFlags(CallForwardVarargs);
+ break;
+ case ConstructVarargs:
+ node->setOpAndDefaultFlags(ConstructForwardVarargs);
+ break;
+ case TailCallVarargs:
+ node->setOpAndDefaultFlags(TailCallForwardVarargs);
+ break;
+ case TailCallVarargsInlinedCaller:
+ node->setOpAndDefaultFlags(TailCallForwardVarargsInlinedCaller);
+ break;
+ default:
+ RELEASE_ASSERT_NOT_REACHED();
+ }
</ins><span class="cx"> break;
</span><span class="cx"> }
</span><span class="cx">
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoredfgDFGBasicBlockh"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/dfg/DFGBasicBlock.h (190219 => 190220)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/dfg/DFGBasicBlock.h        2015-09-24 21:39:36 UTC (rev 190219)
+++ trunk/Source/JavaScriptCore/dfg/DFGBasicBlock.h        2015-09-24 21:42:59 UTC (rev 190220)
</span><span class="lines">@@ -92,6 +92,9 @@
</span><span class="cx"> case Branch:
</span><span class="cx"> case Switch:
</span><span class="cx"> case Return:
</span><ins>+ case TailCall:
+ case TailCallVarargs:
+ case TailCallForwardVarargs:
</ins><span class="cx"> case Unreachable:
</span><span class="cx"> return NodeAndIndex(node, i);
</span><span class="cx"> // The bitter end can contain Phantoms and the like. There will probably only be one or two nodes after the terminal. They are all no-ops and will not have any checked children.
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoredfgDFGByteCodeParsercpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/dfg/DFGByteCodeParser.cpp (190219 => 190220)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/dfg/DFGByteCodeParser.cpp        2015-09-24 21:39:36 UTC (rev 190219)
+++ trunk/Source/JavaScriptCore/dfg/DFGByteCodeParser.cpp        2015-09-24 21:42:59 UTC (rev 190220)
</span><span class="lines">@@ -183,14 +183,14 @@
</span><span class="cx"> Node* callTarget, int argCount, int registerOffset, CallLinkStatus,
</span><span class="cx"> SpeculatedType prediction);
</span><span class="cx"> void handleCall(
</span><del>- int result, NodeType op, InlineCallFrame::Kind, unsigned instructionSize,
</del><ins>+ int result, NodeType op, CallMode, unsigned instructionSize,
</ins><span class="cx"> Node* callTarget, int argCount, int registerOffset, CallLinkStatus);
</span><del>- void handleCall(int result, NodeType op, CodeSpecializationKind, unsigned instructionSize, int callee, int argCount, int registerOffset);
- void handleCall(Instruction* pc, NodeType op, CodeSpecializationKind);
- void handleVarargsCall(Instruction* pc, NodeType op, CodeSpecializationKind);
</del><ins>+ void handleCall(int result, NodeType op, CallMode, unsigned instructionSize, int callee, int argCount, int registerOffset);
+ void handleCall(Instruction* pc, NodeType op, CallMode);
+ void handleVarargsCall(Instruction* pc, NodeType op, CallMode);
</ins><span class="cx"> void emitFunctionChecks(CallVariant, Node* callTarget, VirtualRegister thisArgumnt);
</span><span class="cx"> void emitArgumentPhantoms(int registerOffset, int argumentCountIncludingThis);
</span><del>- unsigned inliningCost(CallVariant, int argumentCountIncludingThis, CodeSpecializationKind); // Return UINT_MAX if it's not an inlining candidate. By convention, intrinsics have a cost of 1.
</del><ins>+ unsigned inliningCost(CallVariant, int argumentCountIncludingThis, CallMode); // Return UINT_MAX if it's not an inlining candidate. By convention, intrinsics have a cost of 1.
</ins><span class="cx"> // Handle inlining. Return true if it succeeded, false if we need to plant a call.
</span><span class="cx"> bool handleInlining(Node* callTargetNode, int resultOperand, const CallLinkStatus&, int registerOffset, VirtualRegister thisArgument, VirtualRegister argumentsArgument, unsigned argumentsOffset, int argumentCountIncludingThis, unsigned nextOffset, NodeType callOp, InlineCallFrame::Kind, SpeculatedType prediction);
</span><span class="cx"> enum CallerLinkability { CallerDoesNormalLinking, CallerLinksManually };
</span><span class="lines">@@ -648,6 +648,11 @@
</span><span class="cx"> return m_inlineStackTop->m_inlineCallFrame;
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+ bool allInlineFramesAreTailCalls()
+ {
+ return !inlineCallFrame() || !inlineCallFrame()->getCallerSkippingDeadFrames();
+ }
+
</ins><span class="cx"> CodeOrigin currentCodeOrigin()
</span><span class="cx"> {
</span><span class="cx"> return CodeOrigin(m_currentIndex, inlineCallFrame());
</span><span class="lines">@@ -736,7 +741,7 @@
</span><span class="cx">
</span><span class="cx"> Node* addCallWithoutSettingResult(
</span><span class="cx"> NodeType op, OpInfo opInfo, Node* callee, int argCount, int registerOffset,
</span><del>- SpeculatedType prediction)
</del><ins>+ OpInfo prediction)
</ins><span class="cx"> {
</span><span class="cx"> addVarArgChild(callee);
</span><span class="cx"> size_t frameSize = JSStack::CallFrameHeaderSize + argCount;
</span><span class="lines">@@ -749,15 +754,22 @@
</span><span class="cx"> for (int i = 0; i < argCount; ++i)
</span><span class="cx"> addVarArgChild(get(virtualRegisterForArgument(i, registerOffset)));
</span><span class="cx">
</span><del>- return addToGraph(Node::VarArg, op, opInfo, OpInfo(prediction));
</del><ins>+ return addToGraph(Node::VarArg, op, opInfo, prediction);
</ins><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> Node* addCall(
</span><span class="cx"> int result, NodeType op, OpInfo opInfo, Node* callee, int argCount, int registerOffset,
</span><span class="cx"> SpeculatedType prediction)
</span><span class="cx"> {
</span><ins>+ if (op == TailCall) {
+ if (allInlineFramesAreTailCalls())
+ return addCallWithoutSettingResult(op, OpInfo(), callee, argCount, registerOffset, OpInfo());
+ op = TailCallInlinedCaller;
+ }
+
+
</ins><span class="cx"> Node* call = addCallWithoutSettingResult(
</span><del>- op, opInfo, callee, argCount, registerOffset, prediction);
</del><ins>+ op, opInfo, callee, argCount, registerOffset, OpInfo(prediction));
</ins><span class="cx"> VirtualRegister resultReg(result);
</span><span class="cx"> if (resultReg.isValid())
</span><span class="cx"> set(resultReg, call);
</span><span class="lines">@@ -776,14 +788,62 @@
</span><span class="cx">
</span><span class="cx"> SpeculatedType getPredictionWithoutOSRExit(unsigned bytecodeIndex)
</span><span class="cx"> {
</span><del>- ConcurrentJITLocker locker(m_inlineStackTop->m_profiledBlock->m_lock);
- return m_inlineStackTop->m_profiledBlock->valueProfilePredictionForBytecodeOffset(locker, bytecodeIndex);
</del><ins>+ SpeculatedType prediction;
+ CodeBlock* profiledBlock = nullptr;
+
+ {
+ ConcurrentJITLocker locker(m_inlineStackTop->m_profiledBlock->m_lock);
+ prediction = m_inlineStackTop->m_profiledBlock->valueProfilePredictionForBytecodeOffset(locker, bytecodeIndex);
+
+ if (prediction == SpecNone) {
+ // If we have no information about the values this
+ // node generates, we check if by any chance it is
+ // a tail call opcode. In that case, we walk up the
+ // inline frames to find a call higher in the call
+ // chain and use its prediction. If we only have
+ // inlined tail call frames, we use SpecFullTop
+ // to avoid a spurious OSR exit.
+ Instruction* instruction = m_inlineStackTop->m_profiledBlock->instructions().begin() + bytecodeIndex;
+ OpcodeID opcodeID = m_vm->interpreter->getOpcodeID(instruction->u.opcode);
+
+ switch (opcodeID) {
+ case op_tail_call:
+ case op_tail_call_varargs: {
+ if (!inlineCallFrame()) {
+ prediction = SpecFullTop;
+ break;
+ }
+ CodeOrigin* codeOrigin = inlineCallFrame()->getCallerSkippingDeadFrames();
+ if (!codeOrigin) {
+ prediction = SpecFullTop;
+ break;
+ }
+ InlineStackEntry* stack = m_inlineStackTop;
+ while (stack->m_inlineCallFrame != codeOrigin->inlineCallFrame)
+ stack = stack->m_caller;
+ bytecodeIndex = codeOrigin->bytecodeIndex;
+ profiledBlock = stack->m_profiledBlock;
+ break;
+ }
+
+ default:
+ break;
+ }
+ }
+ }
+
+ if (profiledBlock) {
+ ConcurrentJITLocker locker(profiledBlock->m_lock);
+ prediction = profiledBlock->valueProfilePredictionForBytecodeOffset(locker, bytecodeIndex);
+ }
+
+ return prediction;
</ins><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> SpeculatedType getPrediction(unsigned bytecodeIndex)
</span><span class="cx"> {
</span><span class="cx"> SpeculatedType prediction = getPredictionWithoutOSRExit(bytecodeIndex);
</span><del>-
</del><ins>+
</ins><span class="cx"> if (prediction == SpecNone) {
</span><span class="cx"> // We have no information about what values this node generates. Give up
</span><span class="cx"> // on executing this code, since we're likely to do more damage than good.
</span><span class="lines">@@ -1069,16 +1129,17 @@
</span><span class="cx"> m_exitOK = false; \
</span><span class="cx"> return shouldContinueParsing
</span><span class="cx">
</span><del>-void ByteCodeParser::handleCall(Instruction* pc, NodeType op, CodeSpecializationKind kind)
</del><ins>+void ByteCodeParser::handleCall(Instruction* pc, NodeType op, CallMode callMode)
</ins><span class="cx"> {
</span><span class="cx"> ASSERT(OPCODE_LENGTH(op_call) == OPCODE_LENGTH(op_construct));
</span><ins>+ ASSERT(OPCODE_LENGTH(op_call) == OPCODE_LENGTH(op_tail_call));
</ins><span class="cx"> handleCall(
</span><del>- pc[1].u.operand, op, kind, OPCODE_LENGTH(op_call),
</del><ins>+ pc[1].u.operand, op, callMode, OPCODE_LENGTH(op_call),
</ins><span class="cx"> pc[2].u.operand, pc[3].u.operand, -pc[4].u.operand);
</span><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> void ByteCodeParser::handleCall(
</span><del>- int result, NodeType op, CodeSpecializationKind kind, unsigned instructionSize,
</del><ins>+ int result, NodeType op, CallMode callMode, unsigned instructionSize,
</ins><span class="cx"> int callee, int argumentCountIncludingThis, int registerOffset)
</span><span class="cx"> {
</span><span class="cx"> Node* callTarget = get(VirtualRegister(callee));
</span><span class="lines">@@ -1088,17 +1149,17 @@
</span><span class="cx"> m_inlineStackTop->m_callLinkInfos, m_callContextMap);
</span><span class="cx">
</span><span class="cx"> handleCall(
</span><del>- result, op, InlineCallFrame::kindFor(kind), instructionSize, callTarget,
</del><ins>+ result, op, callMode, instructionSize, callTarget,
</ins><span class="cx"> argumentCountIncludingThis, registerOffset, callLinkStatus);
</span><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> void ByteCodeParser::handleCall(
</span><del>- int result, NodeType op, InlineCallFrame::Kind kind, unsigned instructionSize,
</del><ins>+ int result, NodeType op, CallMode callMode, unsigned instructionSize,
</ins><span class="cx"> Node* callTarget, int argumentCountIncludingThis, int registerOffset,
</span><span class="cx"> CallLinkStatus callLinkStatus)
</span><span class="cx"> {
</span><span class="cx"> handleCall(
</span><del>- result, op, kind, instructionSize, callTarget, argumentCountIncludingThis,
</del><ins>+ result, op, InlineCallFrame::kindFor(callMode), instructionSize, callTarget, argumentCountIncludingThis,
</ins><span class="cx"> registerOffset, callLinkStatus, getPrediction());
</span><span class="cx"> }
</span><span class="cx">
</span><span class="lines">@@ -1118,7 +1179,7 @@
</span><span class="cx"> if (!callLinkStatus.canOptimize()) {
</span><span class="cx"> // Oddly, this conflates calls that haven't executed with calls that behaved sufficiently polymorphically
</span><span class="cx"> // that we cannot optimize them.
</span><del>-
</del><ins>+
</ins><span class="cx"> addCall(result, op, OpInfo(), callTarget, argumentCountIncludingThis, registerOffset, prediction);
</span><span class="cx"> return;
</span><span class="cx"> }
</span><span class="lines">@@ -1136,9 +1197,10 @@
</span><span class="cx"> addCall(result, op, callOpInfo, callTarget, argumentCountIncludingThis, registerOffset, prediction);
</span><span class="cx"> }
</span><span class="cx">
</span><del>-void ByteCodeParser::handleVarargsCall(Instruction* pc, NodeType op, CodeSpecializationKind kind)
</del><ins>+void ByteCodeParser::handleVarargsCall(Instruction* pc, NodeType op, CallMode callMode)
</ins><span class="cx"> {
</span><span class="cx"> ASSERT(OPCODE_LENGTH(op_call_varargs) == OPCODE_LENGTH(op_construct_varargs));
</span><ins>+ ASSERT(OPCODE_LENGTH(op_call_varargs) == OPCODE_LENGTH(op_tail_call_varargs));
</ins><span class="cx">
</span><span class="cx"> int result = pc[1].u.operand;
</span><span class="cx"> int callee = pc[2].u.operand;
</span><span class="lines">@@ -1161,7 +1223,7 @@
</span><span class="cx"> dataLog(" Varargs call link status at ", currentCodeOrigin(), ": ", callLinkStatus, "\n");
</span><span class="cx">
</span><span class="cx"> if (callLinkStatus.canOptimize()
</span><del>- && handleInlining(callTarget, result, callLinkStatus, firstFreeReg, VirtualRegister(thisReg), VirtualRegister(arguments), firstVarArgOffset, 0, m_currentIndex + OPCODE_LENGTH(op_call_varargs), op, InlineCallFrame::varargsKindFor(kind), prediction)) {
</del><ins>+ && handleInlining(callTarget, result, callLinkStatus, firstFreeReg, VirtualRegister(thisReg), VirtualRegister(arguments), firstVarArgOffset, 0, m_currentIndex + OPCODE_LENGTH(op_call_varargs), op, InlineCallFrame::varargsKindFor(callMode), prediction)) {
</ins><span class="cx"> if (m_graph.compilation())
</span><span class="cx"> m_graph.compilation()->noticeInlinedCall();
</span><span class="cx"> return;
</span><span class="lines">@@ -1171,7 +1233,15 @@
</span><span class="cx"> data->firstVarArgOffset = firstVarArgOffset;
</span><span class="cx">
</span><span class="cx"> Node* thisChild = get(VirtualRegister(thisReg));
</span><del>-
</del><ins>+
+ if (op == TailCallVarargs) {
+ if (allInlineFramesAreTailCalls()) {
+ addToGraph(op, OpInfo(data), OpInfo(), callTarget, get(VirtualRegister(arguments)), thisChild);
+ return;
+ }
+ op = TailCallVarargsInlinedCaller;
+ }
+
</ins><span class="cx"> Node* call = addToGraph(op, OpInfo(data), OpInfo(prediction), callTarget, get(VirtualRegister(arguments)), thisChild);
</span><span class="cx"> VirtualRegister resultReg(result);
</span><span class="cx"> if (resultReg.isValid())
</span><span class="lines">@@ -1206,8 +1276,9 @@
</span><span class="cx"> addToGraph(Phantom, get(virtualRegisterForArgument(i, registerOffset)));
</span><span class="cx"> }
</span><span class="cx">
</span><del>-unsigned ByteCodeParser::inliningCost(CallVariant callee, int argumentCountIncludingThis, CodeSpecializationKind kind)
</del><ins>+unsigned ByteCodeParser::inliningCost(CallVariant callee, int argumentCountIncludingThis, CallMode callMode)
</ins><span class="cx"> {
</span><ins>+ CodeSpecializationKind kind = specializationKindFor(callMode);
</ins><span class="cx"> if (verbose)
</span><span class="cx"> dataLog("Considering inlining ", callee, " into ", currentCodeOrigin(), "\n");
</span><span class="cx">
</span><span class="lines">@@ -1249,7 +1320,7 @@
</span><span class="cx"> CapabilityLevel capabilityLevel = inlineFunctionForCapabilityLevel(
</span><span class="cx"> codeBlock, kind, callee.isClosureCall());
</span><span class="cx"> if (verbose) {
</span><del>- dataLog(" Kind: ", kind, "\n");
</del><ins>+ dataLog(" Call mode: ", callMode, "\n");
</ins><span class="cx"> dataLog(" Is closure call: ", callee.isClosureCall(), "\n");
</span><span class="cx"> dataLog(" Capability level: ", capabilityLevel, "\n");
</span><span class="cx"> dataLog(" Might inline function: ", mightInlineFunctionFor(codeBlock, kind), "\n");
</span><span class="lines">@@ -1320,7 +1391,7 @@
</span><span class="cx"> {
</span><span class="cx"> CodeSpecializationKind specializationKind = InlineCallFrame::specializationKindFor(kind);
</span><span class="cx">
</span><del>- ASSERT(inliningCost(callee, argumentCountIncludingThis, specializationKind) != UINT_MAX);
</del><ins>+ ASSERT(inliningCost(callee, argumentCountIncludingThis, InlineCallFrame::callModeFor(kind)) != UINT_MAX);
</ins><span class="cx">
</span><span class="cx"> CodeBlock* codeBlock = callee.functionExecutable()->baselineCodeBlockFor(specializationKind);
</span><span class="cx"> insertChecks(codeBlock);
</span><span class="lines">@@ -1424,7 +1495,7 @@
</span><span class="cx"> m_currentBlock = m_graph.lastBlock();
</span><span class="cx"> return;
</span><span class="cx"> }
</span><del>-
</del><ins>+
</ins><span class="cx"> if (Options::verboseDFGByteCodeParsing())
</span><span class="cx"> dataLog(" Creating new block after inlining.\n");
</span><span class="cx">
</span><span class="lines">@@ -1531,7 +1602,7 @@
</span><span class="cx"> }
</span><span class="cx"> }
</span><span class="cx">
</span><del>- unsigned myInliningCost = inliningCost(callee, argumentCountIncludingThis, specializationKind);
</del><ins>+ unsigned myInliningCost = inliningCost(callee, argumentCountIncludingThis, InlineCallFrame::callModeFor(kind));
</ins><span class="cx"> if (myInliningCost > inliningBalance)
</span><span class="cx"> return false;
</span><span class="cx">
</span><span class="lines">@@ -3003,7 +3074,11 @@
</span><span class="cx"> // logic relies on every bytecode resulting in one or more nodes, which would
</span><span class="cx"> // be true anyway except for op_loop_hint, which emits a Phantom to force this
</span><span class="cx"> // to be true.
</span><del>- if (!m_currentBlock->isEmpty())
</del><ins>+ // We also don't insert a jump if the block already has a terminal,
+ // which could happen after a tail call.
+ ASSERT(m_currentBlock->isEmpty() || !m_currentBlock->terminal()
+ || m_currentBlock->terminal()->op() == TailCall || m_currentBlock->terminal()->op() == TailCallVarargs);
+ if (!m_currentBlock->isEmpty() && !m_currentBlock->terminal())
</ins><span class="cx"> addToGraph(Jump, OpInfo(m_currentIndex));
</span><span class="cx"> return shouldContinueParsing;
</span><span class="cx"> }
</span><span class="lines">@@ -3585,6 +3660,12 @@
</span><span class="cx"> // === Block terminators. ===
</span><span class="cx">
</span><span class="cx"> case op_jmp: {
</span><ins>+ if (m_currentBlock->terminal()) {
+ // We could be the dummy jump to a return after a non-inlined, non-emulated tail call in a ternary operator
+ Node* terminal = m_currentBlock->terminal();
+ ASSERT_UNUSED(terminal, terminal->op() == TailCall || terminal->op() == TailCallVarargs);
+ LAST_OPCODE(op_ret);
+ }
</ins><span class="cx"> int relativeOffset = currentInstruction[1].u.operand;
</span><span class="cx"> addToGraph(Jump, OpInfo(m_currentIndex + relativeOffset));
</span><span class="cx"> if (relativeOffset <= 0)
</span><span class="lines">@@ -3756,6 +3837,12 @@
</span><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> case op_ret:
</span><ins>+ if (m_currentBlock->terminal()) {
+ // We could be the dummy return after a non-inlined, non-emulated tail call
+ Node* terminal = m_currentBlock->terminal();
+ ASSERT_UNUSED(terminal, terminal->op() == TailCall || terminal->op() == TailCallVarargs);
+ LAST_OPCODE(op_ret);
+ }
</ins><span class="cx"> if (inlineCallFrame()) {
</span><span class="cx"> flushForReturn();
</span><span class="cx"> if (m_inlineStackTop->m_returnValue.isValid())
</span><span class="lines">@@ -3807,22 +3894,37 @@
</span><span class="cx"> NEXT_OPCODE(op_catch);
</span><span class="cx">
</span><span class="cx"> case op_call:
</span><del>- handleCall(currentInstruction, Call, CodeForCall);
</del><ins>+ handleCall(currentInstruction, Call, CallMode::Regular);
</ins><span class="cx"> // Verify that handleCall(), which could have inlined the callee, didn't trash m_currentInstruction
</span><span class="cx"> ASSERT(m_currentInstruction == currentInstruction);
</span><span class="cx"> NEXT_OPCODE(op_call);
</span><del>-
</del><ins>+
+ case op_tail_call:
+ flushForReturn();
+ handleCall(currentInstruction, TailCall, CallMode::Tail);
+ // Verify that handleCall(), which could have inlined the callee, didn't trash m_currentInstruction
+ ASSERT(m_currentInstruction == currentInstruction);
+ // We let the following op_ret handle cases related to
+ // inlining to keep things simple.
+ NEXT_OPCODE(op_tail_call);
+
</ins><span class="cx"> case op_construct:
</span><del>- handleCall(currentInstruction, Construct, CodeForConstruct);
</del><ins>+ handleCall(currentInstruction, Construct, CallMode::Construct);
</ins><span class="cx"> NEXT_OPCODE(op_construct);
</span><span class="cx">
</span><span class="cx"> case op_call_varargs: {
</span><del>- handleVarargsCall(currentInstruction, CallVarargs, CodeForCall);
</del><ins>+ handleVarargsCall(currentInstruction, CallVarargs, CallMode::Regular);
</ins><span class="cx"> NEXT_OPCODE(op_call_varargs);
</span><span class="cx"> }
</span><ins>+
+ case op_tail_call_varargs: {
+ flushForReturn();
+ handleVarargsCall(currentInstruction, TailCallVarargs, CallMode::Tail);
+ NEXT_OPCODE(op_tail_call_varargs);
+ }
</ins><span class="cx">
</span><span class="cx"> case op_construct_varargs: {
</span><del>- handleVarargsCall(currentInstruction, ConstructVarargs, CodeForConstruct);
</del><ins>+ handleVarargsCall(currentInstruction, ConstructVarargs, CallMode::Construct);
</ins><span class="cx"> NEXT_OPCODE(op_construct_varargs);
</span><span class="cx"> }
</span><span class="cx">
</span><span class="lines">@@ -4533,7 +4635,7 @@
</span><span class="cx"> m_inlineCallFrame->isClosureCall = false;
</span><span class="cx"> } else
</span><span class="cx"> m_inlineCallFrame->isClosureCall = true;
</span><del>- m_inlineCallFrame->caller = byteCodeParser->currentCodeOrigin();
</del><ins>+ m_inlineCallFrame->directCaller = byteCodeParser->currentCodeOrigin();
</ins><span class="cx"> m_inlineCallFrame->arguments.resizeToFit(argumentCountIncludingThis); // Set the number of arguments including this, but don't configure the value recoveries, yet.
</span><span class="cx"> m_inlineCallFrame->kind = kind;
</span><span class="cx">
</span><span class="lines">@@ -4604,7 +4706,7 @@
</span><span class="cx"> if (UNLIKELY(Options::dumpSourceAtDFGTime())) {
</span><span class="cx"> Vector<DeferredSourceDump>& deferredSourceDump = m_graph.m_plan.callback->ensureDeferredSourceDump();
</span><span class="cx"> if (inlineCallFrame()) {
</span><del>- DeferredSourceDump dump(codeBlock->baselineVersion(), m_codeBlock, JITCode::DFGJIT, inlineCallFrame()->caller);
</del><ins>+ DeferredSourceDump dump(codeBlock->baselineVersion(), m_codeBlock, JITCode::DFGJIT, inlineCallFrame()->directCaller);
</ins><span class="cx"> deferredSourceDump.append(dump);
</span><span class="cx"> } else
</span><span class="cx"> deferredSourceDump.append(DeferredSourceDump(codeBlock->baselineVersion()));
</span><span class="lines">@@ -4615,7 +4717,7 @@
</span><span class="cx"> if (inlineCallFrame()) {
</span><span class="cx"> dataLog(
</span><span class="cx"> " for inlining at ", CodeBlockWithJITType(m_codeBlock, JITCode::DFGJIT),
</span><del>- " ", inlineCallFrame()->caller);
</del><ins>+ " ", inlineCallFrame()->directCaller);
</ins><span class="cx"> }
</span><span class="cx"> dataLog(
</span><span class="cx"> ": needsActivation = ", codeBlock->needsActivation(),
</span><span class="lines">@@ -4699,7 +4801,7 @@
</span><span class="cx"> return;
</span><span class="cx"> }
</span><span class="cx">
</span><del>- m_currentBlock = 0;
</del><ins>+ m_currentBlock = nullptr;
</ins><span class="cx"> } while (m_currentIndex < limit);
</span><span class="cx"> }
</span><span class="cx">
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoredfgDFGCapabilitiescpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/dfg/DFGCapabilities.cpp (190219 => 190220)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/dfg/DFGCapabilities.cpp        2015-09-24 21:39:36 UTC (rev 190219)
+++ trunk/Source/JavaScriptCore/dfg/DFGCapabilities.cpp        2015-09-24 21:42:59 UTC (rev 190220)
</span><span class="lines">@@ -179,8 +179,10 @@
</span><span class="cx"> case op_throw:
</span><span class="cx"> case op_throw_static_error:
</span><span class="cx"> case op_call:
</span><ins>+ case op_tail_call:
</ins><span class="cx"> case op_construct:
</span><span class="cx"> case op_call_varargs:
</span><ins>+ case op_tail_call_varargs:
</ins><span class="cx"> case op_construct_varargs:
</span><span class="cx"> case op_create_direct_arguments:
</span><span class="cx"> case op_create_scoped_arguments:
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoredfgDFGClobberizeh"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/dfg/DFGClobberize.h (190219 => 190220)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/dfg/DFGClobberize.h        2015-09-24 21:39:36 UTC (rev 190219)
+++ trunk/Source/JavaScriptCore/dfg/DFGClobberize.h        2015-09-24 21:42:59 UTC (rev 190220)
</span><span class="lines">@@ -382,9 +382,12 @@
</span><span class="cx"> case ArrayPush:
</span><span class="cx"> case ArrayPop:
</span><span class="cx"> case Call:
</span><ins>+ case TailCallInlinedCaller:
</ins><span class="cx"> case Construct:
</span><span class="cx"> case CallVarargs:
</span><span class="cx"> case CallForwardVarargs:
</span><ins>+ case TailCallVarargsInlinedCaller:
+ case TailCallForwardVarargsInlinedCaller:
</ins><span class="cx"> case ConstructVarargs:
</span><span class="cx"> case ConstructForwardVarargs:
</span><span class="cx"> case ToPrimitive:
</span><span class="lines">@@ -393,6 +396,13 @@
</span><span class="cx"> read(World);
</span><span class="cx"> write(Heap);
</span><span class="cx"> return;
</span><ins>+
+ case TailCall:
+ case TailCallVarargs:
+ case TailCallForwardVarargs:
+ read(World);
+ write(SideState);
+ return;
</ins><span class="cx">
</span><span class="cx"> case GetGetter:
</span><span class="cx"> read(GetterSetter_getter);
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoredfgDFGDoesGCcpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/dfg/DFGDoesGC.cpp (190219 => 190220)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/dfg/DFGDoesGC.cpp        2015-09-24 21:39:36 UTC (rev 190219)
+++ trunk/Source/JavaScriptCore/dfg/DFGDoesGC.cpp        2015-09-24 21:42:59 UTC (rev 190220)
</span><span class="lines">@@ -120,12 +120,16 @@
</span><span class="cx"> case CompareEq:
</span><span class="cx"> case CompareStrictEq:
</span><span class="cx"> case Call:
</span><ins>+ case TailCallInlinedCaller:
</ins><span class="cx"> case Construct:
</span><span class="cx"> case CallVarargs:
</span><ins>+ case TailCallVarargsInlinedCaller:
</ins><span class="cx"> case ConstructVarargs:
</span><span class="cx"> case LoadVarargs:
</span><span class="cx"> case CallForwardVarargs:
</span><span class="cx"> case ConstructForwardVarargs:
</span><ins>+ case TailCallForwardVarargs:
+ case TailCallForwardVarargsInlinedCaller:
</ins><span class="cx"> case Breakpoint:
</span><span class="cx"> case ProfileWillCall:
</span><span class="cx"> case ProfileDidCall:
</span><span class="lines">@@ -150,6 +154,8 @@
</span><span class="cx"> case Branch:
</span><span class="cx"> case Switch:
</span><span class="cx"> case Return:
</span><ins>+ case TailCall:
+ case TailCallVarargs:
</ins><span class="cx"> case Throw:
</span><span class="cx"> case CountExecution:
</span><span class="cx"> case ForceOSRExit:
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoredfgDFGFixupPhasecpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/dfg/DFGFixupPhase.cpp (190219 => 190220)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/dfg/DFGFixupPhase.cpp        2015-09-24 21:39:36 UTC (rev 190219)
+++ trunk/Source/JavaScriptCore/dfg/DFGFixupPhase.cpp        2015-09-24 21:42:59 UTC (rev 190220)
</span><span class="lines">@@ -1343,11 +1343,15 @@
</span><span class="cx"> case NotifyWrite:
</span><span class="cx"> case VarInjectionWatchpoint:
</span><span class="cx"> case Call:
</span><ins>+ case TailCallInlinedCaller:
</ins><span class="cx"> case Construct:
</span><span class="cx"> case CallVarargs:
</span><ins>+ case TailCallVarargsInlinedCaller:
</ins><span class="cx"> case ConstructVarargs:
</span><span class="cx"> case CallForwardVarargs:
</span><span class="cx"> case ConstructForwardVarargs:
</span><ins>+ case TailCallForwardVarargs:
+ case TailCallForwardVarargsInlinedCaller:
</ins><span class="cx"> case LoadVarargs:
</span><span class="cx"> case ProfileControlFlow:
</span><span class="cx"> case NewObject:
</span><span class="lines">@@ -1365,6 +1369,8 @@
</span><span class="cx"> case CreateClonedArguments:
</span><span class="cx"> case Jump:
</span><span class="cx"> case Return:
</span><ins>+ case TailCall:
+ case TailCallVarargs:
</ins><span class="cx"> case Throw:
</span><span class="cx"> case ThrowReferenceError:
</span><span class="cx"> case CountExecution:
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoredfgDFGGraphcpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/dfg/DFGGraph.cpp (190219 => 190220)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/dfg/DFGGraph.cpp        2015-09-24 21:39:36 UTC (rev 190219)
+++ trunk/Source/JavaScriptCore/dfg/DFGGraph.cpp        2015-09-24 21:42:59 UTC (rev 190220)
</span><span class="lines">@@ -961,30 +961,31 @@
</span><span class="cx">
</span><span class="cx"> bool Graph::isLiveInBytecode(VirtualRegister operand, CodeOrigin codeOrigin)
</span><span class="cx"> {
</span><ins>+ CodeOrigin* codeOriginPtr = &codeOrigin;
</ins><span class="cx"> for (;;) {
</span><span class="cx"> VirtualRegister reg = VirtualRegister(
</span><del>- operand.offset() - codeOrigin.stackOffset());
</del><ins>+ operand.offset() - codeOriginPtr->stackOffset());
</ins><span class="cx">
</span><del>- if (operand.offset() < codeOrigin.stackOffset() + JSStack::CallFrameHeaderSize) {
</del><ins>+ if (operand.offset() < codeOriginPtr->stackOffset() + JSStack::CallFrameHeaderSize) {
</ins><span class="cx"> if (reg.isArgument()) {
</span><span class="cx"> RELEASE_ASSERT(reg.offset() < JSStack::CallFrameHeaderSize);
</span><span class="cx">
</span><del>- if (codeOrigin.inlineCallFrame->isClosureCall
</del><ins>+ if (codeOriginPtr->inlineCallFrame->isClosureCall
</ins><span class="cx"> && reg.offset() == JSStack::Callee)
</span><span class="cx"> return true;
</span><span class="cx">
</span><del>- if (codeOrigin.inlineCallFrame->isVarargs()
</del><ins>+ if (codeOriginPtr->inlineCallFrame->isVarargs()
</ins><span class="cx"> && reg.offset() == JSStack::ArgumentCount)
</span><span class="cx"> return true;
</span><span class="cx">
</span><span class="cx"> return false;
</span><span class="cx"> }
</span><span class="cx">
</span><del>- return livenessFor(codeOrigin.inlineCallFrame).operandIsLive(
- reg.offset(), codeOrigin.bytecodeIndex);
</del><ins>+ return livenessFor(codeOriginPtr->inlineCallFrame).operandIsLive(
+ reg.offset(), codeOriginPtr->bytecodeIndex);
</ins><span class="cx"> }
</span><span class="cx">
</span><del>- InlineCallFrame* inlineCallFrame = codeOrigin.inlineCallFrame;
</del><ins>+ InlineCallFrame* inlineCallFrame = codeOriginPtr->inlineCallFrame;
</ins><span class="cx"> if (!inlineCallFrame)
</span><span class="cx"> break;
</span><span class="cx">
</span><span class="lines">@@ -994,7 +995,11 @@
</span><span class="cx"> && static_cast<size_t>(reg.toArgument()) < inlineCallFrame->arguments.size())
</span><span class="cx"> return true;
</span><span class="cx">
</span><del>- codeOrigin = inlineCallFrame->caller;
</del><ins>+ codeOriginPtr = inlineCallFrame->getCallerSkippingDeadFrames();
+
+ // The first inline call frame could be an inline tail call
+ if (!codeOriginPtr)
+ break;
</ins><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> return true;
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoredfgDFGGraphh"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/dfg/DFGGraph.h (190219 => 190220)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/dfg/DFGGraph.h        2015-09-24 21:39:36 UTC (rev 190219)
+++ trunk/Source/JavaScriptCore/dfg/DFGGraph.h        2015-09-24 21:42:59 UTC (rev 190220)
</span><span class="lines">@@ -717,9 +717,11 @@
</span><span class="cx"> // call, both callee and caller will see the variables live.
</span><span class="cx"> VirtualRegister exclusionStart;
</span><span class="cx"> VirtualRegister exclusionEnd;
</span><ins>+
+ CodeOrigin* codeOriginPtr = &codeOrigin;
</ins><span class="cx">
</span><span class="cx"> for (;;) {
</span><del>- InlineCallFrame* inlineCallFrame = codeOrigin.inlineCallFrame;
</del><ins>+ InlineCallFrame* inlineCallFrame = codeOriginPtr->inlineCallFrame;
</ins><span class="cx"> VirtualRegister stackOffset(inlineCallFrame ? inlineCallFrame->stackOffset : 0);
</span><span class="cx">
</span><span class="cx"> if (inlineCallFrame) {
</span><span class="lines">@@ -731,7 +733,7 @@
</span><span class="cx">
</span><span class="cx"> CodeBlock* codeBlock = baselineCodeBlockFor(inlineCallFrame);
</span><span class="cx"> FullBytecodeLiveness& fullLiveness = livenessFor(codeBlock);
</span><del>- const FastBitVector& liveness = fullLiveness.getLiveness(codeOrigin.bytecodeIndex);
</del><ins>+ const FastBitVector& liveness = fullLiveness.getLiveness(codeOriginPtr->bytecodeIndex);
</ins><span class="cx"> for (unsigned relativeLocal = codeBlock->m_numCalleeRegisters; relativeLocal--;) {
</span><span class="cx"> VirtualRegister reg = stackOffset + virtualRegisterForLocal(relativeLocal);
</span><span class="cx">
</span><span class="lines">@@ -758,7 +760,11 @@
</span><span class="cx"> for (VirtualRegister reg = exclusionStart; reg < exclusionEnd; reg += 1)
</span><span class="cx"> functor(reg);
</span><span class="cx">
</span><del>- codeOrigin = inlineCallFrame->caller;
</del><ins>+ codeOriginPtr = inlineCallFrame->getCallerSkippingDeadFrames();
+
+ // The first inline call frame could be an inline tail call
+ if (!codeOriginPtr)
+ break;
</ins><span class="cx"> }
</span><span class="cx"> }
</span><span class="cx">
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoredfgDFGInPlaceAbstractStatecpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/dfg/DFGInPlaceAbstractState.cpp (190219 => 190220)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/dfg/DFGInPlaceAbstractState.cpp        2015-09-24 21:39:36 UTC (rev 190219)
+++ trunk/Source/JavaScriptCore/dfg/DFGInPlaceAbstractState.cpp        2015-09-24 21:42:59 UTC (rev 190220)
</span><span class="lines">@@ -364,6 +364,9 @@
</span><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> case Return:
</span><ins>+ case TailCall:
+ case TailCallVarargs:
+ case TailCallForwardVarargs:
</ins><span class="cx"> case Unreachable:
</span><span class="cx"> ASSERT(basicBlock->cfaBranchDirection == InvalidBranchDirection);
</span><span class="cx"> return false;
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoredfgDFGJITCompilercpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/dfg/DFGJITCompiler.cpp (190219 => 190220)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/dfg/DFGJITCompiler.cpp        2015-09-24 21:39:36 UTC (rev 190219)
+++ trunk/Source/JavaScriptCore/dfg/DFGJITCompiler.cpp        2015-09-24 21:42:59 UTC (rev 190220)
</span><span class="lines">@@ -575,8 +575,8 @@
</span><span class="cx"> if (!inlineCallFrame)
</span><span class="cx"> return false;
</span><span class="cx">
</span><del>- bytecodeIndexToCheck = inlineCallFrame->caller.bytecodeIndex;
- codeOrigin = codeOrigin.inlineCallFrame->caller;
</del><ins>+ bytecodeIndexToCheck = inlineCallFrame->directCaller.bytecodeIndex;
+ codeOrigin = codeOrigin.inlineCallFrame->directCaller;
</ins><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> RELEASE_ASSERT_NOT_REACHED();
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoredfgDFGLiveCatchVariablePreservationPhasecpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/dfg/DFGLiveCatchVariablePreservationPhase.cpp (190219 => 190220)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/dfg/DFGLiveCatchVariablePreservationPhase.cpp        2015-09-24 21:39:36 UTC (rev 190219)
+++ trunk/Source/JavaScriptCore/dfg/DFGLiveCatchVariablePreservationPhase.cpp        2015-09-24 21:42:59 UTC (rev 190220)
</span><span class="lines">@@ -81,8 +81,8 @@
</span><span class="cx"> if (!inlineCallFrame)
</span><span class="cx"> return false;
</span><span class="cx">
</span><del>- bytecodeIndexToCheck = inlineCallFrame->caller.bytecodeIndex;
- origin = inlineCallFrame->caller;
</del><ins>+ bytecodeIndexToCheck = inlineCallFrame->directCaller.bytecodeIndex;
+ origin = inlineCallFrame->directCaller;
</ins><span class="cx"> }
</span><span class="cx"> }
</span><span class="cx">
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoredfgDFGNodeh"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/dfg/DFGNode.h (190219 => 190220)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/dfg/DFGNode.h        2015-09-24 21:39:36 UTC (rev 190219)
+++ trunk/Source/JavaScriptCore/dfg/DFGNode.h        2015-09-24 21:42:59 UTC (rev 190220)
</span><span class="lines">@@ -1007,6 +1007,10 @@
</span><span class="cx"> switch (op()) {
</span><span class="cx"> case CallVarargs:
</span><span class="cx"> case CallForwardVarargs:
</span><ins>+ case TailCallVarargs:
+ case TailCallForwardVarargs:
+ case TailCallVarargsInlinedCaller:
+ case TailCallForwardVarargsInlinedCaller:
</ins><span class="cx"> case ConstructVarargs:
</span><span class="cx"> case ConstructForwardVarargs:
</span><span class="cx"> return true;
</span><span class="lines">@@ -1104,6 +1108,9 @@
</span><span class="cx"> case Branch:
</span><span class="cx"> case Switch:
</span><span class="cx"> case Return:
</span><ins>+ case TailCall:
+ case TailCallVarargs:
+ case TailCallForwardVarargs:
</ins><span class="cx"> case Unreachable:
</span><span class="cx"> return true;
</span><span class="cx"> default:
</span><span class="lines">@@ -1254,10 +1261,13 @@
</span><span class="cx"> case GetByIdFlush:
</span><span class="cx"> case GetByVal:
</span><span class="cx"> case Call:
</span><ins>+ case TailCallInlinedCaller:
</ins><span class="cx"> case Construct:
</span><span class="cx"> case CallVarargs:
</span><ins>+ case TailCallVarargsInlinedCaller:
</ins><span class="cx"> case ConstructVarargs:
</span><span class="cx"> case CallForwardVarargs:
</span><ins>+ case TailCallForwardVarargsInlinedCaller:
</ins><span class="cx"> case GetByOffset:
</span><span class="cx"> case MultiGetByOffset:
</span><span class="cx"> case GetClosureVar:
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoredfgDFGNodeTypeh"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/dfg/DFGNodeType.h (190219 => 190220)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/dfg/DFGNodeType.h        2015-09-24 21:39:36 UTC (rev 190219)
+++ trunk/Source/JavaScriptCore/dfg/DFGNodeType.h        2015-09-24 21:42:59 UTC (rev 190220)
</span><span class="lines">@@ -245,6 +245,9 @@
</span><span class="cx"> macro(CallForwardVarargs, NodeResultJS | NodeMustGenerate) \
</span><span class="cx"> macro(ConstructVarargs, NodeResultJS | NodeMustGenerate) \
</span><span class="cx"> macro(ConstructForwardVarargs, NodeResultJS | NodeMustGenerate) \
</span><ins>+ macro(TailCallInlinedCaller, NodeResultJS | NodeMustGenerate | NodeHasVarArgs) \
+ macro(TailCallVarargsInlinedCaller, NodeResultJS | NodeMustGenerate) \
+ macro(TailCallForwardVarargsInlinedCaller, NodeResultJS | NodeMustGenerate) \
</ins><span class="cx"> \
</span><span class="cx"> /* Allocations. */\
</span><span class="cx"> macro(NewObject, NodeResultJS) \
</span><span class="lines">@@ -310,6 +313,9 @@
</span><span class="cx"> macro(Branch, NodeMustGenerate) \
</span><span class="cx"> macro(Switch, NodeMustGenerate) \
</span><span class="cx"> macro(Return, NodeMustGenerate) \
</span><ins>+ macro(TailCall, NodeMustGenerate | NodeHasVarArgs) \
+ macro(TailCallVarargs, NodeMustGenerate) \
+ macro(TailCallForwardVarargs, NodeMustGenerate) \
</ins><span class="cx"> macro(Unreachable, NodeMustGenerate) \
</span><span class="cx"> \
</span><span class="cx"> /* Count execution. */\
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoredfgDFGOSRExitCompilerCommoncpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/dfg/DFGOSRExitCompilerCommon.cpp (190219 => 190220)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/dfg/DFGOSRExitCompilerCommon.cpp        2015-09-24 21:39:36 UTC (rev 190219)
+++ trunk/Source/JavaScriptCore/dfg/DFGOSRExitCompilerCommon.cpp        2015-09-24 21:42:59 UTC (rev 190220)
</span><span class="lines">@@ -64,7 +64,7 @@
</span><span class="cx">
</span><span class="cx"> AssemblyHelpers::JumpList loopThreshold;
</span><span class="cx">
</span><del>- for (InlineCallFrame* inlineCallFrame = exit.m_codeOrigin.inlineCallFrame; inlineCallFrame; inlineCallFrame = inlineCallFrame->caller.inlineCallFrame) {
</del><ins>+ for (InlineCallFrame* inlineCallFrame = exit.m_codeOrigin.inlineCallFrame; inlineCallFrame; inlineCallFrame = inlineCallFrame->directCaller.inlineCallFrame) {
</ins><span class="cx"> loopThreshold.append(
</span><span class="cx"> jit.branchTest8(
</span><span class="cx"> AssemblyHelpers::NonZero,
</span><span class="lines">@@ -136,63 +136,80 @@
</span><span class="cx">
</span><span class="cx"> void reifyInlinedCallFrames(CCallHelpers& jit, const OSRExitBase& exit)
</span><span class="cx"> {
</span><ins>+ // FIXME: We shouldn't leave holes on the stack when performing an OSR exit
+ // in presence of inlined tail calls.
+ // https://bugs.webkit.org/show_bug.cgi?id=147511
</ins><span class="cx"> ASSERT(jit.baselineCodeBlock()->jitType() == JITCode::BaselineJIT);
</span><span class="cx"> jit.storePtr(AssemblyHelpers::TrustedImmPtr(jit.baselineCodeBlock()), AssemblyHelpers::addressFor((VirtualRegister)JSStack::CodeBlock));
</span><span class="cx">
</span><del>- CodeOrigin codeOrigin;
- for (codeOrigin = exit.m_codeOrigin; codeOrigin.inlineCallFrame; codeOrigin = codeOrigin.inlineCallFrame->caller) {
- InlineCallFrame* inlineCallFrame = codeOrigin.inlineCallFrame;
- CodeBlock* baselineCodeBlock = jit.baselineCodeBlockFor(codeOrigin);
- CodeBlock* baselineCodeBlockForCaller = jit.baselineCodeBlockFor(inlineCallFrame->caller);
- void* jumpTarget = nullptr;
</del><ins>+ const CodeOrigin* codeOrigin;
+ for (codeOrigin = &exit.m_codeOrigin; codeOrigin && codeOrigin->inlineCallFrame; codeOrigin = codeOrigin->inlineCallFrame->getCallerSkippingDeadFrames()) {
+ InlineCallFrame* inlineCallFrame = codeOrigin->inlineCallFrame;
+ CodeBlock* baselineCodeBlock = jit.baselineCodeBlockFor(*codeOrigin);
+ CodeOrigin* trueCaller = inlineCallFrame->getCallerSkippingDeadFrames();
</ins><span class="cx"> void* trueReturnPC = nullptr;
</span><del>-
- unsigned callBytecodeIndex = inlineCallFrame->caller.bytecodeIndex;
-
- switch (inlineCallFrame->kind) {
- case InlineCallFrame::Call:
- case InlineCallFrame::Construct:
- case InlineCallFrame::CallVarargs:
- case InlineCallFrame::ConstructVarargs: {
- CallLinkInfo* callLinkInfo =
- baselineCodeBlockForCaller->getCallLinkInfoForBytecodeIndex(callBytecodeIndex);
- RELEASE_ASSERT(callLinkInfo);
-
- jumpTarget = callLinkInfo->callReturnLocation().executableAddress();
- break;
- }
-
- case InlineCallFrame::GetterCall:
- case InlineCallFrame::SetterCall: {
- StructureStubInfo* stubInfo =
- baselineCodeBlockForCaller->findStubInfo(CodeOrigin(callBytecodeIndex));
- RELEASE_ASSERT(stubInfo);
-
</del><ins>+ GPRReg callerFrameGPR = GPRInfo::callFrameRegister;
+
+ if (!trueCaller) {
+ ASSERT(inlineCallFrame->isTail());
+ jit.loadPtr(AssemblyHelpers::Address(GPRInfo::callFrameRegister, CallFrame::returnPCOffset()), GPRInfo::regT3);
+ jit.storePtr(GPRInfo::regT3, AssemblyHelpers::addressForByteOffset(inlineCallFrame->returnPCOffset()));
+ jit.loadPtr(AssemblyHelpers::Address(GPRInfo::callFrameRegister, CallFrame::callerFrameOffset()), GPRInfo::regT3);
+ callerFrameGPR = GPRInfo::regT3;
+ } else {
+ CodeBlock* baselineCodeBlockForCaller = jit.baselineCodeBlockFor(*trueCaller);
+ unsigned callBytecodeIndex = trueCaller->bytecodeIndex;
+ void* jumpTarget = nullptr;
+
</ins><span class="cx"> switch (inlineCallFrame->kind) {
</span><ins>+ case InlineCallFrame::Call:
+ case InlineCallFrame::Construct:
+ case InlineCallFrame::CallVarargs:
+ case InlineCallFrame::ConstructVarargs:
+ case InlineCallFrame::TailCall:
+ case InlineCallFrame::TailCallVarargs: {
+ CallLinkInfo* callLinkInfo =
+ baselineCodeBlockForCaller->getCallLinkInfoForBytecodeIndex(callBytecodeIndex);
+ RELEASE_ASSERT(callLinkInfo);
+
+ jumpTarget = callLinkInfo->callReturnLocation().executableAddress();
+ break;
+ }
+
</ins><span class="cx"> case InlineCallFrame::GetterCall:
</span><del>- jumpTarget = jit.vm()->getCTIStub(baselineGetterReturnThunkGenerator).code().executableAddress();
</del><ins>+ case InlineCallFrame::SetterCall: {
+ StructureStubInfo* stubInfo =
+ baselineCodeBlockForCaller->findStubInfo(CodeOrigin(callBytecodeIndex));
+ RELEASE_ASSERT(stubInfo);
+
+ switch (inlineCallFrame->kind) {
+ case InlineCallFrame::GetterCall:
+ jumpTarget = jit.vm()->getCTIStub(baselineGetterReturnThunkGenerator).code().executableAddress();
+ break;
+ case InlineCallFrame::SetterCall:
+ jumpTarget = jit.vm()->getCTIStub(baselineSetterReturnThunkGenerator).code().executableAddress();
+ break;
+ default:
+ RELEASE_ASSERT_NOT_REACHED();
+ break;
+ }
+
+ trueReturnPC = stubInfo->callReturnLocation.labelAtOffset(
+ stubInfo->patch.deltaCallToDone).executableAddress();
</ins><span class="cx"> break;
</span><del>- case InlineCallFrame::SetterCall:
- jumpTarget = jit.vm()->getCTIStub(baselineSetterReturnThunkGenerator).code().executableAddress();
- break;
- default:
- RELEASE_ASSERT_NOT_REACHED();
- break;
</del><ins>+ } }
+
+ if (trueCaller->inlineCallFrame) {
+ jit.addPtr(
+ AssemblyHelpers::TrustedImm32(trueCaller->inlineCallFrame->stackOffset * sizeof(EncodedJSValue)),
+ GPRInfo::callFrameRegister,
+ GPRInfo::regT3);
+ callerFrameGPR = GPRInfo::regT3;
</ins><span class="cx"> }
</span><del>-
- trueReturnPC = stubInfo->callReturnLocation.labelAtOffset(
- stubInfo->patch.deltaCallToDone).executableAddress();
- break;
- } }
</del><span class="cx">
</span><del>- GPRReg callerFrameGPR;
- if (inlineCallFrame->caller.inlineCallFrame) {
- jit.addPtr(AssemblyHelpers::TrustedImm32(inlineCallFrame->caller.inlineCallFrame->stackOffset * sizeof(EncodedJSValue)), GPRInfo::callFrameRegister, GPRInfo::regT3);
- callerFrameGPR = GPRInfo::regT3;
- } else
- callerFrameGPR = GPRInfo::callFrameRegister;
-
- jit.storePtr(AssemblyHelpers::TrustedImmPtr(jumpTarget), AssemblyHelpers::addressForByteOffset(inlineCallFrame->returnPCOffset()));
</del><ins>+ jit.storePtr(AssemblyHelpers::TrustedImmPtr(jumpTarget), AssemblyHelpers::addressForByteOffset(inlineCallFrame->returnPCOffset()));
+ }
+
</ins><span class="cx"> if (trueReturnPC)
</span><span class="cx"> jit.storePtr(AssemblyHelpers::TrustedImmPtr(trueReturnPC), AssemblyHelpers::addressFor(inlineCallFrame->stackOffset + virtualRegisterForArgument(inlineCallFrame->arguments.size()).offset()));
</span><span class="cx">
</span><span class="lines">@@ -202,13 +219,13 @@
</span><span class="cx"> jit.store32(AssemblyHelpers::TrustedImm32(inlineCallFrame->arguments.size()), AssemblyHelpers::payloadFor((VirtualRegister)(inlineCallFrame->stackOffset + JSStack::ArgumentCount)));
</span><span class="cx"> #if USE(JSVALUE64)
</span><span class="cx"> jit.store64(callerFrameGPR, AssemblyHelpers::addressForByteOffset(inlineCallFrame->callerFrameOffset()));
</span><del>- uint32_t locationBits = CallSiteIndex(codeOrigin.bytecodeIndex).bits();
</del><ins>+ uint32_t locationBits = CallSiteIndex(codeOrigin->bytecodeIndex).bits();
</ins><span class="cx"> jit.store32(AssemblyHelpers::TrustedImm32(locationBits), AssemblyHelpers::tagFor((VirtualRegister)(inlineCallFrame->stackOffset + JSStack::ArgumentCount)));
</span><span class="cx"> if (!inlineCallFrame->isClosureCall)
</span><span class="cx"> jit.store64(AssemblyHelpers::TrustedImm64(JSValue::encode(JSValue(inlineCallFrame->calleeConstant()))), AssemblyHelpers::addressFor((VirtualRegister)(inlineCallFrame->stackOffset + JSStack::Callee)));
</span><span class="cx"> #else // USE(JSVALUE64) // so this is the 32-bit part
</span><span class="cx"> jit.storePtr(callerFrameGPR, AssemblyHelpers::addressForByteOffset(inlineCallFrame->callerFrameOffset()));
</span><del>- Instruction* instruction = baselineCodeBlock->instructions().begin() + codeOrigin.bytecodeIndex;
</del><ins>+ Instruction* instruction = baselineCodeBlock->instructions().begin() + codeOrigin->bytecodeIndex;
</ins><span class="cx"> uint32_t locationBits = CallSiteIndex(instruction).bits();
</span><span class="cx"> jit.store32(AssemblyHelpers::TrustedImm32(locationBits), AssemblyHelpers::tagFor((VirtualRegister)(inlineCallFrame->stackOffset + JSStack::ArgumentCount)));
</span><span class="cx"> jit.store32(AssemblyHelpers::TrustedImm32(JSValue::CellTag), AssemblyHelpers::tagFor((VirtualRegister)(inlineCallFrame->stackOffset + JSStack::Callee)));
</span><span class="lines">@@ -217,13 +234,16 @@
</span><span class="cx"> #endif // USE(JSVALUE64) // ending the #else part, so directly above is the 32-bit part
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+ // Don't need to set the toplevel code origin if we only did inline tail calls
+ if (codeOrigin) {
</ins><span class="cx"> #if USE(JSVALUE64)
</span><del>- uint32_t locationBits = CallSiteIndex(codeOrigin.bytecodeIndex).bits();
</del><ins>+ uint32_t locationBits = CallSiteIndex(codeOrigin->bytecodeIndex).bits();
</ins><span class="cx"> #else
</span><del>- Instruction* instruction = jit.baselineCodeBlock()->instructions().begin() + codeOrigin.bytecodeIndex;
</del><ins>+ Instruction* instruction = jit.baselineCodeBlock()->instructions().begin() + codeOrigin->bytecodeIndex;
</ins><span class="cx"> uint32_t locationBits = CallSiteIndex(instruction).bits();
</span><span class="cx"> #endif
</span><del>- jit.store32(AssemblyHelpers::TrustedImm32(locationBits), AssemblyHelpers::tagFor((VirtualRegister)(JSStack::ArgumentCount)));
</del><ins>+ jit.store32(AssemblyHelpers::TrustedImm32(locationBits), AssemblyHelpers::tagFor((VirtualRegister)(JSStack::ArgumentCount)));
+ }
</ins><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> static void osrWriteBarrier(CCallHelpers& jit, GPRReg owner, GPRReg scratch)
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoredfgDFGOSRExitPreparationcpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/dfg/DFGOSRExitPreparation.cpp (190219 => 190220)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/dfg/DFGOSRExitPreparation.cpp        2015-09-24 21:39:36 UTC (rev 190219)
+++ trunk/Source/JavaScriptCore/dfg/DFGOSRExitPreparation.cpp        2015-09-24 21:42:59 UTC (rev 190220)
</span><span class="lines">@@ -41,7 +41,7 @@
</span><span class="cx"> VM& vm = exec->vm();
</span><span class="cx"> DeferGC deferGC(vm.heap);
</span><span class="cx">
</span><del>- for (; codeOrigin.inlineCallFrame; codeOrigin = codeOrigin.inlineCallFrame->caller) {
</del><ins>+ for (; codeOrigin.inlineCallFrame; codeOrigin = codeOrigin.inlineCallFrame->directCaller) {
</ins><span class="cx"> CodeBlock* codeBlock = codeOrigin.inlineCallFrame->baselineCodeBlock();
</span><span class="cx"> if (codeBlock->jitType() == JSC::JITCode::BaselineJIT)
</span><span class="cx"> continue;
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoredfgDFGOperationscpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/dfg/DFGOperations.cpp (190219 => 190220)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/dfg/DFGOperations.cpp        2015-09-24 21:39:36 UTC (rev 190219)
+++ trunk/Source/JavaScriptCore/dfg/DFGOperations.cpp        2015-09-24 21:42:59 UTC (rev 190220)
</span><span class="lines">@@ -1306,7 +1306,7 @@
</span><span class="cx"> ASSERT(JITCode::isOptimizingJIT(optimizedCodeBlock->jitType()));
</span><span class="cx">
</span><span class="cx"> bool didTryToEnterIntoInlinedLoops = false;
</span><del>- for (InlineCallFrame* inlineCallFrame = exit->m_codeOrigin.inlineCallFrame; inlineCallFrame; inlineCallFrame = inlineCallFrame->caller.inlineCallFrame) {
</del><ins>+ for (InlineCallFrame* inlineCallFrame = exit->m_codeOrigin.inlineCallFrame; inlineCallFrame; inlineCallFrame = inlineCallFrame->directCaller.inlineCallFrame) {
</ins><span class="cx"> if (inlineCallFrame->executable->didTryToEnterInLoop()) {
</span><span class="cx"> didTryToEnterIntoInlinedLoops = true;
</span><span class="cx"> break;
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoredfgDFGPreciseLocalClobberizeh"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/dfg/DFGPreciseLocalClobberize.h (190219 => 190220)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/dfg/DFGPreciseLocalClobberize.h        2015-09-24 21:39:36 UTC (rev 190219)
+++ trunk/Source/JavaScriptCore/dfg/DFGPreciseLocalClobberize.h        2015-09-24 21:42:59 UTC (rev 190220)
</span><span class="lines">@@ -111,7 +111,9 @@
</span><span class="cx"> case GetMyArgumentByVal:
</span><span class="cx"> case ForwardVarargs:
</span><span class="cx"> case CallForwardVarargs:
</span><del>- case ConstructForwardVarargs: {
</del><ins>+ case ConstructForwardVarargs:
+ case TailCallForwardVarargs:
+ case TailCallForwardVarargsInlinedCaller: {
</ins><span class="cx"> InlineCallFrame* inlineCallFrame = m_node->child1()->origin.semantic.inlineCallFrame;
</span><span class="cx"> if (!inlineCallFrame) {
</span><span class="cx"> // Read the outermost arguments and argument count.
</span><span class="lines">@@ -138,7 +140,7 @@
</span><span class="cx"> m_read(VirtualRegister(i));
</span><span class="cx">
</span><span class="cx"> // Read all of the inline arguments and call frame headers that we didn't already capture.
</span><del>- for (InlineCallFrame* inlineCallFrame = m_node->origin.semantic.inlineCallFrame; inlineCallFrame; inlineCallFrame = inlineCallFrame->caller.inlineCallFrame) {
</del><ins>+ for (InlineCallFrame* inlineCallFrame = m_node->origin.semantic.inlineCallFrame; inlineCallFrame; inlineCallFrame = inlineCallFrame->getCallerInlineFrameSkippingDeadFrames()) {
</ins><span class="cx"> for (unsigned i = inlineCallFrame->arguments.size(); i-- > 1;)
</span><span class="cx"> m_read(VirtualRegister(inlineCallFrame->stackOffset + virtualRegisterForArgument(i).offset()));
</span><span class="cx"> if (inlineCallFrame->isClosureCall)
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoredfgDFGPredictionPropagationPhasecpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/dfg/DFGPredictionPropagationPhase.cpp (190219 => 190220)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/dfg/DFGPredictionPropagationPhase.cpp        2015-09-24 21:39:36 UTC (rev 190219)
+++ trunk/Source/JavaScriptCore/dfg/DFGPredictionPropagationPhase.cpp        2015-09-24 21:42:59 UTC (rev 190220)
</span><span class="lines">@@ -194,11 +194,14 @@
</span><span class="cx"> case MultiGetByOffset:
</span><span class="cx"> case GetDirectPname:
</span><span class="cx"> case Call:
</span><ins>+ case TailCallInlinedCaller:
</ins><span class="cx"> case Construct:
</span><span class="cx"> case CallVarargs:
</span><ins>+ case TailCallVarargsInlinedCaller:
</ins><span class="cx"> case ConstructVarargs:
</span><span class="cx"> case CallForwardVarargs:
</span><span class="cx"> case ConstructForwardVarargs:
</span><ins>+ case TailCallForwardVarargsInlinedCaller:
</ins><span class="cx"> case GetGlobalVar:
</span><span class="cx"> case GetGlobalLexicalVariable:
</span><span class="cx"> case GetClosureVar:
</span><span class="lines">@@ -634,6 +637,9 @@
</span><span class="cx"> case PutClosureVar:
</span><span class="cx"> case PutToArguments:
</span><span class="cx"> case Return:
</span><ins>+ case TailCall:
+ case TailCallVarargs:
+ case TailCallForwardVarargs:
</ins><span class="cx"> case Throw:
</span><span class="cx"> case PutById:
</span><span class="cx"> case PutByIdFlush:
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoredfgDFGSafeToExecuteh"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/dfg/DFGSafeToExecute.h (190219 => 190220)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/dfg/DFGSafeToExecute.h        2015-09-24 21:39:36 UTC (rev 190219)
+++ trunk/Source/JavaScriptCore/dfg/DFGSafeToExecute.h        2015-09-24 21:42:59 UTC (rev 190220)
</span><span class="lines">@@ -216,8 +216,11 @@
</span><span class="cx"> case CompareEq:
</span><span class="cx"> case CompareStrictEq:
</span><span class="cx"> case Call:
</span><ins>+ case TailCallInlinedCaller:
</ins><span class="cx"> case Construct:
</span><span class="cx"> case CallVarargs:
</span><ins>+ case TailCallVarargsInlinedCaller:
+ case TailCallForwardVarargsInlinedCaller:
</ins><span class="cx"> case ConstructVarargs:
</span><span class="cx"> case LoadVarargs:
</span><span class="cx"> case CallForwardVarargs:
</span><span class="lines">@@ -262,6 +265,9 @@
</span><span class="cx"> case Branch:
</span><span class="cx"> case Switch:
</span><span class="cx"> case Return:
</span><ins>+ case TailCall:
+ case TailCallVarargs:
+ case TailCallForwardVarargs:
</ins><span class="cx"> case Throw:
</span><span class="cx"> case ThrowReferenceError:
</span><span class="cx"> case CountExecution:
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoredfgDFGSpeculativeJIT32_64cpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/dfg/DFGSpeculativeJIT32_64.cpp (190219 => 190220)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/dfg/DFGSpeculativeJIT32_64.cpp        2015-09-24 21:39:36 UTC (rev 190219)
+++ trunk/Source/JavaScriptCore/dfg/DFGSpeculativeJIT32_64.cpp        2015-09-24 21:42:59 UTC (rev 190220)
</span><span class="lines">@@ -30,6 +30,7 @@
</span><span class="cx"> #if ENABLE(DFG_JIT)
</span><span class="cx">
</span><span class="cx"> #include "ArrayPrototype.h"
</span><ins>+#include "CallFrameShuffler.h"
</ins><span class="cx"> #include "DFGAbstractInterpreterInlines.h"
</span><span class="cx"> #include "DFGCallArrayAllocatorSlowPathGenerator.h"
</span><span class="cx"> #include "DFGOperations.h"
</span><span class="lines">@@ -632,10 +633,20 @@
</span><span class="cx"> CallLinkInfo::CallType callType;
</span><span class="cx"> bool isVarargs = false;
</span><span class="cx"> bool isForwardVarargs = false;
</span><ins>+ bool isTail = false;
+ bool isEmulatedTail = false;
</ins><span class="cx"> switch (node->op()) {
</span><span class="cx"> case Call:
</span><span class="cx"> callType = CallLinkInfo::Call;
</span><span class="cx"> break;
</span><ins>+ case TailCall:
+ callType = CallLinkInfo::TailCall;
+ isTail = true;
+ break;
+ case TailCallInlinedCaller:
+ callType = CallLinkInfo::Call;
+ isEmulatedTail = true;
+ break;
</ins><span class="cx"> case Construct:
</span><span class="cx"> callType = CallLinkInfo::Construct;
</span><span class="cx"> break;
</span><span class="lines">@@ -643,6 +654,16 @@
</span><span class="cx"> callType = CallLinkInfo::CallVarargs;
</span><span class="cx"> isVarargs = true;
</span><span class="cx"> break;
</span><ins>+ case TailCallVarargs:
+ callType = CallLinkInfo::TailCallVarargs;
+ isVarargs = true;
+ isTail = true;
+ break;
+ case TailCallVarargsInlinedCaller:
+ callType = CallLinkInfo::CallVarargs;
+ isVarargs = true;
+ isEmulatedTail = true;
+ break;
</ins><span class="cx"> case ConstructVarargs:
</span><span class="cx"> callType = CallLinkInfo::ConstructVarargs;
</span><span class="cx"> isVarargs = true;
</span><span class="lines">@@ -651,6 +672,16 @@
</span><span class="cx"> callType = CallLinkInfo::CallVarargs;
</span><span class="cx"> isForwardVarargs = true;
</span><span class="cx"> break;
</span><ins>+ case TailCallForwardVarargs:
+ callType = CallLinkInfo::TailCallVarargs;
+ isTail = true;
+ isForwardVarargs = true;
+ break;
+ case TailCallForwardVarargsInlinedCaller:
+ callType = CallLinkInfo::CallVarargs;
+ isEmulatedTail = true;
+ isForwardVarargs = true;
+ break;
</ins><span class="cx"> case ConstructForwardVarargs:
</span><span class="cx"> callType = CallLinkInfo::ConstructVarargs;
</span><span class="cx"> isForwardVarargs = true;
</span><span class="lines">@@ -661,6 +692,9 @@
</span><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> Edge calleeEdge = m_jit.graph().child(node, 0);
</span><ins>+ GPRReg calleeTagGPR;
+ GPRReg calleePayloadGPR;
+ CallFrameShuffleData shuffleData;
</ins><span class="cx">
</span><span class="cx"> // Gotta load the arguments somehow. Varargs is trickier.
</span><span class="cx"> if (isVarargs || isForwardVarargs) {
</span><span class="lines">@@ -755,28 +789,49 @@
</span><span class="cx"> // receiver (method call). subsequent children are the arguments.
</span><span class="cx"> int numPassedArgs = node->numChildren() - 1;
</span><span class="cx">
</span><del>- m_jit.store32(MacroAssembler::TrustedImm32(numPassedArgs), m_jit.calleeFramePayloadSlot(JSStack::ArgumentCount));
</del><ins>+ if (node->op() == TailCall) {
+ JSValueOperand callee(this, calleeEdge);
+ calleeTagGPR = callee.tagGPR();
+ calleePayloadGPR = callee.payloadGPR();
+ use(calleeEdge);
+
+ shuffleData.numLocals = m_jit.graph().frameRegisterCount();
+ shuffleData.callee = ValueRecovery::inPair(calleeTagGPR, calleePayloadGPR);
+ shuffleData.args.resize(numPassedArgs);
+
+ for (int i = 0; i < numPassedArgs; ++i) {
+ Edge argEdge = m_jit.graph().varArgChild(node, i + 1);
+ GenerationInfo& info = generationInfo(argEdge.node());
+ use(argEdge);
+ shuffleData.args[i] = info.recovery(argEdge->virtualRegister());
+ }
+ } else {
+ m_jit.store32(MacroAssembler::TrustedImm32(numPassedArgs), m_jit.calleeFramePayloadSlot(JSStack::ArgumentCount));
</ins><span class="cx">
</span><del>- for (int i = 0; i < numPassedArgs; i++) {
- Edge argEdge = m_jit.graph().m_varArgChildren[node->firstChild() + 1 + i];
- JSValueOperand arg(this, argEdge);
- GPRReg argTagGPR = arg.tagGPR();
- GPRReg argPayloadGPR = arg.payloadGPR();
- use(argEdge);
</del><ins>+ for (int i = 0; i < numPassedArgs; i++) {
+ Edge argEdge = m_jit.graph().m_varArgChildren[node->firstChild() + 1 + i];
+ JSValueOperand arg(this, argEdge);
+ GPRReg argTagGPR = arg.tagGPR();
+ GPRReg argPayloadGPR = arg.payloadGPR();
+ use(argEdge);
</ins><span class="cx">
</span><del>- m_jit.store32(argTagGPR, m_jit.calleeArgumentTagSlot(i));
- m_jit.store32(argPayloadGPR, m_jit.calleeArgumentPayloadSlot(i));
</del><ins>+ m_jit.store32(argTagGPR, m_jit.calleeArgumentTagSlot(i));
+ m_jit.store32(argPayloadGPR, m_jit.calleeArgumentPayloadSlot(i));
+ }
</ins><span class="cx"> }
</span><span class="cx"> }
</span><span class="cx">
</span><del>- JSValueOperand callee(this, calleeEdge);
- GPRReg calleeTagGPR = callee.tagGPR();
- GPRReg calleePayloadGPR = callee.payloadGPR();
- use(calleeEdge);
- m_jit.store32(calleePayloadGPR, m_jit.calleeFramePayloadSlot(JSStack::Callee));
- m_jit.store32(calleeTagGPR, m_jit.calleeFrameTagSlot(JSStack::Callee));
</del><ins>+ if (node->op() != TailCall) {
+ JSValueOperand callee(this, calleeEdge);
+ calleeTagGPR = callee.tagGPR();
+ calleePayloadGPR = callee.payloadGPR();
+ use(calleeEdge);
+ m_jit.store32(calleePayloadGPR, m_jit.calleeFramePayloadSlot(JSStack::Callee));
+ m_jit.store32(calleeTagGPR, m_jit.calleeFrameTagSlot(JSStack::Callee));
</ins><span class="cx">
</span><del>- flushRegisters();
</del><ins>+ if (!isTail)
+ flushRegisters();
+ }
</ins><span class="cx">
</span><span class="cx"> GPRFlushedCallResult resultPayload(this);
</span><span class="cx"> GPRFlushedCallResult2 resultTag(this);
</span><span class="lines">@@ -786,48 +841,77 @@
</span><span class="cx"> JITCompiler::DataLabelPtr targetToCheck;
</span><span class="cx"> JITCompiler::JumpList slowPath;
</span><span class="cx">
</span><del>- CallSiteIndex callSite = m_jit.recordCallSiteAndGenerateExceptionHandlingOSRExitIfNeeded(node->origin.semantic, m_stream->size());
</del><ins>+ CodeOrigin staticOrigin = node->origin.semantic;
+ ASSERT(!isTail || !staticOrigin.inlineCallFrame || !staticOrigin.inlineCallFrame->getCallerSkippingDeadFrames());
+ ASSERT(!isEmulatedTail || (staticOrigin.inlineCallFrame && staticOrigin.inlineCallFrame->getCallerSkippingDeadFrames()));
+ CodeOrigin dynamicOrigin =
+ isEmulatedTail ? *staticOrigin.inlineCallFrame->getCallerSkippingDeadFrames() : staticOrigin;
+ CallSiteIndex callSite = m_jit.recordCallSiteAndGenerateExceptionHandlingOSRExitIfNeeded(dynamicOrigin, m_stream->size());
</ins><span class="cx"> m_jit.emitStoreCallSiteIndex(callSite);
</span><span class="cx">
</span><span class="cx"> CallLinkInfo* info = m_jit.codeBlock()->addCallLinkInfo();
</span><span class="cx">
</span><del>- slowPath.append(m_jit.branchIfNotCell(callee.jsValueRegs()));
</del><ins>+ slowPath.append(m_jit.branchIfNotCell(JSValueRegs(calleeTagGPR, calleePayloadGPR)));
</ins><span class="cx"> slowPath.append(m_jit.branchPtrWithPatch(MacroAssembler::NotEqual, calleePayloadGPR, targetToCheck));
</span><span class="cx">
</span><del>- JITCompiler::Call fastCall = m_jit.nearCall();
</del><ins>+ if (isTail) {
+ if (node->op() == TailCall) {
+ info->setFrameShuffleData(shuffleData);
+ CallFrameShuffler(m_jit, shuffleData).prepareForTailCall();
+ } else {
+ m_jit.emitRestoreCalleeSaves();
+ m_jit.prepareForTailCallSlow();
+ }
+ }
</ins><span class="cx">
</span><ins>+ JITCompiler::Call fastCall = isTail ? m_jit.nearTailCall() : m_jit.nearCall();
+
</ins><span class="cx"> JITCompiler::Jump done = m_jit.jump();
</span><span class="cx">
</span><span class="cx"> slowPath.link(&m_jit);
</span><span class="cx">
</span><del>- // Callee payload needs to be in regT0, tag in regT1
- if (calleeTagGPR == GPRInfo::regT0) {
- if (calleePayloadGPR == GPRInfo::regT1)
- m_jit.swap(GPRInfo::regT1, GPRInfo::regT0);
- else {
</del><ins>+ if (node->op() == TailCall) {
+ CallFrameShuffler callFrameShuffler(m_jit, shuffleData);
+ callFrameShuffler.setCalleeJSValueRegs(JSValueRegs(
+ GPRInfo::regT1, GPRInfo::regT0));
+ callFrameShuffler.prepareForSlowPath();
+ } else {
+ // Callee payload needs to be in regT0, tag in regT1
+ if (calleeTagGPR == GPRInfo::regT0) {
+ if (calleePayloadGPR == GPRInfo::regT1)
+ m_jit.swap(GPRInfo::regT1, GPRInfo::regT0);
+ else {
+ m_jit.move(calleeTagGPR, GPRInfo::regT1);
+ m_jit.move(calleePayloadGPR, GPRInfo::regT0);
+ }
+ } else {
+ m_jit.move(calleePayloadGPR, GPRInfo::regT0);
</ins><span class="cx"> m_jit.move(calleeTagGPR, GPRInfo::regT1);
</span><del>- m_jit.move(calleePayloadGPR, GPRInfo::regT0);
</del><span class="cx"> }
</span><del>- } else {
- m_jit.move(calleePayloadGPR, GPRInfo::regT0);
- m_jit.move(calleeTagGPR, GPRInfo::regT1);
</del><ins>+
+ if (isTail)
+ m_jit.emitRestoreCalleeSaves();
</ins><span class="cx"> }
</span><ins>+
</ins><span class="cx"> m_jit.move(MacroAssembler::TrustedImmPtr(info), GPRInfo::regT2);
</span><span class="cx"> JITCompiler::Call slowCall = m_jit.nearCall();
</span><span class="cx">
</span><span class="cx"> done.link(&m_jit);
</span><span class="cx">
</span><del>- m_jit.setupResults(resultPayloadGPR, resultTagGPR);
</del><ins>+ if (isTail)
+ m_jit.abortWithReason(JITDidReturnFromTailCall);
+ else {
+ m_jit.setupResults(resultPayloadGPR, resultTagGPR);
</ins><span class="cx">
</span><del>- jsValueResult(resultTagGPR, resultPayloadGPR, node, DataFormatJS, UseChildrenCalledExplicitly);
</del><ins>+ jsValueResult(resultTagGPR, resultPayloadGPR, node, DataFormatJS, UseChildrenCalledExplicitly);
+ // After the calls are done, we need to reestablish our stack
+ // pointer. We rely on this for varargs calls, calls with arity
+ // mismatch (the callframe is slided) and tail calls.
+ m_jit.addPtr(TrustedImm32(m_jit.graph().stackPointerOffset() * sizeof(Register)), GPRInfo::callFrameRegister, JITCompiler::stackPointerRegister);
+ }
</ins><span class="cx">
</span><span class="cx"> info->setUpCall(callType, node->origin.semantic, calleePayloadGPR);
</span><span class="cx"> m_jit.addJSCall(fastCall, slowCall, targetToCheck, info);
</span><del>-
- // After the calls are done, we need to reestablish our stack
- // pointer. We rely on this for varargs calls, calls with arity
- // mismatch (the callframe is slided) and tail calls.
- m_jit.addPtr(TrustedImm32(m_jit.graph().stackPointerOffset() * sizeof(Register)), GPRInfo::callFrameRegister, JITCompiler::stackPointerRegister);
</del><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> template<bool strict>
</span><span class="lines">@@ -4253,10 +4337,16 @@
</span><span class="cx"> break;
</span><span class="cx">
</span><span class="cx"> case Call:
</span><ins>+ case TailCall:
+ case TailCallInlinedCaller:
</ins><span class="cx"> case Construct:
</span><span class="cx"> case CallVarargs:
</span><ins>+ case TailCallVarargs:
+ case TailCallVarargsInlinedCaller:
+ case ConstructVarargs:
</ins><span class="cx"> case CallForwardVarargs:
</span><del>- case ConstructVarargs:
</del><ins>+ case TailCallForwardVarargs:
+ case TailCallForwardVarargsInlinedCaller:
</ins><span class="cx"> case ConstructForwardVarargs:
</span><span class="cx"> emitCall(node);
</span><span class="cx"> break;
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoredfgDFGSpeculativeJIT64cpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/dfg/DFGSpeculativeJIT64.cpp (190219 => 190220)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/dfg/DFGSpeculativeJIT64.cpp        2015-09-24 21:39:36 UTC (rev 190219)
+++ trunk/Source/JavaScriptCore/dfg/DFGSpeculativeJIT64.cpp        2015-09-24 21:42:59 UTC (rev 190220)
</span><span class="lines">@@ -29,6 +29,7 @@
</span><span class="cx"> #if ENABLE(DFG_JIT)
</span><span class="cx">
</span><span class="cx"> #include "ArrayPrototype.h"
</span><ins>+#include "CallFrameShuffler.h"
</ins><span class="cx"> #include "DFGAbstractInterpreterInlines.h"
</span><span class="cx"> #include "DFGCallArrayAllocatorSlowPathGenerator.h"
</span><span class="cx"> #include "DFGOperations.h"
</span><span class="lines">@@ -613,10 +614,20 @@
</span><span class="cx"> CallLinkInfo::CallType callType;
</span><span class="cx"> bool isVarargs = false;
</span><span class="cx"> bool isForwardVarargs = false;
</span><ins>+ bool isTail = false;
+ bool isEmulatedTail = false;
</ins><span class="cx"> switch (node->op()) {
</span><span class="cx"> case Call:
</span><span class="cx"> callType = CallLinkInfo::Call;
</span><span class="cx"> break;
</span><ins>+ case TailCall:
+ callType = CallLinkInfo::TailCall;
+ isTail = true;
+ break;
+ case TailCallInlinedCaller:
+ callType = CallLinkInfo::Call;
+ isEmulatedTail = true;
+ break;
</ins><span class="cx"> case Construct:
</span><span class="cx"> callType = CallLinkInfo::Construct;
</span><span class="cx"> break;
</span><span class="lines">@@ -624,6 +635,16 @@
</span><span class="cx"> callType = CallLinkInfo::CallVarargs;
</span><span class="cx"> isVarargs = true;
</span><span class="cx"> break;
</span><ins>+ case TailCallVarargs:
+ callType = CallLinkInfo::TailCallVarargs;
+ isVarargs = true;
+ isTail = true;
+ break;
+ case TailCallVarargsInlinedCaller:
+ callType = CallLinkInfo::CallVarargs;
+ isVarargs = true;
+ isEmulatedTail = true;
+ break;
</ins><span class="cx"> case ConstructVarargs:
</span><span class="cx"> callType = CallLinkInfo::ConstructVarargs;
</span><span class="cx"> isVarargs = true;
</span><span class="lines">@@ -636,12 +657,23 @@
</span><span class="cx"> callType = CallLinkInfo::ConstructVarargs;
</span><span class="cx"> isForwardVarargs = true;
</span><span class="cx"> break;
</span><ins>+ case TailCallForwardVarargs:
+ callType = CallLinkInfo::TailCallVarargs;
+ isTail = true;
+ isForwardVarargs = true;
+ break;
+ case TailCallForwardVarargsInlinedCaller:
+ callType = CallLinkInfo::CallVarargs;
+ isEmulatedTail = true;
+ isForwardVarargs = true;
+ break;
</ins><span class="cx"> default:
</span><span class="cx"> DFG_CRASH(m_jit.graph(), node, "bad node type");
</span><span class="cx"> break;
</span><span class="cx"> }
</span><span class="cx">
</span><del>- Edge calleeEdge = m_jit.graph().child(node, 0);
</del><ins>+ GPRReg calleeGPR;
+ CallFrameShuffleData shuffleData;
</ins><span class="cx">
</span><span class="cx"> // Gotta load the arguments somehow. Varargs is trickier.
</span><span class="cx"> if (isVarargs || isForwardVarargs) {
</span><span class="lines">@@ -732,60 +764,112 @@
</span><span class="cx"> int numPassedArgs = node->numChildren() - 1;
</span><span class="cx">
</span><span class="cx"> m_jit.store32(MacroAssembler::TrustedImm32(numPassedArgs), JITCompiler::calleeFramePayloadSlot(JSStack::ArgumentCount));
</span><del>-
- for (int i = 0; i < numPassedArgs; i++) {
- Edge argEdge = m_jit.graph().m_varArgChildren[node->firstChild() + 1 + i];
- JSValueOperand arg(this, argEdge);
- GPRReg argGPR = arg.gpr();
- use(argEdge);
-
- m_jit.store64(argGPR, JITCompiler::calleeArgumentSlot(i));
</del><ins>+
+ if (node->op() == TailCall) {
+ Edge calleeEdge = m_jit.graph().child(node, 0);
+ JSValueOperand callee(this, calleeEdge);
+ calleeGPR = callee.gpr();
+ callee.use();
+
+ shuffleData.numLocals = m_jit.graph().frameRegisterCount();
+ shuffleData.callee = ValueRecovery::inGPR(calleeGPR, DataFormatJS);
+ shuffleData.args.resize(numPassedArgs);
+
+ for (int i = 0; i < numPassedArgs; ++i) {
+ Edge argEdge = m_jit.graph().varArgChild(node, i + 1);
+ GenerationInfo& info = generationInfo(argEdge.node());
+ use(argEdge);
+ shuffleData.args[i] = info.recovery(argEdge->virtualRegister());
+ }
+
+ shuffleData.setupCalleeSaveRegisters(m_jit.codeBlock());
+ } else {
+ m_jit.store32(MacroAssembler::TrustedImm32(numPassedArgs), JITCompiler::calleeFramePayloadSlot(JSStack::ArgumentCount));
+
+ for (int i = 0; i < numPassedArgs; i++) {
+ Edge argEdge = m_jit.graph().m_varArgChildren[node->firstChild() + 1 + i];
+ JSValueOperand arg(this, argEdge);
+ GPRReg argGPR = arg.gpr();
+ use(argEdge);
+
+ m_jit.store64(argGPR, JITCompiler::calleeArgumentSlot(i));
+ }
</ins><span class="cx"> }
</span><span class="cx"> }
</span><span class="cx">
</span><del>- JSValueOperand callee(this, calleeEdge);
- GPRReg calleeGPR = callee.gpr();
- callee.use();
- m_jit.store64(calleeGPR, JITCompiler::calleeFrameSlot(JSStack::Callee));
-
- flushRegisters();
</del><ins>+ if (node->op() != TailCall) {
+ Edge calleeEdge = m_jit.graph().child(node, 0);
+ JSValueOperand callee(this, calleeEdge);
+ calleeGPR = callee.gpr();
+ callee.use();
+ m_jit.store64(calleeGPR, JITCompiler::calleeFrameSlot(JSStack::Callee));
</ins><span class="cx">
</span><del>- GPRFlushedCallResult result(this);
- GPRReg resultGPR = result.gpr();
</del><ins>+ flushRegisters();
+ }
</ins><span class="cx">
</span><del>- JITCompiler::DataLabelPtr targetToCheck;
- JITCompiler::Jump slowPath;
</del><ins>+ CodeOrigin staticOrigin = node->origin.semantic;
+ ASSERT(!isTail || !staticOrigin.inlineCallFrame || !staticOrigin.inlineCallFrame->getCallerSkippingDeadFrames());
+ ASSERT(!isEmulatedTail || (staticOrigin.inlineCallFrame && staticOrigin.inlineCallFrame->getCallerSkippingDeadFrames()));
+ CodeOrigin dynamicOrigin =
+ isEmulatedTail ? *staticOrigin.inlineCallFrame->getCallerSkippingDeadFrames() : staticOrigin;
</ins><span class="cx">
</span><del>- CallSiteIndex callSite = m_jit.recordCallSiteAndGenerateExceptionHandlingOSRExitIfNeeded(node->origin.semantic, m_stream->size());
</del><ins>+ CallSiteIndex callSite = m_jit.recordCallSiteAndGenerateExceptionHandlingOSRExitIfNeeded(dynamicOrigin, m_stream->size());
</ins><span class="cx"> m_jit.emitStoreCallSiteIndex(callSite);
</span><span class="cx">
</span><span class="cx"> CallLinkInfo* callLinkInfo = m_jit.codeBlock()->addCallLinkInfo();
</span><del>-
- slowPath = m_jit.branchPtrWithPatch(MacroAssembler::NotEqual, calleeGPR, targetToCheck, MacroAssembler::TrustedImmPtr(0));
</del><span class="cx">
</span><del>- JITCompiler::Call fastCall = m_jit.nearCall();
</del><ins>+ JITCompiler::DataLabelPtr targetToCheck;
+ JITCompiler::Jump slowPath = m_jit.branchPtrWithPatch(MacroAssembler::NotEqual, calleeGPR, targetToCheck, MacroAssembler::TrustedImmPtr(0));
</ins><span class="cx">
</span><ins>+ if (isTail) {
+ if (node->op() == TailCall) {
+ callLinkInfo->setFrameShuffleData(shuffleData);
+ CallFrameShuffler(m_jit, shuffleData).prepareForTailCall();
+ } else {
+ m_jit.emitRestoreCalleeSaves();
+ m_jit.prepareForTailCallSlow();
+ }
+ }
+
+ JITCompiler::Call fastCall = isTail ? m_jit.nearTailCall() : m_jit.nearCall();
+
</ins><span class="cx"> JITCompiler::Jump done = m_jit.jump();
</span><del>-
</del><ins>+
</ins><span class="cx"> slowPath.link(&m_jit);
</span><del>-
- m_jit.move(calleeGPR, GPRInfo::regT0); // Callee needs to be in regT0
</del><ins>+
+ if (node->op() == TailCall) {
+ CallFrameShuffler callFrameShuffler(m_jit, shuffleData);
+ callFrameShuffler.setCalleeJSValueRegs(JSValueRegs(GPRInfo::regT0));
+ callFrameShuffler.prepareForSlowPath();
+ } else {
+ m_jit.move(calleeGPR, GPRInfo::regT0); // Callee needs to be in regT0
+
+ if (isTail)
+ m_jit.emitRestoreCalleeSaves(); // This needs to happen after we moved calleeGPR to regT0
+ }
+
</ins><span class="cx"> m_jit.move(MacroAssembler::TrustedImmPtr(callLinkInfo), GPRInfo::regT2); // Link info needs to be in regT2
</span><span class="cx"> JITCompiler::Call slowCall = m_jit.nearCall();
</span><del>-
</del><ins>+
</ins><span class="cx"> done.link(&m_jit);
</span><del>-
- m_jit.move(GPRInfo::returnValueGPR, resultGPR);
-
- jsValueResult(resultGPR, m_currentNode, DataFormatJS, UseChildrenCalledExplicitly);
-
</del><ins>+
+ if (isTail)
+ m_jit.abortWithReason(JITDidReturnFromTailCall);
+ else {
+ GPRFlushedCallResult result(this);
+ GPRReg resultGPR = result.gpr();
+ m_jit.move(GPRInfo::returnValueGPR, resultGPR);
+
+ jsValueResult(resultGPR, m_currentNode, DataFormatJS, UseChildrenCalledExplicitly);
+
+ // After the calls are done, we need to reestablish our stack
+ // pointer. We rely on this for varargs calls, calls with arity
+ // mismatch (the callframe is slided) and tail calls.
+ m_jit.addPtr(TrustedImm32(m_jit.graph().stackPointerOffset() * sizeof(Register)), GPRInfo::callFrameRegister, JITCompiler::stackPointerRegister);
+ }
+
</ins><span class="cx"> callLinkInfo->setUpCall(callType, m_currentNode->origin.semantic, calleeGPR);
</span><span class="cx"> m_jit.addJSCall(fastCall, slowCall, targetToCheck, callLinkInfo);
</span><del>-
- // After the calls are done, we need to reestablish our stack
- // pointer. We rely on this for varargs calls, calls with arity
- // mismatch (the callframe is slided) and tail calls.
- m_jit.addPtr(TrustedImm32(m_jit.graph().stackPointerOffset() * sizeof(Register)), GPRInfo::callFrameRegister, JITCompiler::stackPointerRegister);
</del><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> // Clang should allow unreachable [[clang::fallthrough]] in template functions if any template expansion uses it
</span><span class="lines">@@ -4248,14 +4332,20 @@
</span><span class="cx"> break;
</span><span class="cx">
</span><span class="cx"> case Call:
</span><ins>+ case TailCall:
+ case TailCallInlinedCaller:
</ins><span class="cx"> case Construct:
</span><span class="cx"> case CallVarargs:
</span><ins>+ case TailCallVarargs:
+ case TailCallVarargsInlinedCaller:
</ins><span class="cx"> case CallForwardVarargs:
</span><span class="cx"> case ConstructVarargs:
</span><span class="cx"> case ConstructForwardVarargs:
</span><ins>+ case TailCallForwardVarargs:
+ case TailCallForwardVarargsInlinedCaller:
</ins><span class="cx"> emitCall(node);
</span><span class="cx"> break;
</span><del>-
</del><ins>+
</ins><span class="cx"> case LoadVarargs: {
</span><span class="cx"> LoadVarargsData* data = node->loadVarargsData();
</span><span class="cx">
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoredfgDFGValidatecpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/dfg/DFGValidate.cpp (190219 => 190220)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/dfg/DFGValidate.cpp        2015-09-24 21:39:36 UTC (rev 190219)
+++ trunk/Source/JavaScriptCore/dfg/DFGValidate.cpp        2015-09-24 21:42:59 UTC (rev 190220)
</span><span class="lines">@@ -570,6 +570,8 @@
</span><span class="cx"> case Upsilon:
</span><span class="cx"> case ForwardVarargs:
</span><span class="cx"> case CallForwardVarargs:
</span><ins>+ case TailCallForwardVarargs:
+ case TailCallForwardVarargsInlinedCaller:
</ins><span class="cx"> case ConstructForwardVarargs:
</span><span class="cx"> case GetMyArgumentByVal:
</span><span class="cx"> break;
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoredfgDFGVarargsForwardingPhasecpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/dfg/DFGVarargsForwardingPhase.cpp (190219 => 190220)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/dfg/DFGVarargsForwardingPhase.cpp        2015-09-24 21:39:36 UTC (rev 190219)
+++ trunk/Source/JavaScriptCore/dfg/DFGVarargsForwardingPhase.cpp        2015-09-24 21:42:59 UTC (rev 190220)
</span><span class="lines">@@ -135,6 +135,8 @@
</span><span class="cx">
</span><span class="cx"> case CallVarargs:
</span><span class="cx"> case ConstructVarargs:
</span><ins>+ case TailCallVarargs:
+ case TailCallVarargsInlinedCaller:
</ins><span class="cx"> if (node->child1() == candidate || node->child3() == candidate) {
</span><span class="cx"> if (verbose)
</span><span class="cx"> dataLog(" Escape at ", node, "\n");
</span><span class="lines">@@ -282,7 +284,19 @@
</span><span class="cx"> break;
</span><span class="cx"> node->setOpAndDefaultFlags(ConstructForwardVarargs);
</span><span class="cx"> break;
</span><del>-
</del><ins>+
+ case TailCallVarargs:
+ if (node->child2() != candidate)
+ break;
+ node->setOpAndDefaultFlags(TailCallForwardVarargs);
+ break;
+
+ case TailCallVarargsInlinedCaller:
+ if (node->child2() != candidate)
+ break;
+ node->setOpAndDefaultFlags(TailCallForwardVarargsInlinedCaller);
+ break;
+
</ins><span class="cx"> case SetLocal:
</span><span class="cx"> // This is super odd. We don't have to do anything here, since in DFG IR, the phantom
</span><span class="cx"> // arguments nodes do produce a JSValue. Also, we know that if this SetLocal referenecs a
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreinterpreterCallFramecpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/interpreter/CallFrame.cpp (190219 => 190220)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/interpreter/CallFrame.cpp        2015-09-24 21:39:36 UTC (rev 190219)
+++ trunk/Source/JavaScriptCore/interpreter/CallFrame.cpp        2015-09-24 21:42:59 UTC (rev 190220)
</span><span class="lines">@@ -95,7 +95,7 @@
</span><span class="cx"> ASSERT(codeBlock());
</span><span class="cx"> CodeOrigin codeOrigin = this->codeOrigin();
</span><span class="cx"> for (InlineCallFrame* inlineCallFrame = codeOrigin.inlineCallFrame; inlineCallFrame;) {
</span><del>- codeOrigin = inlineCallFrame->caller;
</del><ins>+ codeOrigin = inlineCallFrame->directCaller;
</ins><span class="cx"> inlineCallFrame = codeOrigin.inlineCallFrame;
</span><span class="cx"> }
</span><span class="cx"> return codeOrigin.bytecodeIndex;
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreinterpreterStackVisitorcpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/interpreter/StackVisitor.cpp (190219 => 190220)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/interpreter/StackVisitor.cpp        2015-09-24 21:39:36 UTC (rev 190219)
+++ trunk/Source/JavaScriptCore/interpreter/StackVisitor.cpp        2015-09-24 21:42:59 UTC (rev 190220)
</span><span class="lines">@@ -60,8 +60,16 @@
</span><span class="cx"> #if ENABLE(DFG_JIT)
</span><span class="cx"> if (m_frame.isInlinedFrame()) {
</span><span class="cx"> InlineCallFrame* inlineCallFrame = m_frame.inlineCallFrame();
</span><del>- CodeOrigin* callerCodeOrigin = &inlineCallFrame->caller;
- readInlinedFrame(m_frame.callFrame(), callerCodeOrigin);
</del><ins>+ CodeOrigin* callerCodeOrigin = inlineCallFrame->getCallerSkippingDeadFrames();
+ if (!callerCodeOrigin) {
+ while (inlineCallFrame) {
+ readInlinedFrame(m_frame.callFrame(), &inlineCallFrame->directCaller);
+ inlineCallFrame = m_frame.inlineCallFrame();
+ }
+ m_frame.m_VMEntryFrame = m_frame.m_CallerVMEntryFrame;
+ readFrame(m_frame.callerFrame());
+ } else
+ readInlinedFrame(m_frame.callFrame(), callerCodeOrigin);
</ins><span class="cx"> return;
</span><span class="cx"> }
</span><span class="cx"> #endif // ENABLE(DFG_JIT)
</span></span></pre>
</div>
</div>
</body>
</html>