<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
"http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head><meta http-equiv="content-type" content="text/html; charset=utf-8" />
<title>[207475] trunk</title>
</head>
<body>
<style type="text/css"><!--
#msg dl.meta { border: 1px #006 solid; background: #369; padding: 6px; color: #fff; }
#msg dl.meta dt { float: left; width: 6em; font-weight: bold; }
#msg dt:after { content:':';}
#msg dl, #msg dt, #msg ul, #msg li, #header, #footer, #logmsg { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt; }
#msg dl a { font-weight: bold}
#msg dl a:link { color:#fc3; }
#msg dl a:active { color:#ff0; }
#msg dl a:visited { color:#cc6; }
h3 { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt; font-weight: bold; }
#msg pre { overflow: auto; background: #ffc; border: 1px #fa0 solid; padding: 6px; }
#logmsg { background: #ffc; border: 1px #fa0 solid; padding: 1em 1em 0 1em; }
#logmsg p, #logmsg pre, #logmsg blockquote { margin: 0 0 1em 0; }
#logmsg p, #logmsg li, #logmsg dt, #logmsg dd { line-height: 14pt; }
#logmsg h1, #logmsg h2, #logmsg h3, #logmsg h4, #logmsg h5, #logmsg h6 { margin: .5em 0; }
#logmsg h1:first-child, #logmsg h2:first-child, #logmsg h3:first-child, #logmsg h4:first-child, #logmsg h5:first-child, #logmsg h6:first-child { margin-top: 0; }
#logmsg ul, #logmsg ol { padding: 0; list-style-position: inside; margin: 0 0 0 1em; }
#logmsg ul { text-indent: -1em; padding-left: 1em; }#logmsg ol { text-indent: -1.5em; padding-left: 1.5em; }
#logmsg > ul, #logmsg > ol { margin: 0 0 1em 0; }
#logmsg pre { background: #eee; padding: 1em; }
#logmsg blockquote { border: 1px solid #fa0; border-left-width: 10px; padding: 1em 1em 0 1em; background: white;}
#logmsg dl { margin: 0; }
#logmsg dt { font-weight: bold; }
#logmsg dd { margin: 0; padding: 0 0 0.5em 0; }
#logmsg dd:before { content:'\00bb';}
#logmsg table { border-spacing: 0px; border-collapse: collapse; border-top: 4px solid #fa0; border-bottom: 1px solid #fa0; background: #fff; }
#logmsg table th { text-align: left; font-weight: normal; padding: 0.2em 0.5em; border-top: 1px dotted #fa0; }
#logmsg table td { text-align: right; border-top: 1px dotted #fa0; padding: 0.2em 0.5em; }
#logmsg table thead th { text-align: center; border-bottom: 1px solid #fa0; }
#logmsg table th.Corner { text-align: left; }
#logmsg hr { border: none 0; border-top: 2px dashed #fa0; height: 1px; }
#header, #footer { color: #fff; background: #636; border: 1px #300 solid; padding: 6px; }
#patch { width: 100%; }
#patch h4 {font-family: verdana,arial,helvetica,sans-serif;font-size:10pt;padding:8px;background:#369;color:#fff;margin:0;}
#patch .propset h4, #patch .binary h4 {margin:0;}
#patch pre {padding:0;line-height:1.2em;margin:0;}
#patch .diff {width:100%;background:#eee;padding: 0 0 10px 0;overflow:auto;}
#patch .propset .diff, #patch .binary .diff {padding:10px 0;}
#patch span {display:block;padding:0 10px;}
#patch .modfile, #patch .addfile, #patch .delfile, #patch .propset, #patch .binary, #patch .copfile {border:1px solid #ccc;margin:10px 0;}
#patch ins {background:#dfd;text-decoration:none;display:block;padding:0 10px;}
#patch del {background:#fdd;text-decoration:none;display:block;padding:0 10px;}
#patch .lines, .info {color:#888;background:#fff;}
--></style>
<div id="msg">
<dl class="meta">
<dt>Revision</dt> <dd><a href="http://trac.webkit.org/projects/webkit/changeset/207475">207475</a></dd>
<dt>Author</dt> <dd>fpizlo@apple.com</dd>
<dt>Date</dt> <dd>2016-10-18 11:30:05 -0700 (Tue, 18 Oct 2016)</dd>
</dl>
<h3>Log Message</h3>
<pre>DFG and FTL should be able to use DirectCall ICs when they proved the callee or its executable
https://bugs.webkit.org/show_bug.cgi?id=163371
Reviewed by Geoffrey Garen and Saam Barati.
JSTests:
Add microbenchmarks for all of the cases that this patch optimizes.
* microbenchmarks/direct-call-arity-mismatch.js: Added.
(foo):
(bar):
* microbenchmarks/direct-call.js: Added.
(foo):
(bar):
* microbenchmarks/direct-construct-arity-mismatch.js: Added.
(Foo):
(bar):
* microbenchmarks/direct-construct.js: Added.
(Foo):
(bar):
* microbenchmarks/direct-tail-call-arity-mismatch.js: Added.
(foo):
(bar):
* microbenchmarks/direct-tail-call-inlined-caller-arity-mismatch.js: Added.
(foo):
(bar):
(baz):
* microbenchmarks/direct-tail-call-inlined-caller.js: Added.
(foo):
(bar):
(baz):
* microbenchmarks/direct-tail-call.js: Added.
(foo):
(bar):
Source/JavaScriptCore:
This adds a new kind of call inline cache for when the DFG can prove what the callee
executable is. In those cases, we can skip some of the things that the traditional call IC
would do:
- No need to check who the callee is.
- No need to do arity checks.
This case isn't as simple as just emitting a call instruction since the callee may not be
compiled at the time that the caller is compiled. So, we need lazy resolution. Also, the
callee may be jettisoned independently of the caller, so we need to be able to revert the
call to an unlinked state. This means that we need almost all of the things that
CallLinkInfo has. CallLinkInfo already knows about different kinds of calls. This patch
teaches it about new "Direct" call types.
The direct non-tail call IC looks like this:
set up arguments
FastPath:
call _SlowPath
lea -FrameSize(%rbp), %rsp
SlowPath:
pop
call operationLinkDirectCall
check exception
jmp FastPath
The job of operationLinkDirectCall is to link the fast path's call entrypoint of the callee.
This means that in steady state, a call is just that: a call. There are no extra branches or
checks.
The direct tail call IC is a bit more complicated because the act of setting up arguments
destroys our frame, which would prevent us from being able to throw an exception if we
failed to compile the callee. So, direct tail call ICs look like this:
jmp _SlowPath
FastPath:
set up arguments
jmp 0 // patch to jump to callee
SlowPath:
silent spill
call operationLinkDirectCall
silent fill
check exception
jmp FastPath
The jmp to the slow path is patched to be a fall-through jmp when we link the call.
Direct calls mean less code at call sites, fewer checks on the steady state call fast path,
and no need for arity fixup. This looks like a slight speed-up (~0.8%) on both Octane and
AsmBench.
* assembler/ARM64Assembler.h:
(JSC::ARM64Assembler::relinkJumpToNop):
* assembler/ARMv7Assembler.h:
(JSC::ARMv7Assembler::relinkJumpToNop):
(JSC::ARMv7Assembler::relinkJump): Deleted.
* assembler/AbstractMacroAssembler.h:
(JSC::AbstractMacroAssembler::repatchJumpToNop):
(JSC::AbstractMacroAssembler::repatchJump): Deleted.
* assembler/X86Assembler.h:
(JSC::X86Assembler::relinkJumpToNop):
* bytecode/CallLinkInfo.cpp:
(JSC::CallLinkInfo::CallLinkInfo):
(JSC::CallLinkInfo::callReturnLocation):
(JSC::CallLinkInfo::patchableJump):
(JSC::CallLinkInfo::hotPathBegin):
(JSC::CallLinkInfo::slowPathStart):
(JSC::CallLinkInfo::setCallee):
(JSC::CallLinkInfo::clearCallee):
(JSC::CallLinkInfo::callee):
(JSC::CallLinkInfo::setCodeBlock):
(JSC::CallLinkInfo::clearCodeBlock):
(JSC::CallLinkInfo::codeBlock):
(JSC::CallLinkInfo::setLastSeenCallee):
(JSC::CallLinkInfo::clearLastSeenCallee):
(JSC::CallLinkInfo::lastSeenCallee):
(JSC::CallLinkInfo::haveLastSeenCallee):
(JSC::CallLinkInfo::setExecutableDuringCompilation):
(JSC::CallLinkInfo::executable):
(JSC::CallLinkInfo::setMaxNumArguments):
(JSC::CallLinkInfo::visitWeak):
* bytecode/CallLinkInfo.h:
(JSC::CallLinkInfo::specializationKindFor):
(JSC::CallLinkInfo::callModeFor):
(JSC::CallLinkInfo::isDirect):
(JSC::CallLinkInfo::nearCallMode):
(JSC::CallLinkInfo::isLinked):
(JSC::CallLinkInfo::setCallLocations):
(JSC::CallLinkInfo::addressOfMaxNumArguments):
(JSC::CallLinkInfo::maxNumArguments):
(JSC::CallLinkInfo::isTailCall): Deleted.
(JSC::CallLinkInfo::setUpCallFromFTL): Deleted.
(JSC::CallLinkInfo::callReturnLocation): Deleted.
(JSC::CallLinkInfo::hotPathBegin): Deleted.
(JSC::CallLinkInfo::callee): Deleted.
(JSC::CallLinkInfo::setLastSeenCallee): Deleted.
(JSC::CallLinkInfo::clearLastSeenCallee): Deleted.
(JSC::CallLinkInfo::lastSeenCallee): Deleted.
(JSC::CallLinkInfo::haveLastSeenCallee): Deleted.
* bytecode/CallLinkStatus.cpp:
(JSC::CallLinkStatus::computeDFGStatuses):
* bytecode/PolymorphicAccess.cpp:
(JSC::AccessCase::generateImpl):
* bytecode/UnlinkedFunctionExecutable.h:
* bytecode/ValueRecovery.h:
(JSC::ValueRecovery::forEachReg):
* dfg/DFGAbstractInterpreterInlines.h:
(JSC::DFG::AbstractInterpreter<AbstractStateType>::executeEffects):
* dfg/DFGBasicBlock.h:
(JSC::DFG::BasicBlock::findTerminal):
* dfg/DFGByteCodeParser.cpp:
(JSC::DFG::ByteCodeParser::addCallWithoutSettingResult):
(JSC::DFG::ByteCodeParser::handleCall):
* dfg/DFGClobberize.h:
(JSC::DFG::clobberize):
* dfg/DFGDoesGC.cpp:
(JSC::DFG::doesGC):
* dfg/DFGFixupPhase.cpp:
(JSC::DFG::FixupPhase::fixupNode):
* dfg/DFGGraph.cpp:
(JSC::DFG::Graph::parameterSlotsForArgCount):
* dfg/DFGGraph.h:
* dfg/DFGInPlaceAbstractState.cpp:
(JSC::DFG::InPlaceAbstractState::mergeToSuccessors):
* dfg/DFGJITCompiler.cpp:
(JSC::DFG::JITCompiler::link):
* dfg/DFGJITCompiler.h:
(JSC::DFG::JITCompiler::addJSDirectCall):
(JSC::DFG::JITCompiler::addJSDirectTailCall):
(JSC::DFG::JITCompiler::JSCallRecord::JSCallRecord):
(JSC::DFG::JITCompiler::JSDirectCallRecord::JSDirectCallRecord):
(JSC::DFG::JITCompiler::JSDirectTailCallRecord::JSDirectTailCallRecord):
(JSC::DFG::JITCompiler::currentJSCallIndex): Deleted.
* dfg/DFGNode.cpp:
(JSC::DFG::Node::convertToDirectCall):
* dfg/DFGNode.h:
(JSC::DFG::Node::isTerminal):
(JSC::DFG::Node::hasHeapPrediction):
(JSC::DFG::Node::hasCellOperand):
* dfg/DFGNodeType.h:
* dfg/DFGPredictionPropagationPhase.cpp:
* dfg/DFGSafeToExecute.h:
(JSC::DFG::safeToExecute):
* dfg/DFGSpeculativeJIT.h:
(JSC::DFG::SpeculativeJIT::callOperation):
* dfg/DFGSpeculativeJIT64.cpp:
(JSC::DFG::SpeculativeJIT::emitCall):
(JSC::DFG::SpeculativeJIT::compile):
* dfg/DFGStrengthReductionPhase.cpp:
(JSC::DFG::StrengthReductionPhase::handleNode):
* ftl/FTLCapabilities.cpp:
(JSC::FTL::canCompile):
* ftl/FTLLowerDFGToB3.cpp:
(JSC::FTL::DFG::LowerDFGToB3::compileNode):
(JSC::FTL::DFG::LowerDFGToB3::compileCallOrConstruct):
(JSC::FTL::DFG::LowerDFGToB3::compileDirectCallOrConstruct):
(JSC::FTL::DFG::LowerDFGToB3::compileTailCall):
(JSC::FTL::DFG::LowerDFGToB3::compileCallOrConstructVarargs):
* interpreter/Interpreter.cpp:
(JSC::Interpreter::execute):
(JSC::Interpreter::executeCall):
(JSC::Interpreter::executeConstruct):
(JSC::Interpreter::prepareForRepeatCall):
* jit/JIT.cpp:
(JSC::JIT::link):
* jit/JITCall.cpp:
(JSC::JIT::compileSetupVarargsFrame):
* jit/JITCall32_64.cpp:
(JSC::JIT::compileSetupVarargsFrame):
* jit/JITOperations.cpp:
* jit/JITOperations.h:
* jit/Repatch.cpp:
(JSC::linkDirectFor):
(JSC::revertCall):
* jit/Repatch.h:
* llint/LLIntSlowPaths.cpp:
(JSC::LLInt::setUpCall):
* runtime/Executable.cpp:
(JSC::ScriptExecutable::prepareForExecutionImpl):
* runtime/Executable.h:
(JSC::ScriptExecutable::prepareForExecution):
* runtime/Options.h:</pre>
<h3>Modified Paths</h3>
<ul>
<li><a href="#trunkJSTestsChangeLog">trunk/JSTests/ChangeLog</a></li>
<li><a href="#trunkSourceJavaScriptCoreChangeLog">trunk/Source/JavaScriptCore/ChangeLog</a></li>
<li><a href="#trunkSourceJavaScriptCoreassemblerARM64Assemblerh">trunk/Source/JavaScriptCore/assembler/ARM64Assembler.h</a></li>
<li><a href="#trunkSourceJavaScriptCoreassemblerARMv7Assemblerh">trunk/Source/JavaScriptCore/assembler/ARMv7Assembler.h</a></li>
<li><a href="#trunkSourceJavaScriptCoreassemblerAbstractMacroAssemblerh">trunk/Source/JavaScriptCore/assembler/AbstractMacroAssembler.h</a></li>
<li><a href="#trunkSourceJavaScriptCoreassemblerX86Assemblerh">trunk/Source/JavaScriptCore/assembler/X86Assembler.h</a></li>
<li><a href="#trunkSourceJavaScriptCorebytecodeCallLinkInfocpp">trunk/Source/JavaScriptCore/bytecode/CallLinkInfo.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCorebytecodeCallLinkInfoh">trunk/Source/JavaScriptCore/bytecode/CallLinkInfo.h</a></li>
<li><a href="#trunkSourceJavaScriptCorebytecodeCallLinkStatuscpp">trunk/Source/JavaScriptCore/bytecode/CallLinkStatus.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCorebytecodePolymorphicAccesscpp">trunk/Source/JavaScriptCore/bytecode/PolymorphicAccess.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCorebytecodeUnlinkedFunctionExecutableh">trunk/Source/JavaScriptCore/bytecode/UnlinkedFunctionExecutable.h</a></li>
<li><a href="#trunkSourceJavaScriptCorebytecodeValueRecoveryh">trunk/Source/JavaScriptCore/bytecode/ValueRecovery.h</a></li>
<li><a href="#trunkSourceJavaScriptCoredfgDFGAbstractInterpreterInlinesh">trunk/Source/JavaScriptCore/dfg/DFGAbstractInterpreterInlines.h</a></li>
<li><a href="#trunkSourceJavaScriptCoredfgDFGBasicBlockh">trunk/Source/JavaScriptCore/dfg/DFGBasicBlock.h</a></li>
<li><a href="#trunkSourceJavaScriptCoredfgDFGByteCodeParsercpp">trunk/Source/JavaScriptCore/dfg/DFGByteCodeParser.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoredfgDFGClobberizeh">trunk/Source/JavaScriptCore/dfg/DFGClobberize.h</a></li>
<li><a href="#trunkSourceJavaScriptCoredfgDFGDoesGCcpp">trunk/Source/JavaScriptCore/dfg/DFGDoesGC.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoredfgDFGFixupPhasecpp">trunk/Source/JavaScriptCore/dfg/DFGFixupPhase.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoredfgDFGGraphcpp">trunk/Source/JavaScriptCore/dfg/DFGGraph.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoredfgDFGGraphh">trunk/Source/JavaScriptCore/dfg/DFGGraph.h</a></li>
<li><a href="#trunkSourceJavaScriptCoredfgDFGInPlaceAbstractStatecpp">trunk/Source/JavaScriptCore/dfg/DFGInPlaceAbstractState.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoredfgDFGJITCompilercpp">trunk/Source/JavaScriptCore/dfg/DFGJITCompiler.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoredfgDFGJITCompilerh">trunk/Source/JavaScriptCore/dfg/DFGJITCompiler.h</a></li>
<li><a href="#trunkSourceJavaScriptCoredfgDFGNodecpp">trunk/Source/JavaScriptCore/dfg/DFGNode.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoredfgDFGNodeh">trunk/Source/JavaScriptCore/dfg/DFGNode.h</a></li>
<li><a href="#trunkSourceJavaScriptCoredfgDFGNodeTypeh">trunk/Source/JavaScriptCore/dfg/DFGNodeType.h</a></li>
<li><a href="#trunkSourceJavaScriptCoredfgDFGPredictionPropagationPhasecpp">trunk/Source/JavaScriptCore/dfg/DFGPredictionPropagationPhase.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoredfgDFGSafeToExecuteh">trunk/Source/JavaScriptCore/dfg/DFGSafeToExecute.h</a></li>
<li><a href="#trunkSourceJavaScriptCoredfgDFGSpeculativeJITh">trunk/Source/JavaScriptCore/dfg/DFGSpeculativeJIT.h</a></li>
<li><a href="#trunkSourceJavaScriptCoredfgDFGSpeculativeJIT32_64cpp">trunk/Source/JavaScriptCore/dfg/DFGSpeculativeJIT32_64.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoredfgDFGSpeculativeJIT64cpp">trunk/Source/JavaScriptCore/dfg/DFGSpeculativeJIT64.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoredfgDFGStrengthReductionPhasecpp">trunk/Source/JavaScriptCore/dfg/DFGStrengthReductionPhase.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoreftlFTLCapabilitiescpp">trunk/Source/JavaScriptCore/ftl/FTLCapabilities.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoreftlFTLLowerDFGToB3cpp">trunk/Source/JavaScriptCore/ftl/FTLLowerDFGToB3.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoreinterpreterInterpretercpp">trunk/Source/JavaScriptCore/interpreter/Interpreter.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCorejitJITcpp">trunk/Source/JavaScriptCore/jit/JIT.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCorejitJITCallcpp">trunk/Source/JavaScriptCore/jit/JITCall.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCorejitJITCall32_64cpp">trunk/Source/JavaScriptCore/jit/JITCall32_64.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCorejitJITOperationscpp">trunk/Source/JavaScriptCore/jit/JITOperations.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCorejitJITOperationsh">trunk/Source/JavaScriptCore/jit/JITOperations.h</a></li>
<li><a href="#trunkSourceJavaScriptCorejitRepatchcpp">trunk/Source/JavaScriptCore/jit/Repatch.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCorejitRepatchh">trunk/Source/JavaScriptCore/jit/Repatch.h</a></li>
<li><a href="#trunkSourceJavaScriptCorellintLLIntSlowPathscpp">trunk/Source/JavaScriptCore/llint/LLIntSlowPaths.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoreruntimeExecutablecpp">trunk/Source/JavaScriptCore/runtime/Executable.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoreruntimeExecutableh">trunk/Source/JavaScriptCore/runtime/Executable.h</a></li>
<li><a href="#trunkSourceJavaScriptCoreruntimeOptionsh">trunk/Source/JavaScriptCore/runtime/Options.h</a></li>
</ul>
<h3>Added Paths</h3>
<ul>
<li><a href="#trunkJSTestsmicrobenchmarksdirectcallaritymismatchjs">trunk/JSTests/microbenchmarks/direct-call-arity-mismatch.js</a></li>
<li><a href="#trunkJSTestsmicrobenchmarksdirectcalljs">trunk/JSTests/microbenchmarks/direct-call.js</a></li>
<li><a href="#trunkJSTestsmicrobenchmarksdirectconstructaritymismatchjs">trunk/JSTests/microbenchmarks/direct-construct-arity-mismatch.js</a></li>
<li><a href="#trunkJSTestsmicrobenchmarksdirectconstructjs">trunk/JSTests/microbenchmarks/direct-construct.js</a></li>
<li><a href="#trunkJSTestsmicrobenchmarksdirecttailcallaritymismatchjs">trunk/JSTests/microbenchmarks/direct-tail-call-arity-mismatch.js</a></li>
<li><a href="#trunkJSTestsmicrobenchmarksdirecttailcallinlinedcalleraritymismatchjs">trunk/JSTests/microbenchmarks/direct-tail-call-inlined-caller-arity-mismatch.js</a></li>
<li><a href="#trunkJSTestsmicrobenchmarksdirecttailcallinlinedcallerjs">trunk/JSTests/microbenchmarks/direct-tail-call-inlined-caller.js</a></li>
<li><a href="#trunkJSTestsmicrobenchmarksdirecttailcalljs">trunk/JSTests/microbenchmarks/direct-tail-call.js</a></li>
</ul>
</div>
<div id="patch">
<h3>Diff</h3>
<a id="trunkJSTestsChangeLog"></a>
<div class="modfile"><h4>Modified: trunk/JSTests/ChangeLog (207474 => 207475)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/JSTests/ChangeLog 2016-10-18 18:23:56 UTC (rev 207474)
+++ trunk/JSTests/ChangeLog 2016-10-18 18:30:05 UTC (rev 207475)
</span><span class="lines">@@ -1,3 +1,39 @@
</span><ins>+2016-10-15 Filip Pizlo <fpizlo@apple.com>
+
+ DFG and FTL should be able to use DirectCall ICs when they proved the callee or its executable
+ https://bugs.webkit.org/show_bug.cgi?id=163371
+
+ Reviewed by Geoffrey Garen and Saam Barati.
+
+ Add microbenchmarks for all of the cases that this patch optimizes.
+
+ * microbenchmarks/direct-call-arity-mismatch.js: Added.
+ (foo):
+ (bar):
+ * microbenchmarks/direct-call.js: Added.
+ (foo):
+ (bar):
+ * microbenchmarks/direct-construct-arity-mismatch.js: Added.
+ (Foo):
+ (bar):
+ * microbenchmarks/direct-construct.js: Added.
+ (Foo):
+ (bar):
+ * microbenchmarks/direct-tail-call-arity-mismatch.js: Added.
+ (foo):
+ (bar):
+ * microbenchmarks/direct-tail-call-inlined-caller-arity-mismatch.js: Added.
+ (foo):
+ (bar):
+ (baz):
+ * microbenchmarks/direct-tail-call-inlined-caller.js: Added.
+ (foo):
+ (bar):
+ (baz):
+ * microbenchmarks/direct-tail-call.js: Added.
+ (foo):
+ (bar):
+
</ins><span class="cx"> 2016-10-18 Caitlin Potter <caitp@igalia.com>
</span><span class="cx">
</span><span class="cx"> [JSC] ES6 Method functions should not have prototype
</span></span></pre></div>
<a id="trunkJSTestsmicrobenchmarksdirectcallaritymismatchjs"></a>
<div class="addfile"><h4>Added: trunk/JSTests/microbenchmarks/direct-call-arity-mismatch.js (0 => 207475)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/JSTests/microbenchmarks/direct-call-arity-mismatch.js (rev 0)
+++ trunk/JSTests/microbenchmarks/direct-call-arity-mismatch.js 2016-10-18 18:30:05 UTC (rev 207475)
</span><span class="lines">@@ -0,0 +1,21 @@
</span><ins>+function foo(a, b, c, d, e, f, g)
+{
+ return 42;
+}
+
+function bar()
+{
+ return foo();
+}
+
+noInline(foo);
+noInline(bar);
+
+(function() {
+ for (var i = 0; i < 10000000; ++i) {
+ var result = bar();
+ if (result != 42)
+ throw "Error: bad result: " + result;
+ }
+})();
+
</ins></span></pre></div>
<a id="trunkJSTestsmicrobenchmarksdirectcalljs"></a>
<div class="addfile"><h4>Added: trunk/JSTests/microbenchmarks/direct-call.js (0 => 207475)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/JSTests/microbenchmarks/direct-call.js (rev 0)
+++ trunk/JSTests/microbenchmarks/direct-call.js 2016-10-18 18:30:05 UTC (rev 207475)
</span><span class="lines">@@ -0,0 +1,21 @@
</span><ins>+function foo()
+{
+ return 42;
+}
+
+function bar()
+{
+ return foo();
+}
+
+noInline(foo);
+noInline(bar);
+
+(function() {
+ for (var i = 0; i < 10000000; ++i) {
+ var result = bar();
+ if (result != 42)
+ throw "Error: bad result: " + result;
+ }
+})();
+
</ins></span></pre></div>
<a id="trunkJSTestsmicrobenchmarksdirectconstructaritymismatchjs"></a>
<div class="addfile"><h4>Added: trunk/JSTests/microbenchmarks/direct-construct-arity-mismatch.js (0 => 207475)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/JSTests/microbenchmarks/direct-construct-arity-mismatch.js (rev 0)
+++ trunk/JSTests/microbenchmarks/direct-construct-arity-mismatch.js 2016-10-18 18:30:05 UTC (rev 207475)
</span><span class="lines">@@ -0,0 +1,21 @@
</span><ins>+function Foo(a, b, c, d, e, f, g)
+{
+ this.f = 42;
+}
+
+function bar()
+{
+ return new Foo();
+}
+
+noInline(Foo);
+noInline(bar);
+
+(function() {
+ for (var i = 0; i < 10000000; ++i) {
+ var result = bar();
+ if (result.f != 42)
+ throw "Error: bad result: " + result;
+ }
+})();
+
</ins></span></pre></div>
<a id="trunkJSTestsmicrobenchmarksdirectconstructjs"></a>
<div class="addfile"><h4>Added: trunk/JSTests/microbenchmarks/direct-construct.js (0 => 207475)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/JSTests/microbenchmarks/direct-construct.js (rev 0)
+++ trunk/JSTests/microbenchmarks/direct-construct.js 2016-10-18 18:30:05 UTC (rev 207475)
</span><span class="lines">@@ -0,0 +1,21 @@
</span><ins>+function Foo()
+{
+ this.f = 42;
+}
+
+function bar()
+{
+ return new Foo();
+}
+
+noInline(Foo);
+noInline(bar);
+
+(function() {
+ for (var i = 0; i < 10000000; ++i) {
+ var result = bar();
+ if (result.f != 42)
+ throw "Error: bad result: " + result;
+ }
+})();
+
</ins></span></pre></div>
<a id="trunkJSTestsmicrobenchmarksdirecttailcallaritymismatchjs"></a>
<div class="addfile"><h4>Added: trunk/JSTests/microbenchmarks/direct-tail-call-arity-mismatch.js (0 => 207475)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/JSTests/microbenchmarks/direct-tail-call-arity-mismatch.js (rev 0)
+++ trunk/JSTests/microbenchmarks/direct-tail-call-arity-mismatch.js 2016-10-18 18:30:05 UTC (rev 207475)
</span><span class="lines">@@ -0,0 +1,23 @@
</span><ins>+"use strict";
+
+function foo(a, b, c, d, e, f, g)
+{
+ return 42;
+}
+
+function bar()
+{
+ return foo();
+}
+
+noInline(foo);
+noInline(bar);
+
+(function() {
+ for (var i = 0; i < 10000000; ++i) {
+ var result = bar();
+ if (result != 42)
+ throw "Error: bad result: " + result;
+ }
+})();
+
</ins></span></pre></div>
<a id="trunkJSTestsmicrobenchmarksdirecttailcallinlinedcalleraritymismatchjs"></a>
<div class="addfile"><h4>Added: trunk/JSTests/microbenchmarks/direct-tail-call-inlined-caller-arity-mismatch.js (0 => 207475)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/JSTests/microbenchmarks/direct-tail-call-inlined-caller-arity-mismatch.js (rev 0)
+++ trunk/JSTests/microbenchmarks/direct-tail-call-inlined-caller-arity-mismatch.js 2016-10-18 18:30:05 UTC (rev 207475)
</span><span class="lines">@@ -0,0 +1,28 @@
</span><ins>+"use strict";
+
+function foo(a, b, c, d, e, f, g)
+{
+ return 42;
+}
+
+function bar()
+{
+ return foo();
+}
+
+function baz()
+{
+ return bar() + 1;
+}
+
+noInline(foo);
+noInline(baz);
+
+(function() {
+ for (var i = 0; i < 10000000; ++i) {
+ var result = baz();
+ if (result != 43)
+ throw "Error: bad result: " + result;
+ }
+})();
+
</ins></span></pre></div>
<a id="trunkJSTestsmicrobenchmarksdirecttailcallinlinedcallerjs"></a>
<div class="addfile"><h4>Added: trunk/JSTests/microbenchmarks/direct-tail-call-inlined-caller.js (0 => 207475)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/JSTests/microbenchmarks/direct-tail-call-inlined-caller.js (rev 0)
+++ trunk/JSTests/microbenchmarks/direct-tail-call-inlined-caller.js 2016-10-18 18:30:05 UTC (rev 207475)
</span><span class="lines">@@ -0,0 +1,28 @@
</span><ins>+"use strict";
+
+function foo()
+{
+ return 42;
+}
+
+function bar()
+{
+ return foo();
+}
+
+function baz()
+{
+ return bar() + 1;
+}
+
+noInline(foo);
+noInline(baz);
+
+(function() {
+ for (var i = 0; i < 10000000; ++i) {
+ var result = baz();
+ if (result != 43)
+ throw "Error: bad result: " + result;
+ }
+})();
+
</ins></span></pre></div>
<a id="trunkJSTestsmicrobenchmarksdirecttailcalljs"></a>
<div class="addfile"><h4>Added: trunk/JSTests/microbenchmarks/direct-tail-call.js (0 => 207475)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/JSTests/microbenchmarks/direct-tail-call.js (rev 0)
+++ trunk/JSTests/microbenchmarks/direct-tail-call.js 2016-10-18 18:30:05 UTC (rev 207475)
</span><span class="lines">@@ -0,0 +1,23 @@
</span><ins>+"use strict";
+
+function foo()
+{
+ return 42;
+}
+
+function bar()
+{
+ return foo();
+}
+
+noInline(foo);
+noInline(bar);
+
+(function() {
+ for (var i = 0; i < 10000000; ++i) {
+ var result = bar();
+ if (result != 42)
+ throw "Error: bad result: " + result;
+ }
+})();
+
</ins></span></pre></div>
<a id="trunkSourceJavaScriptCoreChangeLog"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/ChangeLog (207474 => 207475)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/ChangeLog 2016-10-18 18:23:56 UTC (rev 207474)
+++ trunk/Source/JavaScriptCore/ChangeLog 2016-10-18 18:30:05 UTC (rev 207475)
</span><span class="lines">@@ -1,3 +1,195 @@
</span><ins>+2016-10-15 Filip Pizlo <fpizlo@apple.com>
+
+ DFG and FTL should be able to use DirectCall ICs when they proved the callee or its executable
+ https://bugs.webkit.org/show_bug.cgi?id=163371
+
+ Reviewed by Geoffrey Garen and Saam Barati.
+
+ This adds a new kind of call inline cache for when the DFG can prove what the callee
+ executable is. In those cases, we can skip some of the things that the traditional call IC
+ would do:
+
+ - No need to check who the callee is.
+ - No need to do arity checks.
+
+ This case isn't as simple as just emitting a call instruction since the callee may not be
+ compiled at the time that the caller is compiled. So, we need lazy resolution. Also, the
+ callee may be jettisoned independently of the caller, so we need to be able to revert the
+ call to an unlinked state. This means that we need almost all of the things that
+ CallLinkInfo has. CallLinkInfo already knows about different kinds of calls. This patch
+ teaches it about new "Direct" call types.
+
+ The direct non-tail call IC looks like this:
+
+ set up arguments
+ FastPath:
+ call _SlowPath
+ lea -FrameSize(%rbp), %rsp
+
+ SlowPath:
+ pop
+ call operationLinkDirectCall
+ check exception
+ jmp FastPath
+
+ The job of operationLinkDirectCall is to link the fast path's call entrypoint of the callee.
+ This means that in steady state, a call is just that: a call. There are no extra branches or
+ checks.
+
+ The direct tail call IC is a bit more complicated because the act of setting up arguments
+ destroys our frame, which would prevent us from being able to throw an exception if we
+ failed to compile the callee. So, direct tail call ICs look like this:
+
+ jmp _SlowPath
+ FastPath:
+ set up arguments
+ jmp 0 // patch to jump to callee
+
+ SlowPath:
+ silent spill
+ call operationLinkDirectCall
+ silent fill
+ check exception
+ jmp FastPath
+
+ The jmp to the slow path is patched to be a fall-through jmp when we link the call.
+
+ Direct calls mean less code at call sites, fewer checks on the steady state call fast path,
+ and no need for arity fixup. This looks like a slight speed-up (~0.8%) on both Octane and
+ AsmBench.
+
+ * assembler/ARM64Assembler.h:
+ (JSC::ARM64Assembler::relinkJumpToNop):
+ * assembler/ARMv7Assembler.h:
+ (JSC::ARMv7Assembler::relinkJumpToNop):
+ (JSC::ARMv7Assembler::relinkJump): Deleted.
+ * assembler/AbstractMacroAssembler.h:
+ (JSC::AbstractMacroAssembler::repatchJumpToNop):
+ (JSC::AbstractMacroAssembler::repatchJump): Deleted.
+ * assembler/X86Assembler.h:
+ (JSC::X86Assembler::relinkJumpToNop):
+ * bytecode/CallLinkInfo.cpp:
+ (JSC::CallLinkInfo::CallLinkInfo):
+ (JSC::CallLinkInfo::callReturnLocation):
+ (JSC::CallLinkInfo::patchableJump):
+ (JSC::CallLinkInfo::hotPathBegin):
+ (JSC::CallLinkInfo::slowPathStart):
+ (JSC::CallLinkInfo::setCallee):
+ (JSC::CallLinkInfo::clearCallee):
+ (JSC::CallLinkInfo::callee):
+ (JSC::CallLinkInfo::setCodeBlock):
+ (JSC::CallLinkInfo::clearCodeBlock):
+ (JSC::CallLinkInfo::codeBlock):
+ (JSC::CallLinkInfo::setLastSeenCallee):
+ (JSC::CallLinkInfo::clearLastSeenCallee):
+ (JSC::CallLinkInfo::lastSeenCallee):
+ (JSC::CallLinkInfo::haveLastSeenCallee):
+ (JSC::CallLinkInfo::setExecutableDuringCompilation):
+ (JSC::CallLinkInfo::executable):
+ (JSC::CallLinkInfo::setMaxNumArguments):
+ (JSC::CallLinkInfo::visitWeak):
+ * bytecode/CallLinkInfo.h:
+ (JSC::CallLinkInfo::specializationKindFor):
+ (JSC::CallLinkInfo::callModeFor):
+ (JSC::CallLinkInfo::isDirect):
+ (JSC::CallLinkInfo::nearCallMode):
+ (JSC::CallLinkInfo::isLinked):
+ (JSC::CallLinkInfo::setCallLocations):
+ (JSC::CallLinkInfo::addressOfMaxNumArguments):
+ (JSC::CallLinkInfo::maxNumArguments):
+ (JSC::CallLinkInfo::isTailCall): Deleted.
+ (JSC::CallLinkInfo::setUpCallFromFTL): Deleted.
+ (JSC::CallLinkInfo::callReturnLocation): Deleted.
+ (JSC::CallLinkInfo::hotPathBegin): Deleted.
+ (JSC::CallLinkInfo::callee): Deleted.
+ (JSC::CallLinkInfo::setLastSeenCallee): Deleted.
+ (JSC::CallLinkInfo::clearLastSeenCallee): Deleted.
+ (JSC::CallLinkInfo::lastSeenCallee): Deleted.
+ (JSC::CallLinkInfo::haveLastSeenCallee): Deleted.
+ * bytecode/CallLinkStatus.cpp:
+ (JSC::CallLinkStatus::computeDFGStatuses):
+ * bytecode/PolymorphicAccess.cpp:
+ (JSC::AccessCase::generateImpl):
+ * bytecode/UnlinkedFunctionExecutable.h:
+ * bytecode/ValueRecovery.h:
+ (JSC::ValueRecovery::forEachReg):
+ * dfg/DFGAbstractInterpreterInlines.h:
+ (JSC::DFG::AbstractInterpreter<AbstractStateType>::executeEffects):
+ * dfg/DFGBasicBlock.h:
+ (JSC::DFG::BasicBlock::findTerminal):
+ * dfg/DFGByteCodeParser.cpp:
+ (JSC::DFG::ByteCodeParser::addCallWithoutSettingResult):
+ (JSC::DFG::ByteCodeParser::handleCall):
+ * dfg/DFGClobberize.h:
+ (JSC::DFG::clobberize):
+ * dfg/DFGDoesGC.cpp:
+ (JSC::DFG::doesGC):
+ * dfg/DFGFixupPhase.cpp:
+ (JSC::DFG::FixupPhase::fixupNode):
+ * dfg/DFGGraph.cpp:
+ (JSC::DFG::Graph::parameterSlotsForArgCount):
+ * dfg/DFGGraph.h:
+ * dfg/DFGInPlaceAbstractState.cpp:
+ (JSC::DFG::InPlaceAbstractState::mergeToSuccessors):
+ * dfg/DFGJITCompiler.cpp:
+ (JSC::DFG::JITCompiler::link):
+ * dfg/DFGJITCompiler.h:
+ (JSC::DFG::JITCompiler::addJSDirectCall):
+ (JSC::DFG::JITCompiler::addJSDirectTailCall):
+ (JSC::DFG::JITCompiler::JSCallRecord::JSCallRecord):
+ (JSC::DFG::JITCompiler::JSDirectCallRecord::JSDirectCallRecord):
+ (JSC::DFG::JITCompiler::JSDirectTailCallRecord::JSDirectTailCallRecord):
+ (JSC::DFG::JITCompiler::currentJSCallIndex): Deleted.
+ * dfg/DFGNode.cpp:
+ (JSC::DFG::Node::convertToDirectCall):
+ * dfg/DFGNode.h:
+ (JSC::DFG::Node::isTerminal):
+ (JSC::DFG::Node::hasHeapPrediction):
+ (JSC::DFG::Node::hasCellOperand):
+ * dfg/DFGNodeType.h:
+ * dfg/DFGPredictionPropagationPhase.cpp:
+ * dfg/DFGSafeToExecute.h:
+ (JSC::DFG::safeToExecute):
+ * dfg/DFGSpeculativeJIT.h:
+ (JSC::DFG::SpeculativeJIT::callOperation):
+ * dfg/DFGSpeculativeJIT64.cpp:
+ (JSC::DFG::SpeculativeJIT::emitCall):
+ (JSC::DFG::SpeculativeJIT::compile):
+ * dfg/DFGStrengthReductionPhase.cpp:
+ (JSC::DFG::StrengthReductionPhase::handleNode):
+ * ftl/FTLCapabilities.cpp:
+ (JSC::FTL::canCompile):
+ * ftl/FTLLowerDFGToB3.cpp:
+ (JSC::FTL::DFG::LowerDFGToB3::compileNode):
+ (JSC::FTL::DFG::LowerDFGToB3::compileCallOrConstruct):
+ (JSC::FTL::DFG::LowerDFGToB3::compileDirectCallOrConstruct):
+ (JSC::FTL::DFG::LowerDFGToB3::compileTailCall):
+ (JSC::FTL::DFG::LowerDFGToB3::compileCallOrConstructVarargs):
+ * interpreter/Interpreter.cpp:
+ (JSC::Interpreter::execute):
+ (JSC::Interpreter::executeCall):
+ (JSC::Interpreter::executeConstruct):
+ (JSC::Interpreter::prepareForRepeatCall):
+ * jit/JIT.cpp:
+ (JSC::JIT::link):
+ * jit/JITCall.cpp:
+ (JSC::JIT::compileSetupVarargsFrame):
+ * jit/JITCall32_64.cpp:
+ (JSC::JIT::compileSetupVarargsFrame):
+ * jit/JITOperations.cpp:
+ * jit/JITOperations.h:
+ * jit/Repatch.cpp:
+ (JSC::linkDirectFor):
+ (JSC::revertCall):
+ * jit/Repatch.h:
+ * llint/LLIntSlowPaths.cpp:
+ (JSC::LLInt::setUpCall):
+ * runtime/Executable.cpp:
+ (JSC::ScriptExecutable::prepareForExecutionImpl):
+ * runtime/Executable.h:
+ (JSC::ScriptExecutable::prepareForExecution):
+ * runtime/Options.h:
+
</ins><span class="cx"> 2016-10-18 Yusuke Suzuki <utatane.tea@gmail.com>
</span><span class="cx">
</span><span class="cx"> [DOMJIT] Not emit exception case if it is not necessary
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreassemblerARM64Assemblerh"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/assembler/ARM64Assembler.h (207474 => 207475)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/assembler/ARM64Assembler.h 2016-10-18 18:23:56 UTC (rev 207474)
+++ trunk/Source/JavaScriptCore/assembler/ARM64Assembler.h 2016-10-18 18:30:05 UTC (rev 207475)
</span><span class="lines">@@ -2679,6 +2679,11 @@
</span><span class="cx"> cacheFlush(from, sizeof(int));
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+ static void relinkJumpToNop(void* from)
+ {
+ relinkJump(from, static_cast<char*>(from) + 4);
+ }
+
</ins><span class="cx"> static void relinkCall(void* from, void* to)
</span><span class="cx"> {
</span><span class="cx"> relinkJumpOrCall<true>(reinterpret_cast<int*>(from) - 1, reinterpret_cast<const int*>(from) - 1, to);
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreassemblerARMv7Assemblerh"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/assembler/ARMv7Assembler.h (207474 => 207475)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/assembler/ARMv7Assembler.h 2016-10-18 18:23:56 UTC (rev 207474)
+++ trunk/Source/JavaScriptCore/assembler/ARMv7Assembler.h 2016-10-18 18:30:05 UTC (rev 207475)
</span><span class="lines">@@ -2256,6 +2256,11 @@
</span><span class="cx">
</span><span class="cx"> cacheFlush(reinterpret_cast<uint16_t*>(from) - 5, 5 * sizeof(uint16_t));
</span><span class="cx"> }
</span><ins>+
+ static void relinkJumpToNop(void* from)
+ {
+ relinkJump(from, from);
+ }
</ins><span class="cx">
</span><span class="cx"> static void relinkCall(void* from, void* to)
</span><span class="cx"> {
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreassemblerAbstractMacroAssemblerh"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/assembler/AbstractMacroAssembler.h (207474 => 207475)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/assembler/AbstractMacroAssembler.h 2016-10-18 18:23:56 UTC (rev 207474)
+++ trunk/Source/JavaScriptCore/assembler/AbstractMacroAssembler.h 2016-10-18 18:30:05 UTC (rev 207475)
</span><span class="lines">@@ -932,6 +932,11 @@
</span><span class="cx"> {
</span><span class="cx"> AssemblerType::relinkJump(jump.dataLocation(), destination.dataLocation());
</span><span class="cx"> }
</span><ins>+
+ static void repatchJumpToNop(CodeLocationJump jump)
+ {
+ AssemblerType::relinkJumpToNop(jump.dataLocation());
+ }
</ins><span class="cx">
</span><span class="cx"> static void repatchNearCall(CodeLocationNearCall nearCall, CodeLocationLabel destination)
</span><span class="cx"> {
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreassemblerX86Assemblerh"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/assembler/X86Assembler.h (207474 => 207475)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/assembler/X86Assembler.h 2016-10-18 18:23:56 UTC (rev 207474)
+++ trunk/Source/JavaScriptCore/assembler/X86Assembler.h 2016-10-18 18:30:05 UTC (rev 207475)
</span><span class="lines">@@ -2778,6 +2778,11 @@
</span><span class="cx"> setRel32(from, to);
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+ static void relinkJumpToNop(void* from)
+ {
+ setInt32(from, 0);
+ }
+
</ins><span class="cx"> static void relinkCall(void* from, void* to)
</span><span class="cx"> {
</span><span class="cx"> setRel32(from, to);
</span></span></pre></div>
<a id="trunkSourceJavaScriptCorebytecodeCallLinkInfocpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/bytecode/CallLinkInfo.cpp (207474 => 207475)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/bytecode/CallLinkInfo.cpp 2016-10-18 18:23:56 UTC (rev 207474)
+++ trunk/Source/JavaScriptCore/bytecode/CallLinkInfo.cpp 2016-10-18 18:30:05 UTC (rev 207475)
</span><span class="lines">@@ -59,7 +59,9 @@
</span><span class="cx"> , m_hasSeenClosure(false)
</span><span class="cx"> , m_clearedByGC(false)
</span><span class="cx"> , m_allowStubs(true)
</span><ins>+ , m_isLinked(false)
</ins><span class="cx"> , m_callType(None)
</span><ins>+ , m_calleeGPR(255)
</ins><span class="cx"> , m_maxNumArguments(0)
</span><span class="cx"> , m_slowPathCount(0)
</span><span class="cx"> {
</span><span class="lines">@@ -84,32 +86,125 @@
</span><span class="cx">
</span><span class="cx"> void CallLinkInfo::unlink(VM& vm)
</span><span class="cx"> {
</span><del>- if (!isLinked()) {
- // We could be called even if we're not linked anymore because of how polymorphic calls
- // work. Each callsite within the polymorphic call stub may separately ask us to unlink().
- RELEASE_ASSERT(!isOnList());
- return;
- }
-
- unlinkFor(vm, *this);
</del><ins>+ // We could be called even if we're not linked anymore because of how polymorphic calls
+ // work. Each callsite within the polymorphic call stub may separately ask us to unlink().
+ if (isLinked())
+ unlinkFor(vm, *this);
</ins><span class="cx">
</span><del>- // It will be on a list if the callee has a code block.
- if (isOnList())
- remove();
</del><ins>+ // Either we were unlinked, in which case we should not have been on any list, or we unlinked
+ // ourselves so that we're not on any list anymore.
+ RELEASE_ASSERT(!isOnList());
</ins><span class="cx"> }
</span><span class="cx">
</span><ins>+CodeLocationNearCall CallLinkInfo::callReturnLocation()
+{
+ RELEASE_ASSERT(!isDirect());
+ return CodeLocationNearCall(m_callReturnLocationOrPatchableJump, Regular);
+}
+
+CodeLocationJump CallLinkInfo::patchableJump()
+{
+ RELEASE_ASSERT(callType() == DirectTailCall);
+ return CodeLocationJump(m_callReturnLocationOrPatchableJump);
+}
+
+CodeLocationDataLabelPtr CallLinkInfo::hotPathBegin()
+{
+ RELEASE_ASSERT(!isDirect());
+ return CodeLocationDataLabelPtr(m_hotPathBeginOrSlowPathStart);
+}
+
+CodeLocationLabel CallLinkInfo::slowPathStart()
+{
+ RELEASE_ASSERT(isDirect());
+ return m_hotPathBeginOrSlowPathStart;
+}
+
</ins><span class="cx"> void CallLinkInfo::setCallee(VM& vm, JSCell* owner, JSFunction* callee)
</span><span class="cx"> {
</span><del>- MacroAssembler::repatchPointer(m_hotPathBegin, callee);
- m_callee.set(vm, owner, callee);
</del><ins>+ RELEASE_ASSERT(!isDirect());
+ MacroAssembler::repatchPointer(hotPathBegin(), callee);
+ m_calleeOrCodeBlock.set(vm, owner, callee);
+ m_isLinked = true;
</ins><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> void CallLinkInfo::clearCallee()
</span><span class="cx"> {
</span><del>- MacroAssembler::repatchPointer(m_hotPathBegin, nullptr);
- m_callee.clear();
</del><ins>+ RELEASE_ASSERT(!isDirect());
+ MacroAssembler::repatchPointer(hotPathBegin(), nullptr);
+ m_calleeOrCodeBlock.clear();
+ m_isLinked = false;
</ins><span class="cx"> }
</span><span class="cx">
</span><ins>+JSFunction* CallLinkInfo::callee()
+{
+ RELEASE_ASSERT(!isDirect());
+ return jsCast<JSFunction*>(m_calleeOrCodeBlock.get());
+}
+
+void CallLinkInfo::setCodeBlock(VM& vm, JSCell* owner, FunctionCodeBlock* codeBlock)
+{
+ RELEASE_ASSERT(isDirect());
+ m_calleeOrCodeBlock.setMayBeNull(vm, owner, codeBlock);
+ m_isLinked = true;
+}
+
+void CallLinkInfo::clearCodeBlock()
+{
+ RELEASE_ASSERT(isDirect());
+ m_calleeOrCodeBlock.clear();
+ m_isLinked = false;
+}
+
+FunctionCodeBlock* CallLinkInfo::codeBlock()
+{
+ RELEASE_ASSERT(isDirect());
+ return jsCast<FunctionCodeBlock*>(m_calleeOrCodeBlock.get());
+}
+
+void CallLinkInfo::setLastSeenCallee(VM& vm, const JSCell* owner, JSFunction* callee)
+{
+ RELEASE_ASSERT(!isDirect());
+ m_lastSeenCalleeOrExecutable.set(vm, owner, callee);
+}
+
+void CallLinkInfo::clearLastSeenCallee()
+{
+ RELEASE_ASSERT(!isDirect());
+ m_lastSeenCalleeOrExecutable.clear();
+}
+
+JSFunction* CallLinkInfo::lastSeenCallee()
+{
+ RELEASE_ASSERT(!isDirect());
+ return jsCast<JSFunction*>(m_lastSeenCalleeOrExecutable.get());
+}
+
+bool CallLinkInfo::haveLastSeenCallee()
+{
+ RELEASE_ASSERT(!isDirect());
+ return !!m_lastSeenCalleeOrExecutable;
+}
+
+void CallLinkInfo::setExecutableDuringCompilation(ExecutableBase* executable)
+{
+ RELEASE_ASSERT(isDirect());
+ m_lastSeenCalleeOrExecutable.setWithoutWriteBarrier(executable);
+}
+
+ExecutableBase* CallLinkInfo::executable()
+{
+ RELEASE_ASSERT(isDirect());
+ return jsCast<ExecutableBase*>(m_lastSeenCalleeOrExecutable.get());
+}
+
+void CallLinkInfo::setMaxNumArguments(unsigned value)
+{
+ RELEASE_ASSERT(isDirect());
+ RELEASE_ASSERT(value);
+ m_maxNumArguments = value;
+}
+
</ins><span class="cx"> void CallLinkInfo::visitWeak(VM& vm)
</span><span class="cx"> {
</span><span class="cx"> auto handleSpecificCallee = [&] (JSFunction* callee) {
</span><span class="lines">@@ -131,19 +226,37 @@
</span><span class="cx"> unlink(vm);
</span><span class="cx"> m_clearedByGC = true;
</span><span class="cx"> }
</span><del>- } else if (!Heap::isMarked(m_callee.get())) {
</del><ins>+ } else if (!Heap::isMarked(m_calleeOrCodeBlock.get())) {
+ if (isDirect()) {
+ if (Options::verboseOSR()) {
+ dataLog(
+ "Clearing call to ", RawPointer(codeBlock()), " (",
+ pointerDump(codeBlock()), ").\n");
+ }
+ } else {
+ if (Options::verboseOSR()) {
+ dataLog(
+ "Clearing call to ",
+ RawPointer(callee()), " (",
+ callee()->executable()->hashFor(specializationKind()),
+ ").\n");
+ }
+ handleSpecificCallee(callee());
+ }
+ unlink(vm);
+ } else if (isDirect() && !Heap::isMarked(m_lastSeenCalleeOrExecutable.get())) {
</ins><span class="cx"> if (Options::verboseOSR()) {
</span><span class="cx"> dataLog(
</span><del>- "Clearing call to ",
- RawPointer(m_callee.get()), " (",
- m_callee.get()->executable()->hashFor(specializationKind()),
- ").\n");
</del><ins>+ "Clearing call to ", RawPointer(executable()),
+ " because the executable is dead.\n");
</ins><span class="cx"> }
</span><del>- handleSpecificCallee(m_callee.get());
</del><span class="cx"> unlink(vm);
</span><ins>+ // We should only get here once the owning CodeBlock is dying, since the executable must
+ // already be in the owner's weak references.
+ m_lastSeenCalleeOrExecutable.clear();
</ins><span class="cx"> }
</span><span class="cx"> }
</span><del>- if (haveLastSeenCallee() && !Heap::isMarked(lastSeenCallee())) {
</del><ins>+ if (!isDirect() && haveLastSeenCallee() && !Heap::isMarked(lastSeenCallee())) {
</ins><span class="cx"> handleSpecificCallee(lastSeenCallee());
</span><span class="cx"> clearLastSeenCallee();
</span><span class="cx"> }
</span></span></pre></div>
<a id="trunkSourceJavaScriptCorebytecodeCallLinkInfoh"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/bytecode/CallLinkInfo.h (207474 => 207475)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/bytecode/CallLinkInfo.h 2016-10-18 18:23:56 UTC (rev 207474)
+++ trunk/Source/JavaScriptCore/bytecode/CallLinkInfo.h 2016-10-18 18:30:05 UTC (rev 207475)
</span><span class="lines">@@ -28,7 +28,6 @@
</span><span class="cx"> #include "CallMode.h"
</span><span class="cx"> #include "CodeLocation.h"
</span><span class="cx"> #include "CodeSpecializationKind.h"
</span><del>-#include "JSFunction.h"
</del><span class="cx"> #include "PolymorphicCallStubRoutine.h"
</span><span class="cx"> #include "WriteBarrier.h"
</span><span class="cx"> #include <wtf/SentinelLinkedList.h>
</span><span class="lines">@@ -37,12 +36,26 @@
</span><span class="cx">
</span><span class="cx"> #if ENABLE(JIT)
</span><span class="cx">
</span><ins>+class FunctionCodeBlock;
+class JSFunction;
</ins><span class="cx"> enum OpcodeID : unsigned;
</span><span class="cx"> struct CallFrameShuffleData;
</span><span class="cx">
</span><span class="cx"> class CallLinkInfo : public BasicRawSentinelNode<CallLinkInfo> {
</span><span class="cx"> public:
</span><del>- enum CallType { None, Call, CallVarargs, Construct, ConstructVarargs, TailCall, TailCallVarargs };
</del><ins>+ enum CallType {
+ None,
+ Call,
+ CallVarargs,
+ Construct,
+ ConstructVarargs,
+ TailCall,
+ TailCallVarargs,
+ DirectCall,
+ DirectConstruct,
+ DirectTailCall
+ };
+
</ins><span class="cx"> static CallType callTypeFor(OpcodeID opcodeID);
</span><span class="cx">
</span><span class="cx"> static bool isVarargsCallType(CallType callType)
</span><span class="lines">@@ -64,24 +77,27 @@
</span><span class="cx">
</span><span class="cx"> static CodeSpecializationKind specializationKindFor(CallType callType)
</span><span class="cx"> {
</span><del>- return specializationFromIsConstruct(callType == Construct || callType == ConstructVarargs);
</del><ins>+ return specializationFromIsConstruct(callType == Construct || callType == ConstructVarargs || callType == DirectConstruct);
</ins><span class="cx"> }
</span><span class="cx"> CodeSpecializationKind specializationKind() const
</span><span class="cx"> {
</span><span class="cx"> return specializationKindFor(static_cast<CallType>(m_callType));
</span><span class="cx"> }
</span><del>-
</del><ins>+
</ins><span class="cx"> static CallMode callModeFor(CallType callType)
</span><span class="cx"> {
</span><span class="cx"> switch (callType) {
</span><span class="cx"> case Call:
</span><span class="cx"> case CallVarargs:
</span><ins>+ case DirectCall:
</ins><span class="cx"> return CallMode::Regular;
</span><span class="cx"> case TailCall:
</span><span class="cx"> case TailCallVarargs:
</span><ins>+ case DirectTailCall:
</ins><span class="cx"> return CallMode::Tail;
</span><span class="cx"> case Construct:
</span><span class="cx"> case ConstructVarargs:
</span><ins>+ case DirectConstruct:
</ins><span class="cx"> return CallMode::Construct;
</span><span class="cx"> case None:
</span><span class="cx"> RELEASE_ASSERT_NOT_REACHED();
</span><span class="lines">@@ -89,16 +105,49 @@
</span><span class="cx">
</span><span class="cx"> RELEASE_ASSERT_NOT_REACHED();
</span><span class="cx"> }
</span><ins>+
+ static bool isDirect(CallType callType)
+ {
+ switch (callType) {
+ case DirectCall:
+ case DirectTailCall:
+ case DirectConstruct:
+ return true;
+ case Call:
+ case CallVarargs:
+ case TailCall:
+ case TailCallVarargs:
+ case Construct:
+ case ConstructVarargs:
+ return false;
+ case None:
+ RELEASE_ASSERT_NOT_REACHED();
+ return false;
+ }
</ins><span class="cx">
</span><ins>+ RELEASE_ASSERT_NOT_REACHED();
+ return false;
+ }
+
</ins><span class="cx"> CallMode callMode() const
</span><span class="cx"> {
</span><span class="cx"> return callModeFor(static_cast<CallType>(m_callType));
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+ bool isDirect()
+ {
+ return isDirect(static_cast<CallType>(m_callType));
+ }
+
</ins><span class="cx"> bool isTailCall() const
</span><span class="cx"> {
</span><span class="cx"> return callMode() == CallMode::Tail;
</span><span class="cx"> }
</span><ins>+
+ NearCallMode nearCallMode() const
+ {
+ return isTailCall() ? Tail : Regular;
+ }
</ins><span class="cx">
</span><span class="cx"> bool isVarargs() const
</span><span class="cx"> {
</span><span class="lines">@@ -105,7 +154,7 @@
</span><span class="cx"> return isVarargsCallType(static_cast<CallType>(m_callType));
</span><span class="cx"> }
</span><span class="cx">
</span><del>- bool isLinked() { return m_stub || m_callee; }
</del><ins>+ bool isLinked() { return m_stub || m_calleeOrCodeBlock; }
</ins><span class="cx"> void unlink(VM&);
</span><span class="cx">
</span><span class="cx"> void setUpCall(CallType callType, CodeOrigin codeOrigin, unsigned calleeGPR)
</span><span class="lines">@@ -115,11 +164,13 @@
</span><span class="cx"> m_calleeGPR = calleeGPR;
</span><span class="cx"> }
</span><span class="cx">
</span><del>- void setCallLocations(CodeLocationNearCall callReturnLocation, CodeLocationDataLabelPtr hotPathBegin,
</del><ins>+ void setCallLocations(
+ CodeLocationLabel callReturnLocationOrPatchableJump,
+ CodeLocationLabel hotPathBeginOrSlowPathStart,
</ins><span class="cx"> CodeLocationNearCall hotPathOther)
</span><span class="cx"> {
</span><del>- m_callReturnLocation = callReturnLocation;
- m_hotPathBegin = hotPathBegin;
</del><ins>+ m_callReturnLocationOrPatchableJump = callReturnLocationOrPatchableJump;
+ m_hotPathBeginOrSlowPathStart = hotPathBeginOrSlowPathStart;
</ins><span class="cx"> m_hotPathOther = hotPathOther;
</span><span class="cx"> }
</span><span class="cx">
</span><span class="lines">@@ -130,28 +181,11 @@
</span><span class="cx"> m_allowStubs = false;
</span><span class="cx"> }
</span><span class="cx">
</span><del>- void setUpCallFromFTL(CallType callType, CodeOrigin codeOrigin,
- CodeLocationNearCall callReturnLocation, CodeLocationDataLabelPtr hotPathBegin,
- CodeLocationNearCall hotPathOther, unsigned calleeGPR)
- {
- m_callType = callType;
- m_codeOrigin = codeOrigin;
- m_callReturnLocation = callReturnLocation;
- m_hotPathBegin = hotPathBegin;
- m_hotPathOther = hotPathOther;
- m_calleeGPR = calleeGPR;
- }
</del><ins>+ CodeLocationNearCall callReturnLocation();
+ CodeLocationJump patchableJump();
+ CodeLocationDataLabelPtr hotPathBegin();
+ CodeLocationLabel slowPathStart();
</ins><span class="cx">
</span><del>- CodeLocationNearCall callReturnLocation()
- {
- return m_callReturnLocation;
- }
-
- CodeLocationDataLabelPtr hotPathBegin()
- {
- return m_hotPathBegin;
- }
-
</del><span class="cx"> CodeLocationNearCall hotPathOther()
</span><span class="cx"> {
</span><span class="cx"> return m_hotPathOther;
</span><span class="lines">@@ -158,34 +192,21 @@
</span><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> void setCallee(VM&, JSCell*, JSFunction* callee);
</span><del>-
</del><span class="cx"> void clearCallee();
</span><ins>+ JSFunction* callee();
</ins><span class="cx">
</span><del>- JSFunction* callee()
- {
- return m_callee.get();
- }
</del><ins>+ void setCodeBlock(VM&, JSCell*, FunctionCodeBlock*);
+ void clearCodeBlock();
+ FunctionCodeBlock* codeBlock();
</ins><span class="cx">
</span><del>- void setLastSeenCallee(VM& vm, const JSCell* owner, JSFunction* callee)
- {
- m_lastSeenCallee.set(vm, owner, callee);
- }
-
- void clearLastSeenCallee()
- {
- m_lastSeenCallee.clear();
- }
-
- JSFunction* lastSeenCallee()
- {
- return m_lastSeenCallee.get();
- }
-
- bool haveLastSeenCallee()
- {
- return !!m_lastSeenCallee;
- }
-
</del><ins>+ void setLastSeenCallee(VM& vm, const JSCell* owner, JSFunction* callee);
+ void clearLastSeenCallee();
+ JSFunction* lastSeenCallee();
+ bool haveLastSeenCallee();
+
+ void setExecutableDuringCompilation(ExecutableBase*);
+ ExecutableBase* executable();
+
</ins><span class="cx"> void setStub(PassRefPtr<PolymorphicCallStubRoutine> newStub)
</span><span class="cx"> {
</span><span class="cx"> clearStub();
</span><span class="lines">@@ -254,15 +275,17 @@
</span><span class="cx"> return static_cast<CallType>(m_callType);
</span><span class="cx"> }
</span><span class="cx">
</span><del>- uint8_t* addressOfMaxNumArguments()
</del><ins>+ uint32_t* addressOfMaxNumArguments()
</ins><span class="cx"> {
</span><span class="cx"> return &m_maxNumArguments;
</span><span class="cx"> }
</span><span class="cx">
</span><del>- uint8_t maxNumArguments()
</del><ins>+ uint32_t maxNumArguments()
</ins><span class="cx"> {
</span><span class="cx"> return m_maxNumArguments;
</span><span class="cx"> }
</span><ins>+
+ void setMaxNumArguments(unsigned);
</ins><span class="cx">
</span><span class="cx"> static ptrdiff_t offsetOfSlowPathCount()
</span><span class="cx"> {
</span><span class="lines">@@ -304,11 +327,11 @@
</span><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> private:
</span><del>- CodeLocationNearCall m_callReturnLocation;
- CodeLocationDataLabelPtr m_hotPathBegin;
</del><ins>+ CodeLocationLabel m_callReturnLocationOrPatchableJump;
+ CodeLocationLabel m_hotPathBeginOrSlowPathStart;
</ins><span class="cx"> CodeLocationNearCall m_hotPathOther;
</span><del>- WriteBarrier<JSFunction> m_callee;
- WriteBarrier<JSFunction> m_lastSeenCallee;
</del><ins>+ WriteBarrier<JSCell> m_calleeOrCodeBlock;
+ WriteBarrier<JSCell> m_lastSeenCalleeOrExecutable;
</ins><span class="cx"> RefPtr<PolymorphicCallStubRoutine> m_stub;
</span><span class="cx"> RefPtr<JITStubRoutine> m_slowStub;
</span><span class="cx"> std::unique_ptr<CallFrameShuffleData> m_frameShuffleData;
</span><span class="lines">@@ -316,9 +339,10 @@
</span><span class="cx"> bool m_hasSeenClosure : 1;
</span><span class="cx"> bool m_clearedByGC : 1;
</span><span class="cx"> bool m_allowStubs : 1;
</span><ins>+ bool m_isLinked : 1;
</ins><span class="cx"> unsigned m_callType : 4; // CallType
</span><span class="cx"> unsigned m_calleeGPR : 8;
</span><del>- uint8_t m_maxNumArguments; // Only used for varargs calls.
</del><ins>+ uint32_t m_maxNumArguments; // For varargs: the profiled maximum number of arguments. For direct: the number of stack slots allocated for arguments.
</ins><span class="cx"> uint32_t m_slowPathCount;
</span><span class="cx"> CodeOrigin m_codeOrigin;
</span><span class="cx"> };
</span></span></pre></div>
<a id="trunkSourceJavaScriptCorebytecodeCallLinkStatuscpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/bytecode/CallLinkStatus.cpp (207474 => 207475)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/bytecode/CallLinkStatus.cpp 2016-10-18 18:23:56 UTC (rev 207474)
+++ trunk/Source/JavaScriptCore/bytecode/CallLinkStatus.cpp 2016-10-18 18:30:05 UTC (rev 207475)
</span><span class="lines">@@ -258,6 +258,12 @@
</span><span class="cx"> CodeBlock* baselineCodeBlock = dfgCodeBlock->alternative();
</span><span class="cx"> for (auto iter = dfgCodeBlock->callLinkInfosBegin(); !!iter; ++iter) {
</span><span class="cx"> CallLinkInfo& info = **iter;
</span><ins>+ if (info.isDirect()) {
+ // If the DFG was able to get a direct call then probably so will we. However, there is
+ // a remote chance that it's bad news to lose information about what the DFG did. We'd
+ // ideally like to just know that the DFG had emitted a DirectCall.
+ continue;
+ }
</ins><span class="cx"> CodeOrigin codeOrigin = info.codeOrigin();
</span><span class="cx">
</span><span class="cx"> // Check if we had already previously made a terrible mistake in the FTL for this
</span></span></pre></div>
<a id="trunkSourceJavaScriptCorebytecodePolymorphicAccesscpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/bytecode/PolymorphicAccess.cpp (207474 => 207475)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/bytecode/PolymorphicAccess.cpp 2016-10-18 18:23:56 UTC (rev 207474)
+++ trunk/Source/JavaScriptCore/bytecode/PolymorphicAccess.cpp 2016-10-18 18:30:05 UTC (rev 207475)
</span><span class="lines">@@ -1109,8 +1109,8 @@
</span><span class="cx"> jit.addLinkTask(
</span><span class="cx"> [=, &vm] (LinkBuffer& linkBuffer) {
</span><span class="cx"> m_rareData->callLinkInfo->setCallLocations(
</span><del>- linkBuffer.locationOfNearCall(slowPathCall),
- linkBuffer.locationOf(addressOfLinkFunctionCheck),
</del><ins>+ CodeLocationLabel(linkBuffer.locationOfNearCall(slowPathCall)),
+ CodeLocationLabel(linkBuffer.locationOf(addressOfLinkFunctionCheck)),
</ins><span class="cx"> linkBuffer.locationOfNearCall(fastPathCall));
</span><span class="cx">
</span><span class="cx"> linkBuffer.link(
</span></span></pre></div>
<a id="trunkSourceJavaScriptCorebytecodeUnlinkedFunctionExecutableh"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/bytecode/UnlinkedFunctionExecutable.h (207474 => 207475)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/bytecode/UnlinkedFunctionExecutable.h 2016-10-18 18:23:56 UTC (rev 207474)
+++ trunk/Source/JavaScriptCore/bytecode/UnlinkedFunctionExecutable.h 2016-10-18 18:30:05 UTC (rev 207475)
</span><span class="lines">@@ -76,7 +76,7 @@
</span><span class="cx"> const Identifier& ecmaName() const { return m_ecmaName; }
</span><span class="cx"> void setEcmaName(const Identifier& name) { m_ecmaName = name; }
</span><span class="cx"> const Identifier& inferredName() const { return m_inferredName; }
</span><del>- unsigned parameterCount() const { return m_parameterCount; };
</del><ins>+ unsigned parameterCount() const { return m_parameterCount; }; // Excluding 'this'!
</ins><span class="cx"> unsigned functionLength() const { return m_functionLength; }
</span><span class="cx"> SourceParseMode parseMode() const { return static_cast<SourceParseMode>(m_sourceParseMode); };
</span><span class="cx">
</span></span></pre></div>
<a id="trunkSourceJavaScriptCorebytecodeValueRecoveryh"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/bytecode/ValueRecovery.h (207474 => 207475)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/bytecode/ValueRecovery.h 2016-10-18 18:23:56 UTC (rev 207474)
+++ trunk/Source/JavaScriptCore/bytecode/ValueRecovery.h 2016-10-18 18:30:05 UTC (rev 207475)
</span><span class="lines">@@ -375,6 +375,33 @@
</span><span class="cx"> JSValue recover(ExecState*) const;
</span><span class="cx">
</span><span class="cx"> #if ENABLE(JIT)
</span><ins>+ template<typename Func>
+ void forEachReg(const Func& func)
+ {
+ switch (m_technique) {
+ case InGPR:
+ case UnboxedInt32InGPR:
+ case UnboxedBooleanInGPR:
+ case UnboxedCellInGPR:
+ case UnboxedInt52InGPR:
+ case UnboxedStrictInt52InGPR:
+ func(gpr());
+ return;
+ case InFPR:
+ case UnboxedDoubleInFPR:
+ func(fpr());
+ return;
+#if USE(JSVALUE32_64)
+ case InPair:
+ func(jsValueRegs().payloadGPR());
+ func(jsValueRegs().tagGPR());
+ return;
+#endif
+ default:
+ return;
+ }
+ }
+
</ins><span class="cx"> void dumpInContext(PrintStream& out, DumpContext* context) const;
</span><span class="cx"> void dump(PrintStream& out) const;
</span><span class="cx"> #endif
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoredfgDFGAbstractInterpreterInlinesh"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/dfg/DFGAbstractInterpreterInlines.h (207474 => 207475)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/dfg/DFGAbstractInterpreterInlines.h 2016-10-18 18:23:56 UTC (rev 207474)
+++ trunk/Source/JavaScriptCore/dfg/DFGAbstractInterpreterInlines.h 2016-10-18 18:30:05 UTC (rev 207475)
</span><span class="lines">@@ -1779,6 +1779,7 @@
</span><span class="cx"> break;
</span><span class="cx">
</span><span class="cx"> case TailCall:
</span><ins>+ case DirectTailCall:
</ins><span class="cx"> case TailCallVarargs:
</span><span class="cx"> case TailCallForwardVarargs:
</span><span class="cx"> clobberWorld(node->origin.semantic, clobberLimit);
</span><span class="lines">@@ -2824,6 +2825,9 @@
</span><span class="cx"> case ConstructForwardVarargs:
</span><span class="cx"> case TailCallForwardVarargsInlinedCaller:
</span><span class="cx"> case CallEval:
</span><ins>+ case DirectCall:
+ case DirectConstruct:
+ case DirectTailCallInlinedCaller:
</ins><span class="cx"> clobberWorld(node->origin.semantic, clobberLimit);
</span><span class="cx"> forNode(node).makeHeapTop();
</span><span class="cx"> break;
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoredfgDFGBasicBlockh"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/dfg/DFGBasicBlock.h (207474 => 207475)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/dfg/DFGBasicBlock.h 2016-10-18 18:23:56 UTC (rev 207474)
+++ trunk/Source/JavaScriptCore/dfg/DFGBasicBlock.h 2016-10-18 18:30:05 UTC (rev 207475)
</span><span class="lines">@@ -90,6 +90,7 @@
</span><span class="cx"> case Switch:
</span><span class="cx"> case Return:
</span><span class="cx"> case TailCall:
</span><ins>+ case DirectTailCall:
</ins><span class="cx"> case TailCallVarargs:
</span><span class="cx"> case TailCallForwardVarargs:
</span><span class="cx"> case Unreachable:
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoredfgDFGByteCodeParsercpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/dfg/DFGByteCodeParser.cpp (207474 => 207475)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/dfg/DFGByteCodeParser.cpp 2016-10-18 18:23:56 UTC (rev 207474)
+++ trunk/Source/JavaScriptCore/dfg/DFGByteCodeParser.cpp 2016-10-18 18:30:05 UTC (rev 207475)
</span><span class="lines">@@ -805,9 +805,7 @@
</span><span class="cx"> OpInfo prediction)
</span><span class="cx"> {
</span><span class="cx"> addVarArgChild(callee);
</span><del>- size_t frameSize = CallFrame::headerSizeInRegisters + argCount;
- size_t alignedFrameSize = WTF::roundUpToMultipleOf(stackAlignmentRegisters(), frameSize);
- size_t parameterSlots = alignedFrameSize - CallerFrameAndPC::sizeInRegisters;
</del><ins>+ size_t parameterSlots = Graph::parameterSlotsForArgCount(argCount);
</ins><span class="cx">
</span><span class="cx"> if (parameterSlots > m_parameterSlots)
</span><span class="cx"> m_parameterSlots = parameterSlots;
</span><span class="lines">@@ -1294,8 +1292,6 @@
</span><span class="cx">
</span><span class="cx"> unsigned nextOffset = m_currentIndex + instructionSize;
</span><span class="cx">
</span><del>- OpInfo callOpInfo;
-
</del><span class="cx"> if (handleInlining(callTarget, result, callLinkStatus, registerOffset, virtualRegisterForArgument(0, registerOffset), VirtualRegister(), 0, argumentCountIncludingThis, nextOffset, op, kind, prediction)) {
</span><span class="cx"> if (m_graph.compilation())
</span><span class="cx"> m_graph.compilation()->noticeInlinedCall();
</span><span class="lines">@@ -1302,7 +1298,7 @@
</span><span class="cx"> return NonTerminal;
</span><span class="cx"> }
</span><span class="cx">
</span><del>- Node* callNode = addCall(result, op, callOpInfo, callTarget, argumentCountIncludingThis, registerOffset, prediction);
</del><ins>+ Node* callNode = addCall(result, op, OpInfo(), callTarget, argumentCountIncludingThis, registerOffset, prediction);
</ins><span class="cx"> if (callNode->op() == TailCall)
</span><span class="cx"> return Terminal;
</span><span class="cx"> ASSERT(callNode->op() != TailCallVarargs && callNode->op() != TailCallForwardVarargs);
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoredfgDFGClobberizeh"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/dfg/DFGClobberize.h (207474 => 207475)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/dfg/DFGClobberize.h 2016-10-18 18:23:56 UTC (rev 207474)
+++ trunk/Source/JavaScriptCore/dfg/DFGClobberize.h 2016-10-18 18:30:05 UTC (rev 207475)
</span><span class="lines">@@ -515,8 +515,11 @@
</span><span class="cx"> case ArrayPush:
</span><span class="cx"> case ArrayPop:
</span><span class="cx"> case Call:
</span><ins>+ case DirectCall:
</ins><span class="cx"> case TailCallInlinedCaller:
</span><ins>+ case DirectTailCallInlinedCaller:
</ins><span class="cx"> case Construct:
</span><ins>+ case DirectConstruct:
</ins><span class="cx"> case CallVarargs:
</span><span class="cx"> case CallForwardVarargs:
</span><span class="cx"> case TailCallVarargsInlinedCaller:
</span><span class="lines">@@ -544,6 +547,7 @@
</span><span class="cx"> return;
</span><span class="cx">
</span><span class="cx"> case TailCall:
</span><ins>+ case DirectTailCall:
</ins><span class="cx"> case TailCallVarargs:
</span><span class="cx"> case TailCallForwardVarargs:
</span><span class="cx"> read(World);
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoredfgDFGDoesGCcpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/dfg/DFGDoesGC.cpp (207474 => 207475)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/dfg/DFGDoesGC.cpp 2016-10-18 18:23:56 UTC (rev 207474)
+++ trunk/Source/JavaScriptCore/dfg/DFGDoesGC.cpp 2016-10-18 18:30:05 UTC (rev 207475)
</span><span class="lines">@@ -145,8 +145,11 @@
</span><span class="cx"> case CompareStrictEq:
</span><span class="cx"> case CompareEqPtr:
</span><span class="cx"> case Call:
</span><ins>+ case DirectCall:
</ins><span class="cx"> case TailCallInlinedCaller:
</span><ins>+ case DirectTailCallInlinedCaller:
</ins><span class="cx"> case Construct:
</span><ins>+ case DirectConstruct:
</ins><span class="cx"> case CallVarargs:
</span><span class="cx"> case CallEval:
</span><span class="cx"> case TailCallVarargsInlinedCaller:
</span><span class="lines">@@ -183,6 +186,7 @@
</span><span class="cx"> case Switch:
</span><span class="cx"> case Return:
</span><span class="cx"> case TailCall:
</span><ins>+ case DirectTailCall:
</ins><span class="cx"> case TailCallVarargs:
</span><span class="cx"> case Throw:
</span><span class="cx"> case CountExecution:
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoredfgDFGFixupPhasecpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/dfg/DFGFixupPhase.cpp (207474 => 207475)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/dfg/DFGFixupPhase.cpp 2016-10-18 18:23:56 UTC (rev 207474)
+++ trunk/Source/JavaScriptCore/dfg/DFGFixupPhase.cpp 2016-10-18 18:30:05 UTC (rev 207475)
</span><span class="lines">@@ -1736,9 +1736,12 @@
</span><span class="cx"> case GetGlobalLexicalVariable:
</span><span class="cx"> case NotifyWrite:
</span><span class="cx"> case Call:
</span><ins>+ case DirectCall:
</ins><span class="cx"> case CheckTypeInfoFlags:
</span><span class="cx"> case TailCallInlinedCaller:
</span><ins>+ case DirectTailCallInlinedCaller:
</ins><span class="cx"> case Construct:
</span><ins>+ case DirectConstruct:
</ins><span class="cx"> case CallVarargs:
</span><span class="cx"> case CallEval:
</span><span class="cx"> case TailCallVarargsInlinedCaller:
</span><span class="lines">@@ -1766,6 +1769,7 @@
</span><span class="cx"> case Jump:
</span><span class="cx"> case Return:
</span><span class="cx"> case TailCall:
</span><ins>+ case DirectTailCall:
</ins><span class="cx"> case TailCallVarargs:
</span><span class="cx"> case Throw:
</span><span class="cx"> case ThrowStaticError:
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoredfgDFGGraphcpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/dfg/DFGGraph.cpp (207474 => 207475)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/dfg/DFGGraph.cpp 2016-10-18 18:23:56 UTC (rev 207474)
+++ trunk/Source/JavaScriptCore/dfg/DFGGraph.cpp 2016-10-18 18:30:05 UTC (rev 207475)
</span><span class="lines">@@ -1159,6 +1159,13 @@
</span><span class="cx"> return result;
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+unsigned Graph::parameterSlotsForArgCount(unsigned argCount)
+{
+ size_t frameSize = CallFrame::headerSizeInRegisters + argCount;
+ size_t alignedFrameSize = WTF::roundUpToMultipleOf(stackAlignmentRegisters(), frameSize);
+ return alignedFrameSize - CallerFrameAndPC::sizeInRegisters;
+}
+
</ins><span class="cx"> unsigned Graph::frameRegisterCount()
</span><span class="cx"> {
</span><span class="cx"> unsigned result = m_nextMachineLocal + std::max(m_parameterSlots, static_cast<unsigned>(maxFrameExtentForSlowPathCallInRegisters));
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoredfgDFGGraphh"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/dfg/DFGGraph.h (207474 => 207475)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/dfg/DFGGraph.h 2016-10-18 18:23:56 UTC (rev 207474)
+++ trunk/Source/JavaScriptCore/dfg/DFGGraph.h 2016-10-18 18:30:05 UTC (rev 207475)
</span><span class="lines">@@ -790,6 +790,8 @@
</span><span class="cx"> BytecodeKills& killsFor(CodeBlock*);
</span><span class="cx"> BytecodeKills& killsFor(InlineCallFrame*);
</span><span class="cx">
</span><ins>+ static unsigned parameterSlotsForArgCount(unsigned);
+
</ins><span class="cx"> unsigned frameRegisterCount();
</span><span class="cx"> unsigned stackPointerOffset();
</span><span class="cx"> unsigned requiredRegisterCountForExit();
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoredfgDFGInPlaceAbstractStatecpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/dfg/DFGInPlaceAbstractState.cpp (207474 => 207475)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/dfg/DFGInPlaceAbstractState.cpp 2016-10-18 18:23:56 UTC (rev 207474)
+++ trunk/Source/JavaScriptCore/dfg/DFGInPlaceAbstractState.cpp 2016-10-18 18:30:05 UTC (rev 207475)
</span><span class="lines">@@ -364,6 +364,7 @@
</span><span class="cx">
</span><span class="cx"> case Return:
</span><span class="cx"> case TailCall:
</span><ins>+ case DirectTailCall:
</ins><span class="cx"> case TailCallVarargs:
</span><span class="cx"> case TailCallForwardVarargs:
</span><span class="cx"> case Unreachable:
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoredfgDFGJITCompilercpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/dfg/DFGJITCompiler.cpp (207474 => 207475)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/dfg/DFGJITCompiler.cpp 2016-10-18 18:23:56 UTC (rev 207474)
+++ trunk/Source/JavaScriptCore/dfg/DFGJITCompiler.cpp 2016-10-18 18:30:05 UTC (rev 207475)
</span><span class="lines">@@ -276,13 +276,31 @@
</span><span class="cx">
</span><span class="cx"> for (unsigned i = 0; i < m_jsCalls.size(); ++i) {
</span><span class="cx"> JSCallRecord& record = m_jsCalls[i];
</span><del>- CallLinkInfo& info = *record.m_info;
- linkBuffer.link(record.m_slowCall, FunctionPtr(m_vm->getCTIStub(linkCallThunkGenerator).code().executableAddress()));
- info.setCallLocations(linkBuffer.locationOfNearCall(record.m_slowCall),
- linkBuffer.locationOf(record.m_targetToCheck),
- linkBuffer.locationOfNearCall(record.m_fastCall));
</del><ins>+ CallLinkInfo& info = *record.info;
+ linkBuffer.link(record.slowCall, FunctionPtr(m_vm->getCTIStub(linkCallThunkGenerator).code().executableAddress()));
+ info.setCallLocations(
+ CodeLocationLabel(linkBuffer.locationOfNearCall(record.slowCall)),
+ CodeLocationLabel(linkBuffer.locationOf(record.targetToCheck)),
+ linkBuffer.locationOfNearCall(record.fastCall));
</ins><span class="cx"> }
</span><span class="cx">
</span><ins>+ for (JSDirectCallRecord& record : m_jsDirectCalls) {
+ CallLinkInfo& info = *record.info;
+ linkBuffer.link(record.call, linkBuffer.locationOf(record.slowPath));
+ info.setCallLocations(
+ CodeLocationLabel(),
+ linkBuffer.locationOf(record.slowPath),
+ linkBuffer.locationOfNearCall(record.call));
+ }
+
+ for (JSDirectTailCallRecord& record : m_jsDirectTailCalls) {
+ CallLinkInfo& info = *record.info;
+ info.setCallLocations(
+ linkBuffer.locationOf(record.patchableJump),
+ linkBuffer.locationOf(record.slowPath),
+ linkBuffer.locationOfNearCall(record.call));
+ }
+
</ins><span class="cx"> MacroAssemblerCodeRef osrExitThunk = vm()->getCTIStub(osrExitGenerationThunkGenerator);
</span><span class="cx"> CodeLocationLabel target = CodeLocationLabel(osrExitThunk.code());
</span><span class="cx"> for (unsigned i = 0; i < m_jitCode->osrExit.size(); ++i) {
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoredfgDFGJITCompilerh"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/dfg/DFGJITCompiler.h (207474 => 207475)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/dfg/DFGJITCompiler.h 2016-10-18 18:23:56 UTC (rev 207474)
+++ trunk/Source/JavaScriptCore/dfg/DFGJITCompiler.h 2016-10-18 18:30:05 UTC (rev 207475)
</span><span class="lines">@@ -207,16 +207,21 @@
</span><span class="cx"> m_ins.append(record);
</span><span class="cx"> }
</span><span class="cx">
</span><del>- unsigned currentJSCallIndex() const
- {
- return m_jsCalls.size();
- }
-
</del><span class="cx"> void addJSCall(Call fastCall, Call slowCall, DataLabelPtr targetToCheck, CallLinkInfo* info)
</span><span class="cx"> {
</span><span class="cx"> m_jsCalls.append(JSCallRecord(fastCall, slowCall, targetToCheck, info));
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+ void addJSDirectCall(Call call, Label slowPath, CallLinkInfo* info)
+ {
+ m_jsDirectCalls.append(JSDirectCallRecord(call, slowPath, info));
+ }
+
+ void addJSDirectTailCall(PatchableJump patchableJump, Call call, Label slowPath, CallLinkInfo* info)
+ {
+ m_jsDirectTailCalls.append(JSDirectTailCallRecord(patchableJump, call, slowPath, info));
+ }
+
</ins><span class="cx"> void addWeakReference(JSCell* target)
</span><span class="cx"> {
</span><span class="cx"> m_graph.m_plan.weakReferences.addLazily(target);
</span><span class="lines">@@ -292,23 +297,53 @@
</span><span class="cx">
</span><span class="cx"> struct JSCallRecord {
</span><span class="cx"> JSCallRecord(Call fastCall, Call slowCall, DataLabelPtr targetToCheck, CallLinkInfo* info)
</span><del>- : m_fastCall(fastCall)
- , m_slowCall(slowCall)
- , m_targetToCheck(targetToCheck)
- , m_info(info)
</del><ins>+ : fastCall(fastCall)
+ , slowCall(slowCall)
+ , targetToCheck(targetToCheck)
+ , info(info)
</ins><span class="cx"> {
</span><span class="cx"> }
</span><span class="cx">
</span><del>- Call m_fastCall;
- Call m_slowCall;
- DataLabelPtr m_targetToCheck;
- CallLinkInfo* m_info;
</del><ins>+ Call fastCall;
+ Call slowCall;
+ DataLabelPtr targetToCheck;
+ CallLinkInfo* info;
</ins><span class="cx"> };
</span><span class="cx">
</span><ins>+ struct JSDirectCallRecord {
+ JSDirectCallRecord(Call call, Label slowPath, CallLinkInfo* info)
+ : call(call)
+ , slowPath(slowPath)
+ , info(info)
+ {
+ }
+
+ Call call;
+ Label slowPath;
+ CallLinkInfo* info;
+ };
+
+ struct JSDirectTailCallRecord {
+ JSDirectTailCallRecord(PatchableJump patchableJump, Call call, Label slowPath, CallLinkInfo* info)
+ : patchableJump(patchableJump)
+ , call(call)
+ , slowPath(slowPath)
+ , info(info)
+ {
+ }
+
+ PatchableJump patchableJump;
+ Call call;
+ Label slowPath;
+ CallLinkInfo* info;
+ };
+
</ins><span class="cx"> Vector<InlineCacheWrapper<JITGetByIdGenerator>, 4> m_getByIds;
</span><span class="cx"> Vector<InlineCacheWrapper<JITPutByIdGenerator>, 4> m_putByIds;
</span><span class="cx"> Vector<InRecord, 4> m_ins;
</span><span class="cx"> Vector<JSCallRecord, 4> m_jsCalls;
</span><ins>+ Vector<JSDirectCallRecord, 4> m_jsDirectCalls;
+ Vector<JSDirectTailCallRecord, 4> m_jsDirectTailCalls;
</ins><span class="cx"> SegmentedVector<OSRExitCompilationInfo, 4> m_exitCompilationInfo;
</span><span class="cx"> Vector<Vector<Label>> m_exitSiteLabels;
</span><span class="cx">
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoredfgDFGNodecpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/dfg/DFGNode.cpp (207474 => 207475)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/dfg/DFGNode.cpp 2016-10-18 18:23:56 UTC (rev 207474)
+++ trunk/Source/JavaScriptCore/dfg/DFGNode.cpp 2016-10-18 18:30:05 UTC (rev 207475)
</span><span class="lines">@@ -195,6 +195,31 @@
</span><span class="cx"> child1().node(), child2().node());
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+void Node::convertToDirectCall(FrozenValue* executable)
+{
+ NodeType newOp = LastNodeType;
+ switch (op()) {
+ case Call:
+ newOp = DirectCall;
+ break;
+ case Construct:
+ newOp = DirectConstruct;
+ break;
+ case TailCallInlinedCaller:
+ newOp = DirectTailCallInlinedCaller;
+ break;
+ case TailCall:
+ newOp = DirectTailCall;
+ break;
+ default:
+ RELEASE_ASSERT_NOT_REACHED();
+ break;
+ }
+
+ m_op = newOp;
+ m_opInfo = executable;
+}
+
</ins><span class="cx"> String Node::tryGetString(Graph& graph)
</span><span class="cx"> {
</span><span class="cx"> if (hasConstant())
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoredfgDFGNodeh"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/dfg/DFGNode.h (207474 => 207475)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/dfg/DFGNode.h 2016-10-18 18:23:56 UTC (rev 207474)
+++ trunk/Source/JavaScriptCore/dfg/DFGNode.h 2016-10-18 18:30:05 UTC (rev 207475)
</span><span class="lines">@@ -643,6 +643,8 @@
</span><span class="cx"> m_op = ArithNegate;
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+ void convertToDirectCall(FrozenValue*);
+
</ins><span class="cx"> JSValue asJSValue()
</span><span class="cx"> {
</span><span class="cx"> return constant()->value();
</span><span class="lines">@@ -1254,6 +1256,7 @@
</span><span class="cx"> case Switch:
</span><span class="cx"> case Return:
</span><span class="cx"> case TailCall:
</span><ins>+ case DirectTailCall:
</ins><span class="cx"> case TailCallVarargs:
</span><span class="cx"> case TailCallForwardVarargs:
</span><span class="cx"> case Unreachable:
</span><span class="lines">@@ -1425,8 +1428,11 @@
</span><span class="cx"> case GetByVal:
</span><span class="cx"> case GetByValWithThis:
</span><span class="cx"> case Call:
</span><ins>+ case DirectCall:
</ins><span class="cx"> case TailCallInlinedCaller:
</span><ins>+ case DirectTailCallInlinedCaller:
</ins><span class="cx"> case Construct:
</span><ins>+ case DirectConstruct:
</ins><span class="cx"> case CallVarargs:
</span><span class="cx"> case CallEval:
</span><span class="cx"> case TailCallVarargsInlinedCaller:
</span><span class="lines">@@ -1477,6 +1483,10 @@
</span><span class="cx"> case MaterializeCreateActivation:
</span><span class="cx"> case NewRegexp:
</span><span class="cx"> case CompareEqPtr:
</span><ins>+ case DirectCall:
+ case DirectTailCall:
+ case DirectConstruct:
+ case DirectTailCallInlinedCaller:
</ins><span class="cx"> return true;
</span><span class="cx"> default:
</span><span class="cx"> return false;
</span><span class="lines">@@ -2405,8 +2415,7 @@
</span><span class="cx"> unsigned m_refCount;
</span><span class="cx"> // The prediction ascribed to this node after propagation.
</span><span class="cx"> SpeculatedType m_prediction { SpecNone };
</span><del>- // Immediate values, accesses type-checked via accessors above. The first one is
- // big enough to store a pointer.
</del><ins>+ // Immediate values, accesses type-checked via accessors above.
</ins><span class="cx"> struct OpInfoWrapper {
</span><span class="cx"> OpInfoWrapper()
</span><span class="cx"> {
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoredfgDFGNodeTypeh"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/dfg/DFGNodeType.h (207474 => 207475)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/dfg/DFGNodeType.h 2016-10-18 18:23:56 UTC (rev 207474)
+++ trunk/Source/JavaScriptCore/dfg/DFGNodeType.h 2016-10-18 18:30:05 UTC (rev 207475)
</span><span class="lines">@@ -270,12 +270,15 @@
</span><span class="cx"> \
</span><span class="cx"> /* Calls. */\
</span><span class="cx"> macro(Call, NodeResultJS | NodeMustGenerate | NodeHasVarArgs) \
</span><ins>+ macro(DirectCall, NodeResultJS | NodeMustGenerate | NodeHasVarArgs) \
</ins><span class="cx"> macro(Construct, NodeResultJS | NodeMustGenerate | NodeHasVarArgs) \
</span><ins>+ macro(DirectConstruct, NodeResultJS | NodeMustGenerate | NodeHasVarArgs) \
</ins><span class="cx"> macro(CallVarargs, NodeResultJS | NodeMustGenerate) \
</span><span class="cx"> macro(CallForwardVarargs, NodeResultJS | NodeMustGenerate) \
</span><span class="cx"> macro(ConstructVarargs, NodeResultJS | NodeMustGenerate) \
</span><span class="cx"> macro(ConstructForwardVarargs, NodeResultJS | NodeMustGenerate) \
</span><span class="cx"> macro(TailCallInlinedCaller, NodeResultJS | NodeMustGenerate | NodeHasVarArgs) \
</span><ins>+ macro(DirectTailCallInlinedCaller, NodeResultJS | NodeMustGenerate | NodeHasVarArgs) \
</ins><span class="cx"> macro(TailCallVarargsInlinedCaller, NodeResultJS | NodeMustGenerate) \
</span><span class="cx"> macro(TailCallForwardVarargsInlinedCaller, NodeResultJS | NodeMustGenerate) \
</span><span class="cx"> macro(CallEval, NodeResultJS | NodeMustGenerate | NodeHasVarArgs) \
</span><span class="lines">@@ -358,6 +361,7 @@
</span><span class="cx"> macro(Switch, NodeMustGenerate) \
</span><span class="cx"> macro(Return, NodeMustGenerate) \
</span><span class="cx"> macro(TailCall, NodeMustGenerate | NodeHasVarArgs) \
</span><ins>+ macro(DirectTailCall, NodeMustGenerate | NodeHasVarArgs) \
</ins><span class="cx"> macro(TailCallVarargs, NodeMustGenerate) \
</span><span class="cx"> macro(TailCallForwardVarargs, NodeMustGenerate) \
</span><span class="cx"> macro(Unreachable, NodeMustGenerate) \
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoredfgDFGPredictionPropagationPhasecpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/dfg/DFGPredictionPropagationPhase.cpp (207474 => 207475)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/dfg/DFGPredictionPropagationPhase.cpp 2016-10-18 18:23:56 UTC (rev 207474)
+++ trunk/Source/JavaScriptCore/dfg/DFGPredictionPropagationPhase.cpp 2016-10-18 18:30:05 UTC (rev 207475)
</span><span class="lines">@@ -691,8 +691,11 @@
</span><span class="cx"> case MultiGetByOffset:
</span><span class="cx"> case GetDirectPname:
</span><span class="cx"> case Call:
</span><ins>+ case DirectCall:
</ins><span class="cx"> case TailCallInlinedCaller:
</span><ins>+ case DirectTailCallInlinedCaller:
</ins><span class="cx"> case Construct:
</span><ins>+ case DirectConstruct:
</ins><span class="cx"> case CallVarargs:
</span><span class="cx"> case CallEval:
</span><span class="cx"> case TailCallVarargsInlinedCaller:
</span><span class="lines">@@ -1043,6 +1046,7 @@
</span><span class="cx"> case PutToArguments:
</span><span class="cx"> case Return:
</span><span class="cx"> case TailCall:
</span><ins>+ case DirectTailCall:
</ins><span class="cx"> case TailCallVarargs:
</span><span class="cx"> case TailCallForwardVarargs:
</span><span class="cx"> case Throw:
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoredfgDFGSafeToExecuteh"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/dfg/DFGSafeToExecute.h (207474 => 207475)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/dfg/DFGSafeToExecute.h 2016-10-18 18:23:56 UTC (rev 207474)
+++ trunk/Source/JavaScriptCore/dfg/DFGSafeToExecute.h 2016-10-18 18:30:05 UTC (rev 207475)
</span><span class="lines">@@ -245,8 +245,11 @@
</span><span class="cx"> case CompareStrictEq:
</span><span class="cx"> case CompareEqPtr:
</span><span class="cx"> case Call:
</span><ins>+ case DirectCall:
</ins><span class="cx"> case TailCallInlinedCaller:
</span><ins>+ case DirectTailCallInlinedCaller:
</ins><span class="cx"> case Construct:
</span><ins>+ case DirectConstruct:
</ins><span class="cx"> case CallVarargs:
</span><span class="cx"> case CallEval:
</span><span class="cx"> case TailCallVarargsInlinedCaller:
</span><span class="lines">@@ -301,6 +304,7 @@
</span><span class="cx"> case Switch:
</span><span class="cx"> case Return:
</span><span class="cx"> case TailCall:
</span><ins>+ case DirectTailCall:
</ins><span class="cx"> case TailCallVarargs:
</span><span class="cx"> case TailCallForwardVarargs:
</span><span class="cx"> case Throw:
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoredfgDFGSpeculativeJITh"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/dfg/DFGSpeculativeJIT.h (207474 => 207475)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/dfg/DFGSpeculativeJIT.h 2016-10-18 18:23:56 UTC (rev 207474)
+++ trunk/Source/JavaScriptCore/dfg/DFGSpeculativeJIT.h 2016-10-18 18:30:05 UTC (rev 207475)
</span><span class="lines">@@ -1155,6 +1155,12 @@
</span><span class="cx"> return appendCall(operation);
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+ JITCompiler::Call callOperation(V_JITOperation_ECliJsf operation, CallLinkInfo* callLinkInfo, GPRReg arg1)
+ {
+ m_jit.setupArgumentsWithExecState(TrustedImmPtr(callLinkInfo), arg1);
+ return appendCall(operation);
+ }
+
</ins><span class="cx"> JITCompiler::Call callOperation(V_JITOperation_EC operation, JSCell* arg1)
</span><span class="cx"> {
</span><span class="cx"> m_jit.setupArgumentsWithExecState(TrustedImmPtr(arg1));
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoredfgDFGSpeculativeJIT32_64cpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/dfg/DFGSpeculativeJIT32_64.cpp (207474 => 207475)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/dfg/DFGSpeculativeJIT32_64.cpp 2016-10-18 18:23:56 UTC (rev 207474)
+++ trunk/Source/JavaScriptCore/dfg/DFGSpeculativeJIT32_64.cpp 2016-10-18 18:30:05 UTC (rev 207475)
</span><span class="lines">@@ -657,6 +657,7 @@
</span><span class="cx"> bool isVarargs = false;
</span><span class="cx"> bool isForwardVarargs = false;
</span><span class="cx"> bool isTail = false;
</span><ins>+ bool isDirect = false;
</ins><span class="cx"> bool isEmulatedTail = false;
</span><span class="cx"> switch (node->op()) {
</span><span class="cx"> case Call:
</span><span class="lines">@@ -710,6 +711,24 @@
</span><span class="cx"> callType = CallLinkInfo::ConstructVarargs;
</span><span class="cx"> isForwardVarargs = true;
</span><span class="cx"> break;
</span><ins>+ case DirectCall:
+ callType = CallLinkInfo::DirectCall;
+ isDirect = true;
+ break;
+ case DirectConstruct:
+ callType = CallLinkInfo::DirectConstruct;
+ isDirect = true;
+ break;
+ case DirectTailCall:
+ callType = CallLinkInfo::DirectTailCall;
+ isTail = true;
+ isDirect = true;
+ break;
+ case DirectTailCallInlinedCaller:
+ callType = CallLinkInfo::DirectCall;
+ isEmulatedTail = true;
+ isDirect = true;
+ break;
</ins><span class="cx"> default:
</span><span class="cx"> DFG_CRASH(m_jit.graph(), node, "bad node type");
</span><span class="cx"> break;
</span><span class="lines">@@ -720,8 +739,19 @@
</span><span class="cx"> GPRReg calleePayloadGPR = InvalidGPRReg;
</span><span class="cx"> CallFrameShuffleData shuffleData;
</span><span class="cx">
</span><ins>+ ExecutableBase* executable = nullptr;
+ FunctionExecutable* functionExecutable = nullptr;
+ if (isDirect) {
+ executable = node->castOperand<ExecutableBase*>();
+ functionExecutable = jsDynamicCast<FunctionExecutable*>(executable);
+ }
+
+ unsigned numPassedArgs = 0;
+ unsigned numAllocatedArgs = 0;
+
</ins><span class="cx"> // Gotta load the arguments somehow. Varargs is trickier.
</span><span class="cx"> if (isVarargs || isForwardVarargs) {
</span><ins>+ RELEASE_ASSERT(!isDirect);
</ins><span class="cx"> CallVarargsData* data = node->callVarargsData();
</span><span class="cx">
</span><span class="cx"> GPRReg resultGPR;
</span><span class="lines">@@ -817,28 +847,53 @@
</span><span class="cx"> } else {
</span><span class="cx"> // The call instruction's first child is either the function (normal call) or the
</span><span class="cx"> // receiver (method call). subsequent children are the arguments.
</span><del>- int numPassedArgs = node->numChildren() - 1;
</del><ins>+ numPassedArgs = node->numChildren() - 1;
+ numAllocatedArgs = numPassedArgs;
+
+ if (functionExecutable) {
+ // Allocate more args if this would let us avoid arity checks. This is throttled by
+ // CallLinkInfo's limit. It's probably good to throttle it - if the callee wants a
+ // ginormous amount of argument space then it's better for them to do it so that when we
+ // make calls to other things, we don't waste space.
+ unsigned desiredNumAllocatedArgs = static_cast<unsigned>(functionExecutable->parameterCount()) + 1;
+ if (desiredNumAllocatedArgs <= Options::maximumDirectCallStackSize()) {
+ numAllocatedArgs = std::max(numAllocatedArgs, desiredNumAllocatedArgs);
+
+ // Whoever converts to DirectCall should do this adjustment. It's too late for us to
+ // do this adjustment now since we will have already emitted code that relied on the
+ // value of m_parameterSlots.
+ DFG_ASSERT(
+ m_jit.graph(), node,
+ Graph::parameterSlotsForArgCount(numAllocatedArgs)
+ <= m_jit.graph().m_parameterSlots);
+ }
+ }
</ins><span class="cx">
</span><del>- if (node->op() == TailCall) {
</del><ins>+ if (isTail) {
</ins><span class="cx"> JSValueOperand callee(this, calleeEdge);
</span><span class="cx"> calleeTagGPR = callee.tagGPR();
</span><span class="cx"> calleePayloadGPR = callee.payloadGPR();
</span><del>- use(calleeEdge);
</del><ins>+ if (!isDirect)
+ use(calleeEdge);
</ins><span class="cx">
</span><span class="cx"> shuffleData.numLocals = m_jit.graph().frameRegisterCount();
</span><span class="cx"> shuffleData.callee = ValueRecovery::inPair(calleeTagGPR, calleePayloadGPR);
</span><del>- shuffleData.args.resize(numPassedArgs);
</del><ins>+ shuffleData.args.resize(numAllocatedArgs);
</ins><span class="cx">
</span><del>- for (int i = 0; i < numPassedArgs; ++i) {
</del><ins>+ for (unsigned i = 0; i < numPassedArgs; ++i) {
</ins><span class="cx"> Edge argEdge = m_jit.graph().varArgChild(node, i + 1);
</span><span class="cx"> GenerationInfo& info = generationInfo(argEdge.node());
</span><del>- use(argEdge);
</del><ins>+ if (!isDirect)
+ use(argEdge);
</ins><span class="cx"> shuffleData.args[i] = info.recovery(argEdge->virtualRegister());
</span><span class="cx"> }
</span><ins>+
+ for (unsigned i = numPassedArgs; i < numAllocatedArgs; ++i)
+ shuffleData.args[i] = ValueRecovery::constant(jsUndefined());
</ins><span class="cx"> } else {
</span><span class="cx"> m_jit.store32(MacroAssembler::TrustedImm32(numPassedArgs), m_jit.calleeFramePayloadSlot(CallFrameSlot::argumentCount));
</span><span class="cx">
</span><del>- for (int i = 0; i < numPassedArgs; i++) {
</del><ins>+ for (unsigned i = 0; i < numPassedArgs; i++) {
</ins><span class="cx"> Edge argEdge = m_jit.graph().m_varArgChildren[node->firstChild() + 1 + i];
</span><span class="cx"> JSValueOperand arg(this, argEdge);
</span><span class="cx"> GPRReg argTagGPR = arg.tagGPR();
</span><span class="lines">@@ -848,10 +903,13 @@
</span><span class="cx"> m_jit.store32(argTagGPR, m_jit.calleeArgumentTagSlot(i));
</span><span class="cx"> m_jit.store32(argPayloadGPR, m_jit.calleeArgumentPayloadSlot(i));
</span><span class="cx"> }
</span><ins>+
+ for (unsigned i = numPassedArgs; i < numAllocatedArgs; ++i)
+ m_jit.storeTrustedValue(jsUndefined(), JITCompiler::calleeArgumentSlot(i));
</ins><span class="cx"> }
</span><span class="cx"> }
</span><span class="cx">
</span><del>- if (node->op() != TailCall) {
</del><ins>+ if (!isTail || isVarargs || isForwardVarargs) {
</ins><span class="cx"> JSValueOperand callee(this, calleeEdge);
</span><span class="cx"> calleeTagGPR = callee.tagGPR();
</span><span class="cx"> calleePayloadGPR = callee.payloadGPR();
</span><span class="lines">@@ -863,11 +921,6 @@
</span><span class="cx"> flushRegisters();
</span><span class="cx"> }
</span><span class="cx">
</span><del>- GPRFlushedCallResult resultPayload(this);
- GPRFlushedCallResult2 resultTag(this);
- GPRReg resultPayloadGPR = resultPayload.gpr();
- GPRReg resultTagGPR = resultTag.gpr();
-
</del><span class="cx"> JITCompiler::DataLabelPtr targetToCheck;
</span><span class="cx"> JITCompiler::JumpList slowPath;
</span><span class="cx">
</span><span class="lines">@@ -877,12 +930,16 @@
</span><span class="cx"> CodeOrigin dynamicOrigin =
</span><span class="cx"> isEmulatedTail ? *staticOrigin.inlineCallFrame->getCallerSkippingTailCalls() : staticOrigin;
</span><span class="cx"> CallSiteIndex callSite = m_jit.recordCallSiteAndGenerateExceptionHandlingOSRExitIfNeeded(dynamicOrigin, m_stream->size());
</span><del>- m_jit.emitStoreCallSiteIndex(callSite);
</del><span class="cx">
</span><span class="cx"> CallLinkInfo* info = m_jit.codeBlock()->addCallLinkInfo();
</span><span class="cx"> info->setUpCall(callType, node->origin.semantic, calleePayloadGPR);
</span><span class="cx">
</span><span class="cx"> auto setResultAndResetStack = [&] () {
</span><ins>+ GPRFlushedCallResult resultPayload(this);
+ GPRFlushedCallResult2 resultTag(this);
+ GPRReg resultPayloadGPR = resultPayload.gpr();
+ GPRReg resultTagGPR = resultTag.gpr();
+
</ins><span class="cx"> m_jit.setupResults(resultPayloadGPR, resultTagGPR);
</span><span class="cx">
</span><span class="cx"> jsValueResult(resultTagGPR, resultPayloadGPR, node, DataFormatJS, UseChildrenCalledExplicitly);
</span><span class="lines">@@ -898,6 +955,7 @@
</span><span class="cx"> // subtract stack to make room for the call. Lucky for us, at this point we have the whole
</span><span class="cx"> // register file to ourselves.
</span><span class="cx">
</span><ins>+ m_jit.emitStoreCallSiteIndex(callSite);
</ins><span class="cx"> m_jit.addPtr(TrustedImm32(-static_cast<ptrdiff_t>(sizeof(CallerFrameAndPC))), JITCompiler::stackPointerRegister, GPRInfo::regT0);
</span><span class="cx"> m_jit.storePtr(GPRInfo::callFrameRegister, JITCompiler::Address(GPRInfo::regT0, CallFrame::callerFrameOffset()));
</span><span class="cx">
</span><span class="lines">@@ -924,6 +982,63 @@
</span><span class="cx"> return;
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+ if (isDirect) {
+ info->setExecutableDuringCompilation(executable);
+ info->setMaxNumArguments(numAllocatedArgs);
+
+ if (isTail) {
+ RELEASE_ASSERT(node->op() == DirectTailCall);
+
+ JITCompiler::PatchableJump patchableJump = m_jit.patchableJump();
+ JITCompiler::Label mainPath = m_jit.label();
+
+ m_jit.emitStoreCallSiteIndex(callSite);
+
+ info->setFrameShuffleData(shuffleData);
+ CallFrameShuffler(m_jit, shuffleData).prepareForTailCall();
+
+ JITCompiler::Call call = m_jit.nearTailCall();
+
+ JITCompiler::Label slowPath = m_jit.label();
+ patchableJump.m_jump.linkTo(slowPath, &m_jit);
+
+ silentSpillAllRegisters(InvalidGPRReg);
+ callOperation(operationLinkDirectCall, info, calleePayloadGPR);
+ silentFillAllRegisters(InvalidGPRReg);
+ m_jit.exceptionCheck();
+ m_jit.jump().linkTo(mainPath, &m_jit);
+
+ useChildren(node);
+
+ m_jit.addJSDirectTailCall(patchableJump, call, slowPath, info);
+ return;
+ }
+
+ JITCompiler::Label mainPath = m_jit.label();
+
+ m_jit.emitStoreCallSiteIndex(callSite);
+
+ JITCompiler::Call call = m_jit.nearCall();
+ JITCompiler::Jump done = m_jit.jump();
+
+ JITCompiler::Label slowPath = m_jit.label();
+ if (isX86())
+ m_jit.pop(JITCompiler::selectScratchGPR(calleePayloadGPR));
+
+ callOperation(operationLinkDirectCall, info, calleePayloadGPR);
+ m_jit.exceptionCheck();
+ m_jit.jump().linkTo(mainPath, &m_jit);
+
+ done.link(&m_jit);
+
+ setResultAndResetStack();
+
+ m_jit.addJSDirectCall(call, slowPath, info);
+ return;
+ }
+
+ m_jit.emitStoreCallSiteIndex(callSite);
+
</ins><span class="cx"> slowPath.append(m_jit.branchIfNotCell(JSValueRegs(calleeTagGPR, calleePayloadGPR)));
</span><span class="cx"> slowPath.append(m_jit.branchPtrWithPatch(MacroAssembler::NotEqual, calleePayloadGPR, targetToCheck));
</span><span class="cx">
</span><span class="lines">@@ -4780,6 +4895,10 @@
</span><span class="cx"> case TailCallForwardVarargsInlinedCaller:
</span><span class="cx"> case ConstructForwardVarargs:
</span><span class="cx"> case CallEval:
</span><ins>+ case DirectCall:
+ case DirectConstruct:
+ case DirectTailCall:
+ case DirectTailCallInlinedCaller:
</ins><span class="cx"> emitCall(node);
</span><span class="cx"> break;
</span><span class="cx">
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoredfgDFGSpeculativeJIT64cpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/dfg/DFGSpeculativeJIT64.cpp (207474 => 207475)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/dfg/DFGSpeculativeJIT64.cpp 2016-10-18 18:23:56 UTC (rev 207474)
+++ trunk/Source/JavaScriptCore/dfg/DFGSpeculativeJIT64.cpp 2016-10-18 18:30:05 UTC (rev 207475)
</span><span class="lines">@@ -636,6 +636,7 @@
</span><span class="cx"> bool isForwardVarargs = false;
</span><span class="cx"> bool isTail = false;
</span><span class="cx"> bool isEmulatedTail = false;
</span><ins>+ bool isDirect = false;
</ins><span class="cx"> switch (node->op()) {
</span><span class="cx"> case Call:
</span><span class="cx"> case CallEval:
</span><span class="lines">@@ -688,6 +689,24 @@
</span><span class="cx"> isEmulatedTail = true;
</span><span class="cx"> isForwardVarargs = true;
</span><span class="cx"> break;
</span><ins>+ case DirectCall:
+ callType = CallLinkInfo::DirectCall;
+ isDirect = true;
+ break;
+ case DirectConstruct:
+ callType = CallLinkInfo::DirectConstruct;
+ isDirect = true;
+ break;
+ case DirectTailCall:
+ callType = CallLinkInfo::DirectTailCall;
+ isTail = true;
+ isDirect = true;
+ break;
+ case DirectTailCallInlinedCaller:
+ callType = CallLinkInfo::DirectCall;
+ isEmulatedTail = true;
+ isDirect = true;
+ break;
</ins><span class="cx"> default:
</span><span class="cx"> DFG_CRASH(m_jit.graph(), node, "bad node type");
</span><span class="cx"> break;
</span><span class="lines">@@ -696,8 +715,19 @@
</span><span class="cx"> GPRReg calleeGPR = InvalidGPRReg;
</span><span class="cx"> CallFrameShuffleData shuffleData;
</span><span class="cx">
</span><ins>+ ExecutableBase* executable = nullptr;
+ FunctionExecutable* functionExecutable = nullptr;
+ if (isDirect) {
+ executable = node->castOperand<ExecutableBase*>();
+ functionExecutable = jsDynamicCast<FunctionExecutable*>(executable);
+ }
+
+ unsigned numPassedArgs = 0;
+ unsigned numAllocatedArgs = 0;
+
</ins><span class="cx"> // Gotta load the arguments somehow. Varargs is trickier.
</span><span class="cx"> if (isVarargs || isForwardVarargs) {
</span><ins>+ RELEASE_ASSERT(!isDirect);
</ins><span class="cx"> CallVarargsData* data = node->callVarargsData();
</span><span class="cx">
</span><span class="cx"> GPRReg resultGPR;
</span><span class="lines">@@ -788,31 +818,56 @@
</span><span class="cx"> } else {
</span><span class="cx"> // The call instruction's first child is the function; the subsequent children are the
</span><span class="cx"> // arguments.
</span><del>- int numPassedArgs = node->numChildren() - 1;
</del><ins>+ numPassedArgs = node->numChildren() - 1;
+ numAllocatedArgs = numPassedArgs;
+
+ if (functionExecutable) {
+ // Allocate more args if this would let us avoid arity checks. This is throttled by
+ // CallLinkInfo's limit. It's probably good to throttle it - if the callee wants a
+ // ginormous amount of argument space then it's better for them to do it so that when we
+ // make calls to other things, we don't waste space.
+ unsigned desiredNumAllocatedArgs = static_cast<unsigned>(functionExecutable->parameterCount()) + 1;
+ if (desiredNumAllocatedArgs <= Options::maximumDirectCallStackSize()) {
+ numAllocatedArgs = std::max(numAllocatedArgs, desiredNumAllocatedArgs);
+
+ // Whoever converts to DirectCall should do this adjustment. It's too late for us to
+ // do this adjustment now since we will have already emitted code that relied on the
+ // value of m_parameterSlots.
+ DFG_ASSERT(
+ m_jit.graph(), node,
+ Graph::parameterSlotsForArgCount(numAllocatedArgs)
+ <= m_jit.graph().m_parameterSlots);
+ }
+ }
</ins><span class="cx">
</span><del>- if (node->op() == TailCall) {
</del><ins>+ if (isTail) {
</ins><span class="cx"> Edge calleeEdge = m_jit.graph().child(node, 0);
</span><span class="cx"> JSValueOperand callee(this, calleeEdge);
</span><span class="cx"> calleeGPR = callee.gpr();
</span><del>- callee.use();
</del><ins>+ if (!isDirect)
+ callee.use();
</ins><span class="cx">
</span><span class="cx"> shuffleData.tagTypeNumber = GPRInfo::tagTypeNumberRegister;
</span><span class="cx"> shuffleData.numLocals = m_jit.graph().frameRegisterCount();
</span><span class="cx"> shuffleData.callee = ValueRecovery::inGPR(calleeGPR, DataFormatJS);
</span><del>- shuffleData.args.resize(numPassedArgs);
</del><ins>+ shuffleData.args.resize(numAllocatedArgs);
</ins><span class="cx">
</span><del>- for (int i = 0; i < numPassedArgs; ++i) {
</del><ins>+ for (unsigned i = 0; i < numPassedArgs; ++i) {
</ins><span class="cx"> Edge argEdge = m_jit.graph().varArgChild(node, i + 1);
</span><span class="cx"> GenerationInfo& info = generationInfo(argEdge.node());
</span><del>- use(argEdge);
</del><ins>+ if (!isDirect)
+ use(argEdge);
</ins><span class="cx"> shuffleData.args[i] = info.recovery(argEdge->virtualRegister());
</span><span class="cx"> }
</span><ins>+
+ for (unsigned i = numPassedArgs; i < numAllocatedArgs; ++i)
+ shuffleData.args[i] = ValueRecovery::constant(jsUndefined());
</ins><span class="cx">
</span><span class="cx"> shuffleData.setupCalleeSaveRegisters(m_jit.codeBlock());
</span><span class="cx"> } else {
</span><span class="cx"> m_jit.store32(MacroAssembler::TrustedImm32(numPassedArgs), JITCompiler::calleeFramePayloadSlot(CallFrameSlot::argumentCount));
</span><span class="cx">
</span><del>- for (int i = 0; i < numPassedArgs; i++) {
</del><ins>+ for (unsigned i = 0; i < numPassedArgs; i++) {
</ins><span class="cx"> Edge argEdge = m_jit.graph().m_varArgChildren[node->firstChild() + 1 + i];
</span><span class="cx"> JSValueOperand arg(this, argEdge);
</span><span class="cx"> GPRReg argGPR = arg.gpr();
</span><span class="lines">@@ -820,10 +875,13 @@
</span><span class="cx">
</span><span class="cx"> m_jit.store64(argGPR, JITCompiler::calleeArgumentSlot(i));
</span><span class="cx"> }
</span><ins>+
+ for (unsigned i = numPassedArgs; i < numAllocatedArgs; ++i)
+ m_jit.storeTrustedValue(jsUndefined(), JITCompiler::calleeArgumentSlot(i));
</ins><span class="cx"> }
</span><span class="cx"> }
</span><del>-
- if (node->op() != TailCall) {
</del><ins>+
+ if (!isTail || isVarargs || isForwardVarargs) {
</ins><span class="cx"> Edge calleeEdge = m_jit.graph().child(node, 0);
</span><span class="cx"> JSValueOperand callee(this, calleeEdge);
</span><span class="cx"> calleeGPR = callee.gpr();
</span><span class="lines">@@ -840,7 +898,6 @@
</span><span class="cx"> isEmulatedTail ? *staticOrigin.inlineCallFrame->getCallerSkippingTailCalls() : staticOrigin;
</span><span class="cx">
</span><span class="cx"> CallSiteIndex callSite = m_jit.recordCallSiteAndGenerateExceptionHandlingOSRExitIfNeeded(dynamicOrigin, m_stream->size());
</span><del>- m_jit.emitStoreCallSiteIndex(callSite);
</del><span class="cx">
</span><span class="cx"> auto setResultAndResetStack = [&] () {
</span><span class="cx"> GPRFlushedCallResult result(this);
</span><span class="lines">@@ -864,6 +921,7 @@
</span><span class="cx"> // subtract stack to make room for the call. Lucky for us, at this point we have the whole
</span><span class="cx"> // register file to ourselves.
</span><span class="cx">
</span><ins>+ m_jit.emitStoreCallSiteIndex(callSite);
</ins><span class="cx"> m_jit.addPtr(TrustedImm32(-static_cast<ptrdiff_t>(sizeof(CallerFrameAndPC))), JITCompiler::stackPointerRegister, GPRInfo::regT0);
</span><span class="cx"> m_jit.storePtr(GPRInfo::callFrameRegister, JITCompiler::Address(GPRInfo::regT0, CallFrame::callerFrameOffset()));
</span><span class="cx">
</span><span class="lines">@@ -889,6 +947,63 @@
</span><span class="cx"> return;
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+ if (isDirect) {
+ callLinkInfo->setExecutableDuringCompilation(executable);
+ callLinkInfo->setMaxNumArguments(numAllocatedArgs);
+
+ if (isTail) {
+ RELEASE_ASSERT(node->op() == DirectTailCall);
+
+ JITCompiler::PatchableJump patchableJump = m_jit.patchableJump();
+ JITCompiler::Label mainPath = m_jit.label();
+
+ m_jit.emitStoreCallSiteIndex(callSite);
+
+ callLinkInfo->setFrameShuffleData(shuffleData);
+ CallFrameShuffler(m_jit, shuffleData).prepareForTailCall();
+
+ JITCompiler::Call call = m_jit.nearTailCall();
+
+ JITCompiler::Label slowPath = m_jit.label();
+ patchableJump.m_jump.linkTo(slowPath, &m_jit);
+
+ silentSpillAllRegisters(InvalidGPRReg);
+ callOperation(operationLinkDirectCall, callLinkInfo, calleeGPR);
+ silentFillAllRegisters(InvalidGPRReg);
+ m_jit.exceptionCheck();
+ m_jit.jump().linkTo(mainPath, &m_jit);
+
+ useChildren(node);
+
+ m_jit.addJSDirectTailCall(patchableJump, call, slowPath, callLinkInfo);
+ return;
+ }
+
+ JITCompiler::Label mainPath = m_jit.label();
+
+ m_jit.emitStoreCallSiteIndex(callSite);
+
+ JITCompiler::Call call = m_jit.nearCall();
+ JITCompiler::Jump done = m_jit.jump();
+
+ JITCompiler::Label slowPath = m_jit.label();
+ if (isX86())
+ m_jit.pop(JITCompiler::selectScratchGPR(calleeGPR));
+
+ callOperation(operationLinkDirectCall, callLinkInfo, calleeGPR);
+ m_jit.exceptionCheck();
+ m_jit.jump().linkTo(mainPath, &m_jit);
+
+ done.link(&m_jit);
+
+ setResultAndResetStack();
+
+ m_jit.addJSDirectCall(call, slowPath, callLinkInfo);
+ return;
+ }
+
+ m_jit.emitStoreCallSiteIndex(callSite);
+
</ins><span class="cx"> JITCompiler::DataLabelPtr targetToCheck;
</span><span class="cx"> JITCompiler::Jump slowPath = m_jit.branchPtrWithPatch(MacroAssembler::NotEqual, calleeGPR, targetToCheck, MacroAssembler::TrustedImmPtr(0));
</span><span class="cx">
</span><span class="lines">@@ -4955,6 +5070,10 @@
</span><span class="cx"> case TailCallForwardVarargs:
</span><span class="cx"> case TailCallForwardVarargsInlinedCaller:
</span><span class="cx"> case CallEval:
</span><ins>+ case DirectCall:
+ case DirectConstruct:
+ case DirectTailCall:
+ case DirectTailCallInlinedCaller:
</ins><span class="cx"> emitCall(node);
</span><span class="cx"> break;
</span><span class="cx">
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoredfgDFGStrengthReductionPhasecpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/dfg/DFGStrengthReductionPhase.cpp (207474 => 207475)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/dfg/DFGStrengthReductionPhase.cpp 2016-10-18 18:23:56 UTC (rev 207474)
+++ trunk/Source/JavaScriptCore/dfg/DFGStrengthReductionPhase.cpp 2016-10-18 18:30:05 UTC (rev 207475)
</span><span class="lines">@@ -760,6 +760,38 @@
</span><span class="cx"> m_node->origin = origin;
</span><span class="cx"> break;
</span><span class="cx"> }
</span><ins>+
+ case Call:
+ case Construct:
+ case TailCallInlinedCaller:
+ case TailCall: {
+ ExecutableBase* executable = nullptr;
+ Edge callee = m_graph.varArgChild(m_node, 0);
+ if (JSFunction* function = callee->dynamicCastConstant<JSFunction*>())
+ executable = function->executable();
+ else if (callee->isFunctionAllocation())
+ executable = callee->castOperand<FunctionExecutable*>();
+
+ if (!executable)
+ break;
+
+ if (FunctionExecutable* functionExecutable = jsDynamicCast<FunctionExecutable*>(executable)) {
+ // We need to update m_parameterSlots before we get to the backend, but we don't
+ // want to do too much of this.
+ unsigned numAllocatedArgs =
+ static_cast<unsigned>(functionExecutable->parameterCount()) + 1;
+
+ if (numAllocatedArgs <= Options::maximumDirectCallStackSize()) {
+ m_graph.m_parameterSlots = std::max(
+ m_graph.m_parameterSlots,
+ Graph::parameterSlotsForArgCount(numAllocatedArgs));
+ }
+ }
+
+ m_node->convertToDirectCall(m_graph.freeze(executable));
+ m_changed = true;
+ break;
+ }
</ins><span class="cx">
</span><span class="cx"> default:
</span><span class="cx"> break;
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreftlFTLCapabilitiescpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/ftl/FTLCapabilities.cpp (207474 => 207475)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/ftl/FTLCapabilities.cpp 2016-10-18 18:23:56 UTC (rev 207474)
+++ trunk/Source/JavaScriptCore/ftl/FTLCapabilities.cpp 2016-10-18 18:30:05 UTC (rev 207475)
</span><span class="lines">@@ -140,9 +140,13 @@
</span><span class="cx"> case StoreBarrier:
</span><span class="cx"> case FencedStoreBarrier:
</span><span class="cx"> case Call:
</span><ins>+ case DirectCall:
</ins><span class="cx"> case TailCall:
</span><ins>+ case DirectTailCall:
</ins><span class="cx"> case TailCallInlinedCaller:
</span><ins>+ case DirectTailCallInlinedCaller:
</ins><span class="cx"> case Construct:
</span><ins>+ case DirectConstruct:
</ins><span class="cx"> case CallVarargs:
</span><span class="cx"> case CallEval:
</span><span class="cx"> case TailCallVarargs:
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreftlFTLLowerDFGToB3cpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/ftl/FTLLowerDFGToB3.cpp (207474 => 207475)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/ftl/FTLLowerDFGToB3.cpp 2016-10-18 18:23:56 UTC (rev 207474)
+++ trunk/Source/JavaScriptCore/ftl/FTLLowerDFGToB3.cpp 2016-10-18 18:30:05 UTC (rev 207475)
</span><span class="lines">@@ -852,6 +852,12 @@
</span><span class="cx"> case Construct:
</span><span class="cx"> compileCallOrConstruct();
</span><span class="cx"> break;
</span><ins>+ case DirectCall:
+ case DirectTailCallInlinedCaller:
+ case DirectConstruct:
+ case DirectTailCall:
+ compileDirectCallOrConstruct();
+ break;
</ins><span class="cx"> case TailCall:
</span><span class="cx"> compileTailCall();
</span><span class="cx"> break;
</span><span class="lines">@@ -5555,8 +5561,8 @@
</span><span class="cx"> linkBuffer.link(slowCall, FunctionPtr(linkCall.executableAddress()));
</span><span class="cx">
</span><span class="cx"> callLinkInfo->setCallLocations(
</span><del>- linkBuffer.locationOfNearCall(slowCall),
- linkBuffer.locationOf(targetToCheck),
</del><ins>+ CodeLocationLabel(linkBuffer.locationOfNearCall(slowCall)),
+ CodeLocationLabel(linkBuffer.locationOf(targetToCheck)),
</ins><span class="cx"> linkBuffer.locationOfNearCall(fastCall));
</span><span class="cx"> });
</span><span class="cx"> });
</span><span class="lines">@@ -5563,7 +5569,207 @@
</span><span class="cx">
</span><span class="cx"> setJSValue(patchpoint);
</span><span class="cx"> }
</span><ins>+
+ void compileDirectCallOrConstruct()
+ {
+ Node* node = m_node;
+ bool isTail = node->op() == DirectTailCall;
+ bool isConstruct = node->op() == DirectConstruct;
+
+ ExecutableBase* executable = node->castOperand<ExecutableBase*>();
+ FunctionExecutable* functionExecutable = jsDynamicCast<FunctionExecutable*>(executable);
+
+ unsigned numPassedArgs = node->numChildren() - 1;
+ unsigned numAllocatedArgs = numPassedArgs;
+
+ if (functionExecutable) {
+ numAllocatedArgs = std::max(
+ numAllocatedArgs,
+ std::min(
+ static_cast<unsigned>(functionExecutable->parameterCount()) + 1,
+ Options::maximumDirectCallStackSize()));
+ }
+
+ LValue jsCallee = lowJSValue(m_graph.varArgChild(node, 0));
+
+ if (!isTail) {
+ unsigned frameSize = (CallFrame::headerSizeInRegisters + numAllocatedArgs) * sizeof(EncodedJSValue);
+ unsigned alignedFrameSize = WTF::roundUpToMultipleOf(stackAlignmentBytes(), frameSize);
+
+ m_proc.requestCallArgAreaSizeInBytes(alignedFrameSize);
+ }
+
+ Vector<ConstrainedValue> arguments;
+
+ arguments.append(ConstrainedValue(jsCallee, ValueRep::SomeRegister));
+ if (!isTail) {
+ auto addArgument = [&] (LValue value, VirtualRegister reg, int offset) {
+ intptr_t offsetFromSP =
+ (reg.offset() - CallerFrameAndPC::sizeInRegisters) * sizeof(EncodedJSValue) + offset;
+ arguments.append(ConstrainedValue(value, ValueRep::stackArgument(offsetFromSP)));
+ };
+
+ addArgument(jsCallee, VirtualRegister(CallFrameSlot::callee), 0);
+ addArgument(m_out.constInt32(numPassedArgs), VirtualRegister(CallFrameSlot::argumentCount), PayloadOffset);
+ for (unsigned i = 0; i < numPassedArgs; ++i)
+ addArgument(lowJSValue(m_graph.varArgChild(node, 1 + i)), virtualRegisterForArgument(i), 0);
+ for (unsigned i = numPassedArgs; i < numAllocatedArgs; ++i)
+ addArgument(m_out.constInt64(JSValue::encode(jsUndefined())), virtualRegisterForArgument(i), 0);
+ } else {
+ for (unsigned i = 0; i < numPassedArgs; ++i)
+ arguments.append(ConstrainedValue(lowJSValue(m_graph.varArgChild(node, 1 + i)), ValueRep::WarmAny));
+ }
+
+ PatchpointValue* patchpoint = m_out.patchpoint(isTail ? Void : Int64);
+ patchpoint->appendVector(arguments);
+
+ RefPtr<PatchpointExceptionHandle> exceptionHandle = preparePatchpointForExceptions(patchpoint);
+
+ if (isTail) {
+ // The shuffler needs tags.
+ patchpoint->append(m_tagMask, ValueRep::reg(GPRInfo::tagMaskRegister));
+ patchpoint->append(m_tagTypeNumber, ValueRep::reg(GPRInfo::tagTypeNumberRegister));
+ }
+
+ patchpoint->clobber(RegisterSet::macroScratchRegisters());
+ if (!isTail) {
+ patchpoint->clobberLate(RegisterSet::volatileRegistersForJSCall());
+ patchpoint->resultConstraint = ValueRep::reg(GPRInfo::returnValueGPR);
+ }
+
+ CodeOrigin codeOrigin = codeOriginDescriptionOfCallSite();
+ State* state = &m_ftlState;
+ patchpoint->setGenerator(
+ [=] (CCallHelpers& jit, const StackmapGenerationParams& params) {
+ AllowMacroScratchRegisterUsage allowScratch(jit);
+ CallSiteIndex callSiteIndex = state->jitCode->common.addUniqueCallSiteIndex(codeOrigin);
+
+ GPRReg calleeGPR = params[!isTail].gpr();
</ins><span class="cx">
</span><ins>+ exceptionHandle->scheduleExitCreationForUnwind(params, callSiteIndex);
+
+ Box<CCallHelpers::JumpList> exceptions =
+ exceptionHandle->scheduleExitCreation(params)->jumps(jit);
+
+ if (isTail) {
+ CallFrameShuffleData shuffleData;
+ shuffleData.numLocals = state->jitCode->common.frameRegisterCount;
+
+ RegisterSet toSave = params.unavailableRegisters();
+ shuffleData.callee = ValueRecovery::inGPR(calleeGPR, DataFormatCell);
+ toSave.set(calleeGPR);
+ for (unsigned i = 0; i < numPassedArgs; ++i) {
+ ValueRecovery recovery = params[1 + i].recoveryForJSValue();
+ shuffleData.args.append(recovery);
+ recovery.forEachReg(
+ [&] (Reg reg) {
+ toSave.set(reg);
+ });
+ }
+ for (unsigned i = numPassedArgs; i < numAllocatedArgs; ++i)
+ shuffleData.args.append(ValueRecovery::constant(jsUndefined()));
+ shuffleData.setupCalleeSaveRegisters(jit.codeBlock());
+
+ CallLinkInfo* callLinkInfo = jit.codeBlock()->addCallLinkInfo();
+
+ CCallHelpers::PatchableJump patchableJump = jit.patchableJump();
+ CCallHelpers::Label mainPath = jit.label();
+
+ jit.store32(
+ CCallHelpers::TrustedImm32(callSiteIndex.bits()),
+ CCallHelpers::tagFor(VirtualRegister(CallFrameSlot::argumentCount)));
+
+ callLinkInfo->setFrameShuffleData(shuffleData);
+ CallFrameShuffler(jit, shuffleData).prepareForTailCall();
+
+ CCallHelpers::Call call = jit.nearTailCall();
+
+ jit.abortWithReason(JITDidReturnFromTailCall);
+
+ CCallHelpers::Label slowPath = jit.label();
+ patchableJump.m_jump.linkTo(slowPath, &jit);
+ callOperation(
+ *state, toSave, jit,
+ node->origin.semantic, exceptions.get(), operationLinkDirectCall,
+ InvalidGPRReg, CCallHelpers::TrustedImmPtr(callLinkInfo), calleeGPR).call();
+ jit.jump().linkTo(mainPath, &jit);
+
+ callLinkInfo->setUpCall(
+ CallLinkInfo::DirectTailCall, node->origin.semantic, InvalidGPRReg);
+ callLinkInfo->setExecutableDuringCompilation(executable);
+ if (numAllocatedArgs > numPassedArgs)
+ callLinkInfo->setMaxNumArguments(numAllocatedArgs);
+
+ jit.addLinkTask(
+ [=] (LinkBuffer& linkBuffer) {
+ CodeLocationLabel patchableJumpLocation = linkBuffer.locationOf(patchableJump);
+ CodeLocationNearCall callLocation = linkBuffer.locationOfNearCall(call);
+ CodeLocationLabel slowPathLocation = linkBuffer.locationOf(slowPath);
+
+ callLinkInfo->setCallLocations(
+ patchableJumpLocation,
+ slowPathLocation,
+ callLocation);
+ });
+ return;
+ }
+
+ CallLinkInfo* callLinkInfo = jit.codeBlock()->addCallLinkInfo();
+
+ CCallHelpers::Label mainPath = jit.label();
+
+ jit.store32(
+ CCallHelpers::TrustedImm32(callSiteIndex.bits()),
+ CCallHelpers::tagFor(VirtualRegister(CallFrameSlot::argumentCount)));
+
+ CCallHelpers::Call call = jit.nearCall();
+ jit.addPtr(
+ CCallHelpers::TrustedImm32(-params.proc().frameSize()),
+ GPRInfo::callFrameRegister, CCallHelpers::stackPointerRegister);
+
+ callLinkInfo->setUpCall(
+ isConstruct ? CallLinkInfo::DirectConstruct : CallLinkInfo::DirectCall,
+ node->origin.semantic, InvalidGPRReg);
+ callLinkInfo->setExecutableDuringCompilation(executable);
+ if (numAllocatedArgs > numPassedArgs)
+ callLinkInfo->setMaxNumArguments(numAllocatedArgs);
+
+ params.addLatePath(
+ [=] (CCallHelpers& jit) {
+ AllowMacroScratchRegisterUsage allowScratch(jit);
+
+ CCallHelpers::Label slowPath = jit.label();
+ if (isX86())
+ jit.pop(CCallHelpers::selectScratchGPR(calleeGPR));
+
+ callOperation(
+ *state, params.unavailableRegisters(), jit,
+ node->origin.semantic, exceptions.get(), operationLinkDirectCall,
+ InvalidGPRReg, CCallHelpers::TrustedImmPtr(callLinkInfo),
+ calleeGPR).call();
+ jit.jump().linkTo(mainPath, &jit);
+
+ jit.addLinkTask(
+ [=] (LinkBuffer& linkBuffer) {
+ CodeLocationNearCall callLocation = linkBuffer.locationOfNearCall(call);
+ CodeLocationLabel slowPathLocation = linkBuffer.locationOf(slowPath);
+
+ linkBuffer.link(call, slowPathLocation);
+
+ callLinkInfo->setCallLocations(
+ CodeLocationLabel(),
+ slowPathLocation,
+ callLocation);
+ });
+ });
+ });
+
+ if (isTail)
+ patchpoint->effects.terminal = true;
+ else
+ setJSValue(patchpoint);
+ }
+
</ins><span class="cx"> void compileTailCall()
</span><span class="cx"> {
</span><span class="cx"> Node* node = m_node;
</span><span class="lines">@@ -5601,6 +5807,8 @@
</span><span class="cx">
</span><span class="cx"> // Prevent any of the arguments from using the scratch register.
</span><span class="cx"> patchpoint->clobberEarly(RegisterSet::macroScratchRegisters());
</span><ins>+
+ patchpoint->effects.terminal = true;
</ins><span class="cx">
</span><span class="cx"> // We don't have to tell the patchpoint that we will clobber registers, since we won't return
</span><span class="cx"> // anyway.
</span><span class="lines">@@ -5660,13 +5868,11 @@
</span><span class="cx"> linkBuffer.link(slowCall, FunctionPtr(linkCall.executableAddress()));
</span><span class="cx">
</span><span class="cx"> callLinkInfo->setCallLocations(
</span><del>- linkBuffer.locationOfNearCall(slowCall),
- linkBuffer.locationOf(targetToCheck),
</del><ins>+ CodeLocationLabel(linkBuffer.locationOfNearCall(slowCall)),
+ CodeLocationLabel(linkBuffer.locationOf(targetToCheck)),
</ins><span class="cx"> linkBuffer.locationOfNearCall(fastCall));
</span><span class="cx"> });
</span><span class="cx"> });
</span><del>-
- m_out.unreachable();
</del><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> void compileCallOrConstructVarargs()
</span><span class="lines">@@ -5929,8 +6135,8 @@
</span><span class="cx"> linkBuffer.link(slowCall, FunctionPtr(linkCall.executableAddress()));
</span><span class="cx">
</span><span class="cx"> callLinkInfo->setCallLocations(
</span><del>- linkBuffer.locationOfNearCall(slowCall),
- linkBuffer.locationOf(targetToCheck),
</del><ins>+ CodeLocationLabel(linkBuffer.locationOfNearCall(slowCall)),
+ CodeLocationLabel(linkBuffer.locationOf(targetToCheck)),
</ins><span class="cx"> linkBuffer.locationOfNearCall(fastCall));
</span><span class="cx"> });
</span><span class="cx"> });
</span><span class="lines">@@ -12010,7 +12216,8 @@
</span><span class="cx"> CodeOrigin codeOrigin = m_node->origin.semantic;
</span><span class="cx"> if (m_node->op() == TailCallInlinedCaller
</span><span class="cx"> || m_node->op() == TailCallVarargsInlinedCaller
</span><del>- || m_node->op() == TailCallForwardVarargsInlinedCaller) {
</del><ins>+ || m_node->op() == TailCallForwardVarargsInlinedCaller
+ || m_node->op() == DirectTailCallInlinedCaller) {
</ins><span class="cx"> // This case arises when you have a situation like this:
</span><span class="cx"> // foo makes a call to bar, bar is inlined in foo. bar makes a call
</span><span class="cx"> // to baz and baz is inlined in bar. And then baz makes a tail-call to jaz,
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreinterpreterInterpretercpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/interpreter/Interpreter.cpp (207474 => 207475)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/interpreter/Interpreter.cpp 2016-10-18 18:23:56 UTC (rev 207474)
+++ trunk/Source/JavaScriptCore/interpreter/Interpreter.cpp 2016-10-18 18:30:05 UTC (rev 207475)
</span><span class="lines">@@ -870,7 +870,7 @@
</span><span class="cx"> ProgramCodeBlock* codeBlock;
</span><span class="cx"> {
</span><span class="cx"> CodeBlock* tempCodeBlock;
</span><del>- JSObject* error = program->prepareForExecution<ProgramExecutable>(callFrame, nullptr, scope, CodeForCall, tempCodeBlock);
</del><ins>+ JSObject* error = program->prepareForExecution<ProgramExecutable>(vm, nullptr, scope, CodeForCall, tempCodeBlock);
</ins><span class="cx"> ASSERT(!throwScope.exception() || throwScope.exception() == jsDynamicCast<Exception*>(error));
</span><span class="cx"> if (error)
</span><span class="cx"> return checkedReturn(throwException(callFrame, throwScope, error));
</span><span class="lines">@@ -925,7 +925,7 @@
</span><span class="cx">
</span><span class="cx"> if (isJSCall) {
</span><span class="cx"> // Compile the callee:
</span><del>- JSObject* compileError = callData.js.functionExecutable->prepareForExecution<FunctionExecutable>(callFrame, jsCast<JSFunction*>(function), scope, CodeForCall, newCodeBlock);
</del><ins>+ JSObject* compileError = callData.js.functionExecutable->prepareForExecution<FunctionExecutable>(vm, jsCast<JSFunction*>(function), scope, CodeForCall, newCodeBlock);
</ins><span class="cx"> ASSERT(throwScope.exception() == reinterpret_cast<Exception*>(compileError));
</span><span class="cx"> if (UNLIKELY(!!compileError))
</span><span class="cx"> return checkedReturn(throwException(callFrame, throwScope, compileError));
</span><span class="lines">@@ -989,7 +989,7 @@
</span><span class="cx">
</span><span class="cx"> if (isJSConstruct) {
</span><span class="cx"> // Compile the callee:
</span><del>- JSObject* compileError = constructData.js.functionExecutable->prepareForExecution<FunctionExecutable>(callFrame, jsCast<JSFunction*>(constructor), scope, CodeForConstruct, newCodeBlock);
</del><ins>+ JSObject* compileError = constructData.js.functionExecutable->prepareForExecution<FunctionExecutable>(vm, jsCast<JSFunction*>(constructor), scope, CodeForConstruct, newCodeBlock);
</ins><span class="cx"> if (UNLIKELY(!!compileError))
</span><span class="cx"> return checkedReturn(throwException(callFrame, throwScope, compileError));
</span><span class="cx">
</span><span class="lines">@@ -1033,7 +1033,7 @@
</span><span class="cx">
</span><span class="cx"> // Compile the callee:
</span><span class="cx"> CodeBlock* newCodeBlock;
</span><del>- JSObject* error = functionExecutable->prepareForExecution<FunctionExecutable>(callFrame, function, scope, CodeForCall, newCodeBlock);
</del><ins>+ JSObject* error = functionExecutable->prepareForExecution<FunctionExecutable>(vm, function, scope, CodeForCall, newCodeBlock);
</ins><span class="cx"> if (error) {
</span><span class="cx"> throwException(callFrame, throwScope, error);
</span><span class="cx"> return CallFrameClosure();
</span><span class="lines">@@ -1112,7 +1112,7 @@
</span><span class="cx"> EvalCodeBlock* codeBlock;
</span><span class="cx"> {
</span><span class="cx"> CodeBlock* tempCodeBlock;
</span><del>- JSObject* compileError = eval->prepareForExecution<EvalExecutable>(callFrame, nullptr, scope, CodeForCall, tempCodeBlock);
</del><ins>+ JSObject* compileError = eval->prepareForExecution<EvalExecutable>(vm, nullptr, scope, CodeForCall, tempCodeBlock);
</ins><span class="cx"> if (UNLIKELY(!!compileError))
</span><span class="cx"> return checkedReturn(throwException(callFrame, throwScope, compileError));
</span><span class="cx"> codeBlock = jsCast<EvalCodeBlock*>(tempCodeBlock);
</span><span class="lines">@@ -1194,7 +1194,7 @@
</span><span class="cx"> ModuleProgramCodeBlock* codeBlock;
</span><span class="cx"> {
</span><span class="cx"> CodeBlock* tempCodeBlock;
</span><del>- JSObject* compileError = executable->prepareForExecution<ModuleProgramExecutable>(callFrame, nullptr, scope, CodeForCall, tempCodeBlock);
</del><ins>+ JSObject* compileError = executable->prepareForExecution<ModuleProgramExecutable>(vm, nullptr, scope, CodeForCall, tempCodeBlock);
</ins><span class="cx"> if (UNLIKELY(!!compileError))
</span><span class="cx"> return checkedReturn(throwException(callFrame, throwScope, compileError));
</span><span class="cx"> codeBlock = jsCast<ModuleProgramCodeBlock*>(tempCodeBlock);
</span></span></pre></div>
<a id="trunkSourceJavaScriptCorejitJITcpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/jit/JIT.cpp (207474 => 207475)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/jit/JIT.cpp 2016-10-18 18:23:56 UTC (rev 207474)
+++ trunk/Source/JavaScriptCore/jit/JIT.cpp 2016-10-18 18:30:05 UTC (rev 207475)
</span><span class="lines">@@ -760,8 +760,9 @@
</span><span class="cx"> for (unsigned i = 0; i < m_callCompilationInfo.size(); ++i) {
</span><span class="cx"> CallCompilationInfo& compilationInfo = m_callCompilationInfo[i];
</span><span class="cx"> CallLinkInfo& info = *compilationInfo.callLinkInfo;
</span><del>- info.setCallLocations(patchBuffer.locationOfNearCall(compilationInfo.callReturnLocation),
- patchBuffer.locationOf(compilationInfo.hotPathBegin),
</del><ins>+ info.setCallLocations(
+ CodeLocationLabel(patchBuffer.locationOfNearCall(compilationInfo.callReturnLocation)),
+ CodeLocationLabel(patchBuffer.locationOf(compilationInfo.hotPathBegin)),
</ins><span class="cx"> patchBuffer.locationOfNearCall(compilationInfo.hotPathOther));
</span><span class="cx"> }
</span><span class="cx">
</span></span></pre></div>
<a id="trunkSourceJavaScriptCorejitJITCallcpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/jit/JITCall.cpp (207474 => 207475)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/jit/JITCall.cpp 2016-10-18 18:23:56 UTC (rev 207474)
+++ trunk/Source/JavaScriptCore/jit/JITCall.cpp 2016-10-18 18:30:05 UTC (rev 207475)
</span><span class="lines">@@ -81,12 +81,9 @@
</span><span class="cx">
</span><span class="cx"> // Profile the argument count.
</span><span class="cx"> load32(Address(regT1, CallFrameSlot::argumentCount * static_cast<int>(sizeof(Register)) + PayloadOffset), regT2);
</span><del>- load8(info->addressOfMaxNumArguments(), regT0);
</del><ins>+ load32(info->addressOfMaxNumArguments(), regT0);
</ins><span class="cx"> Jump notBiggest = branch32(Above, regT0, regT2);
</span><del>- Jump notSaturated = branch32(BelowOrEqual, regT2, TrustedImm32(255));
- move(TrustedImm32(255), regT2);
- notSaturated.link(this);
- store8(regT2, info->addressOfMaxNumArguments());
</del><ins>+ store32(regT2, info->addressOfMaxNumArguments());
</ins><span class="cx"> notBiggest.link(this);
</span><span class="cx">
</span><span class="cx"> // Initialize 'this'.
</span></span></pre></div>
<a id="trunkSourceJavaScriptCorejitJITCall32_64cpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/jit/JITCall32_64.cpp (207474 => 207475)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/jit/JITCall32_64.cpp 2016-10-18 18:23:56 UTC (rev 207474)
+++ trunk/Source/JavaScriptCore/jit/JITCall32_64.cpp 2016-10-18 18:30:05 UTC (rev 207475)
</span><span class="lines">@@ -171,12 +171,9 @@
</span><span class="cx">
</span><span class="cx"> // Profile the argument count.
</span><span class="cx"> load32(Address(regT1, CallFrameSlot::argumentCount * static_cast<int>(sizeof(Register)) + PayloadOffset), regT2);
</span><del>- load8(info->addressOfMaxNumArguments(), regT0);
</del><ins>+ load32(info->addressOfMaxNumArguments(), regT0);
</ins><span class="cx"> Jump notBiggest = branch32(Above, regT0, regT2);
</span><del>- Jump notSaturated = branch32(BelowOrEqual, regT2, TrustedImm32(255));
- move(TrustedImm32(255), regT2);
- notSaturated.link(this);
- store8(regT2, info->addressOfMaxNumArguments());
</del><ins>+ store32(regT2, info->addressOfMaxNumArguments());
</ins><span class="cx"> notBiggest.link(this);
</span><span class="cx">
</span><span class="cx"> // Initialize 'this'.
</span></span></pre></div>
<a id="trunkSourceJavaScriptCorejitJITOperationscpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/jit/JITOperations.cpp (207474 => 207475)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/jit/JITOperations.cpp 2016-10-18 18:23:56 UTC (rev 207474)
+++ trunk/Source/JavaScriptCore/jit/JITOperations.cpp 2016-10-18 18:30:05 UTC (rev 207475)
</span><span class="lines">@@ -861,6 +861,8 @@
</span><span class="cx"> CodeSpecializationKind kind = callLinkInfo->specializationKind();
</span><span class="cx"> NativeCallFrameTracer tracer(vm, exec);
</span><span class="cx">
</span><ins>+ RELEASE_ASSERT(!callLinkInfo->isDirect());
+
</ins><span class="cx"> JSValue calleeAsValue = execCallee->calleeAsValue();
</span><span class="cx"> JSCell* calleeAsFunctionCell = getJSFunction(calleeAsValue);
</span><span class="cx"> if (!calleeAsFunctionCell) {
</span><span class="lines">@@ -902,7 +904,7 @@
</span><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> CodeBlock** codeBlockSlot = execCallee->addressOfCodeBlock();
</span><del>- JSObject* error = functionExecutable->prepareForExecution<FunctionExecutable>(execCallee, callee, scope, kind, *codeBlockSlot);
</del><ins>+ JSObject* error = functionExecutable->prepareForExecution<FunctionExecutable>(*vm, callee, scope, kind, *codeBlockSlot);
</ins><span class="cx"> ASSERT(throwScope.exception() == reinterpret_cast<Exception*>(error));
</span><span class="cx"> if (error) {
</span><span class="cx"> throwException(exec, throwScope, error);
</span><span class="lines">@@ -926,6 +928,60 @@
</span><span class="cx"> return encodeResult(codePtr.executableAddress(), reinterpret_cast<void*>(callLinkInfo->callMode() == CallMode::Tail ? ReuseTheFrame : KeepTheFrame));
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+void JIT_OPERATION operationLinkDirectCall(ExecState* exec, CallLinkInfo* callLinkInfo, JSFunction* callee)
+{
+ VM* vm = &exec->vm();
+ auto throwScope = DECLARE_THROW_SCOPE(*vm);
+
+ CodeSpecializationKind kind = callLinkInfo->specializationKind();
+ NativeCallFrameTracer tracer(vm, exec);
+
+ RELEASE_ASSERT(callLinkInfo->isDirect());
+
+ // This would happen if the executable died during GC but the CodeBlock did not die. That should
+ // not happen because the CodeBlock should have a weak reference to any executable it uses for
+ // this purpose.
+ RELEASE_ASSERT(callLinkInfo->executable());
+
+ // Having a CodeBlock indicates that this is linked. We shouldn't be taking this path if it's
+ // linked.
+ RELEASE_ASSERT(!callLinkInfo->codeBlock());
+
+ // We just don't support this yet.
+ RELEASE_ASSERT(!callLinkInfo->isVarargs());
+
+ ExecutableBase* executable = callLinkInfo->executable();
+ RELEASE_ASSERT(callee->executable() == callLinkInfo->executable());
+
+ JSScope* scope = callee->scopeUnchecked();
+
+ MacroAssemblerCodePtr codePtr;
+ CodeBlock* codeBlock = nullptr;
+ if (executable->isHostFunction())
+ codePtr = executable->entrypointFor(kind, MustCheckArity);
+ else {
+ FunctionExecutable* functionExecutable = static_cast<FunctionExecutable*>(executable);
+
+ RELEASE_ASSERT(isCall(kind) || functionExecutable->constructAbility() != ConstructAbility::CannotConstruct);
+
+ JSObject* error = functionExecutable->prepareForExecution<FunctionExecutable>(*vm, callee, scope, kind, codeBlock);
+ ASSERT(throwScope.exception() == reinterpret_cast<Exception*>(error));
+ if (error) {
+ throwException(exec, throwScope, error);
+ return;
+ }
+ ArityCheckMode arity;
+ unsigned argumentStackSlots = callLinkInfo->maxNumArguments();
+ if (argumentStackSlots < static_cast<size_t>(codeBlock->numParameters()))
+ arity = MustCheckArity;
+ else
+ arity = ArityCheckNotRequired;
+ codePtr = functionExecutable->entrypointFor(kind, arity);
+ }
+
+ linkDirectFor(exec, *callLinkInfo, codeBlock, codePtr);
+}
+
</ins><span class="cx"> inline SlowPathReturnType virtualForWithFunction(
</span><span class="cx"> ExecState* execCallee, CallLinkInfo* callLinkInfo, JSCell*& calleeAsFunctionCell)
</span><span class="cx"> {
</span><span class="lines">@@ -960,7 +1016,7 @@
</span><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> CodeBlock** codeBlockSlot = execCallee->addressOfCodeBlock();
</span><del>- JSObject* error = functionExecutable->prepareForExecution<FunctionExecutable>(execCallee, function, scope, kind, *codeBlockSlot);
</del><ins>+ JSObject* error = functionExecutable->prepareForExecution<FunctionExecutable>(*vm, function, scope, kind, *codeBlockSlot);
</ins><span class="cx"> if (error) {
</span><span class="cx"> throwException(exec, throwScope, error);
</span><span class="cx"> return encodeResult(
</span></span></pre></div>
<a id="trunkSourceJavaScriptCorejitJITOperationsh"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/jit/JITOperations.h (207474 => 207475)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/jit/JITOperations.h 2016-10-18 18:23:56 UTC (rev 207474)
+++ trunk/Source/JavaScriptCore/jit/JITOperations.h 2016-10-18 18:30:05 UTC (rev 207475)
</span><span class="lines">@@ -255,6 +255,7 @@
</span><span class="cx"> typedef void (JIT_OPERATION *V_JITOperation_ECPSPS)(ExecState*, JSCell*, void*, size_t, void*, size_t);
</span><span class="cx"> typedef void (JIT_OPERATION *V_JITOperation_ECZ)(ExecState*, JSCell*, int32_t);
</span><span class="cx"> typedef void (JIT_OPERATION *V_JITOperation_ECC)(ExecState*, JSCell*, JSCell*);
</span><ins>+typedef void (JIT_OPERATION *V_JITOperation_ECliJsf)(ExecState*, CallLinkInfo*, JSFunction*);
</ins><span class="cx"> typedef void (JIT_OPERATION *V_JITOperation_EZSymtabJ)(ExecState*, int32_t, SymbolTable*, EncodedJSValue);
</span><span class="cx"> typedef void (JIT_OPERATION *V_JITOperation_EJ)(ExecState*, EncodedJSValue);
</span><span class="cx"> typedef void (JIT_OPERATION *V_JITOperation_EJCI)(ExecState*, EncodedJSValue, JSCell*, UniquedStringImpl*);
</span><span class="lines">@@ -353,6 +354,7 @@
</span><span class="cx"> void JIT_OPERATION operationDirectPutByValGeneric(ExecState*, EncodedJSValue, EncodedJSValue, EncodedJSValue, ByValInfo*) WTF_INTERNAL;
</span><span class="cx"> EncodedJSValue JIT_OPERATION operationCallEval(ExecState*, ExecState*) WTF_INTERNAL;
</span><span class="cx"> SlowPathReturnType JIT_OPERATION operationLinkCall(ExecState*, CallLinkInfo*) WTF_INTERNAL;
</span><ins>+void JIT_OPERATION operationLinkDirectCall(ExecState*, CallLinkInfo*, JSFunction*) WTF_INTERNAL;
</ins><span class="cx"> SlowPathReturnType JIT_OPERATION operationLinkPolymorphicCall(ExecState*, CallLinkInfo*) WTF_INTERNAL;
</span><span class="cx"> SlowPathReturnType JIT_OPERATION operationVirtualCall(ExecState*, CallLinkInfo*) WTF_INTERNAL;
</span><span class="cx">
</span></span></pre></div>
<a id="trunkSourceJavaScriptCorejitRepatchcpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/jit/Repatch.cpp (207474 => 207475)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/jit/Repatch.cpp 2016-10-18 18:23:56 UTC (rev 207474)
+++ trunk/Source/JavaScriptCore/jit/Repatch.cpp 2016-10-18 18:30:05 UTC (rev 207475)
</span><span class="lines">@@ -583,6 +583,28 @@
</span><span class="cx"> linkSlowFor(vm, callLinkInfo);
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+void linkDirectFor(
+ ExecState* exec, CallLinkInfo& callLinkInfo, CodeBlock* calleeCodeBlock,
+ MacroAssemblerCodePtr codePtr)
+{
+ ASSERT(!callLinkInfo.stub());
+
+ CodeBlock* callerCodeBlock = exec->codeBlock();
+
+ VM* vm = callerCodeBlock->vm();
+
+ ASSERT(!callLinkInfo.isLinked());
+ callLinkInfo.setCodeBlock(*vm, callerCodeBlock, jsCast<FunctionCodeBlock*>(calleeCodeBlock));
+ if (shouldDumpDisassemblyFor(callerCodeBlock))
+ dataLog("Linking call in ", *callerCodeBlock, " at ", callLinkInfo.codeOrigin(), " to ", pointerDump(calleeCodeBlock), ", entrypoint at ", codePtr, "\n");
+ if (callLinkInfo.callType() == CallLinkInfo::DirectTailCall)
+ MacroAssembler::repatchJumpToNop(callLinkInfo.patchableJump());
+ MacroAssembler::repatchNearCall(callLinkInfo.hotPathOther(), CodeLocationLabel(codePtr));
+
+ if (calleeCodeBlock)
+ calleeCodeBlock->linkIncomingCall(exec, &callLinkInfo);
+}
+
</ins><span class="cx"> void linkSlowFor(
</span><span class="cx"> ExecState* exec, CallLinkInfo& callLinkInfo)
</span><span class="cx"> {
</span><span class="lines">@@ -594,12 +616,20 @@
</span><span class="cx">
</span><span class="cx"> static void revertCall(VM* vm, CallLinkInfo& callLinkInfo, MacroAssemblerCodeRef codeRef)
</span><span class="cx"> {
</span><del>- MacroAssembler::revertJumpReplacementToBranchPtrWithPatch(
- MacroAssembler::startOfBranchPtrWithPatchOnRegister(callLinkInfo.hotPathBegin()),
- static_cast<MacroAssembler::RegisterID>(callLinkInfo.calleeGPR()), 0);
- linkSlowFor(vm, callLinkInfo, codeRef);
</del><ins>+ if (callLinkInfo.isDirect()) {
+ callLinkInfo.clearCodeBlock();
+ if (callLinkInfo.callType() == CallLinkInfo::DirectTailCall)
+ MacroAssembler::repatchJump(callLinkInfo.patchableJump(), callLinkInfo.slowPathStart());
+ else
+ MacroAssembler::repatchNearCall(callLinkInfo.hotPathOther(), callLinkInfo.slowPathStart());
+ } else {
+ MacroAssembler::revertJumpReplacementToBranchPtrWithPatch(
+ MacroAssembler::startOfBranchPtrWithPatchOnRegister(callLinkInfo.hotPathBegin()),
+ static_cast<MacroAssembler::RegisterID>(callLinkInfo.calleeGPR()), 0);
+ linkSlowFor(vm, callLinkInfo, codeRef);
+ callLinkInfo.clearCallee();
+ }
</ins><span class="cx"> callLinkInfo.clearSeen();
</span><del>- callLinkInfo.clearCallee();
</del><span class="cx"> callLinkInfo.clearStub();
</span><span class="cx"> callLinkInfo.clearSlowStub();
</span><span class="cx"> if (callLinkInfo.isOnList())
</span><span class="lines">@@ -609,7 +639,7 @@
</span><span class="cx"> void unlinkFor(VM& vm, CallLinkInfo& callLinkInfo)
</span><span class="cx"> {
</span><span class="cx"> if (Options::dumpDisassembly())
</span><del>- dataLog("Unlinking call from ", callLinkInfo.callReturnLocation(), "\n");
</del><ins>+ dataLog("Unlinking call at ", callLinkInfo.hotPathOther(), "\n");
</ins><span class="cx">
</span><span class="cx"> revertCall(&vm, callLinkInfo, vm.getCTIStub(linkCallThunkGenerator));
</span><span class="cx"> }
</span></span></pre></div>
<a id="trunkSourceJavaScriptCorejitRepatchh"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/jit/Repatch.h (207474 => 207475)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/jit/Repatch.h 2016-10-18 18:23:56 UTC (rev 207474)
+++ trunk/Source/JavaScriptCore/jit/Repatch.h 2016-10-18 18:30:05 UTC (rev 207475)
</span><span class="lines">@@ -46,6 +46,7 @@
</span><span class="cx"> void buildPutByIdList(ExecState*, JSValue, Structure*, const Identifier&, const PutPropertySlot&, StructureStubInfo&, PutKind);
</span><span class="cx"> void repatchIn(ExecState*, JSCell*, const Identifier&, bool wasFound, const PropertySlot&, StructureStubInfo&);
</span><span class="cx"> void linkFor(ExecState*, CallLinkInfo&, CodeBlock*, JSFunction* callee, MacroAssemblerCodePtr);
</span><ins>+void linkDirectFor(ExecState*, CallLinkInfo&, CodeBlock*, MacroAssemblerCodePtr);
</ins><span class="cx"> void linkSlowFor(ExecState*, CallLinkInfo&);
</span><span class="cx"> void unlinkFor(VM&, CallLinkInfo&);
</span><span class="cx"> void linkVirtualFor(ExecState*, CallLinkInfo&);
</span></span></pre></div>
<a id="trunkSourceJavaScriptCorellintLLIntSlowPathscpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/llint/LLIntSlowPaths.cpp (207474 => 207475)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/llint/LLIntSlowPaths.cpp 2016-10-18 18:23:56 UTC (rev 207474)
+++ trunk/Source/JavaScriptCore/llint/LLIntSlowPaths.cpp 2016-10-18 18:30:05 UTC (rev 207475)
</span><span class="lines">@@ -1275,7 +1275,7 @@
</span><span class="cx"> LLINT_CALL_THROW(exec, createNotAConstructorError(exec, callee));
</span><span class="cx">
</span><span class="cx"> CodeBlock** codeBlockSlot = execCallee->addressOfCodeBlock();
</span><del>- JSObject* error = functionExecutable->prepareForExecution<FunctionExecutable>(execCallee, callee, scope, kind, *codeBlockSlot);
</del><ins>+ JSObject* error = functionExecutable->prepareForExecution<FunctionExecutable>(vm, callee, scope, kind, *codeBlockSlot);
</ins><span class="cx"> if (error)
</span><span class="cx"> LLINT_CALL_THROW(exec, error);
</span><span class="cx"> codeBlock = *codeBlockSlot;
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreruntimeExecutablecpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/runtime/Executable.cpp (207474 => 207475)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/runtime/Executable.cpp 2016-10-18 18:23:56 UTC (rev 207474)
+++ trunk/Source/JavaScriptCore/runtime/Executable.cpp 2016-10-18 18:30:05 UTC (rev 207475)
</span><span class="lines">@@ -398,13 +398,12 @@
</span><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> JSObject* ScriptExecutable::prepareForExecutionImpl(
</span><del>- ExecState* exec, JSFunction* function, JSScope* scope, CodeSpecializationKind kind, CodeBlock*& resultCodeBlock)
</del><ins>+ VM& vm, JSFunction* function, JSScope* scope, CodeSpecializationKind kind, CodeBlock*& resultCodeBlock)
</ins><span class="cx"> {
</span><del>- VM& vm = exec->vm();
</del><span class="cx"> DeferGCForAWhile deferGC(vm.heap);
</span><span class="cx">
</span><span class="cx"> if (vm.getAndClearFailNextNewCodeBlock())
</span><del>- return createError(exec->callerFrame(), ASCIILiteral("Forced Failure"));
</del><ins>+ return createError(scope->globalObject()->globalExec(), ASCIILiteral("Forced Failure"));
</ins><span class="cx">
</span><span class="cx"> JSObject* exception = 0;
</span><span class="cx"> CodeBlock* codeBlock = newCodeBlockFor(kind, function, scope, exception);
</span><span class="lines">@@ -422,7 +421,7 @@
</span><span class="cx"> else
</span><span class="cx"> setupJIT(vm, codeBlock);
</span><span class="cx">
</span><del>- installCode(*codeBlock->vm(), codeBlock, codeBlock->codeType(), codeBlock->specializationKind());
</del><ins>+ installCode(vm, codeBlock, codeBlock->codeType(), codeBlock->specializationKind());
</ins><span class="cx"> return nullptr;
</span><span class="cx"> }
</span><span class="cx">
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreruntimeExecutableh"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/runtime/Executable.h (207474 => 207475)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/runtime/Executable.h 2016-10-18 18:23:56 UTC (rev 207474)
+++ trunk/Source/JavaScriptCore/runtime/Executable.h 2016-10-18 18:30:05 UTC (rev 207475)
</span><span class="lines">@@ -371,13 +371,13 @@
</span><span class="cx"> // to point to it. This forces callers to have a CodeBlock* in a register or on the stack that will be marked
</span><span class="cx"> // by conservative GC if a GC happens after we create the CodeBlock.
</span><span class="cx"> template <typename ExecutableType>
</span><del>- JSObject* prepareForExecution(ExecState*, JSFunction*, JSScope*, CodeSpecializationKind, CodeBlock*& resultCodeBlock);
</del><ins>+ JSObject* prepareForExecution(VM&, JSFunction*, JSScope*, CodeSpecializationKind, CodeBlock*& resultCodeBlock);
</ins><span class="cx">
</span><span class="cx"> template <typename Functor> void forEachCodeBlock(Functor&&);
</span><span class="cx">
</span><span class="cx"> private:
</span><span class="cx"> friend class ExecutableBase;
</span><del>- JSObject* prepareForExecutionImpl(ExecState*, JSFunction*, JSScope*, CodeSpecializationKind, CodeBlock*&);
</del><ins>+ JSObject* prepareForExecutionImpl(VM&, JSFunction*, JSScope*, CodeSpecializationKind, CodeBlock*&);
</ins><span class="cx">
</span><span class="cx"> protected:
</span><span class="cx"> ScriptExecutable(Structure*, VM&, const SourceCode&, bool isInStrictContext, DerivedContextType, bool isInArrowFunctionContext, EvalContextType, Intrinsic);
</span><span class="lines">@@ -735,7 +735,7 @@
</span><span class="cx">
</span><span class="cx"> DECLARE_INFO;
</span><span class="cx">
</span><del>- void prepareForExecution(ExecState*);
</del><ins>+ void prepareForExecution(VM&);
</ins><span class="cx">
</span><span class="cx"> WebAssemblyCodeBlock* codeBlockForCall()
</span><span class="cx"> {
</span><span class="lines">@@ -757,7 +757,7 @@
</span><span class="cx"> #endif
</span><span class="cx">
</span><span class="cx"> template <typename ExecutableType>
</span><del>-JSObject* ScriptExecutable::prepareForExecution(ExecState* exec, JSFunction* function, JSScope* scope, CodeSpecializationKind kind, CodeBlock*& resultCodeBlock)
</del><ins>+JSObject* ScriptExecutable::prepareForExecution(VM& vm, JSFunction* function, JSScope* scope, CodeSpecializationKind kind, CodeBlock*& resultCodeBlock)
</ins><span class="cx"> {
</span><span class="cx"> if (hasJITCodeFor(kind)) {
</span><span class="cx"> if (std::is_same<ExecutableType, EvalExecutable>::value)
</span><span class="lines">@@ -772,7 +772,7 @@
</span><span class="cx"> RELEASE_ASSERT_NOT_REACHED();
</span><span class="cx"> return nullptr;
</span><span class="cx"> }
</span><del>- return prepareForExecutionImpl(exec, function, scope, kind, resultCodeBlock);
</del><ins>+ return prepareForExecutionImpl(vm, function, scope, kind, resultCodeBlock);
</ins><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> } // namespace JSC
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreruntimeOptionsh"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/runtime/Options.h (207474 => 207475)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/runtime/Options.h 2016-10-18 18:23:56 UTC (rev 207474)
+++ trunk/Source/JavaScriptCore/runtime/Options.h 2016-10-18 18:30:05 UTC (rev 207475)
</span><span class="lines">@@ -301,6 +301,8 @@
</span><span class="cx"> v(double, structureCheckVoteRatioForHoisting, 1, Normal, nullptr) \
</span><span class="cx"> v(double, checkArrayVoteRatioForHoisting, 1, Normal, nullptr) \
</span><span class="cx"> \
</span><ins>+ v(unsigned, maximumDirectCallStackSize, 200, Normal, nullptr) \
+ \
</ins><span class="cx"> v(unsigned, minimumNumberOfScansBetweenRebalance, 100, Normal, nullptr) \
</span><span class="cx"> v(unsigned, numberOfGCMarkers, computeNumberOfGCMarkers(7), Normal, nullptr) \
</span><span class="cx"> v(unsigned, opaqueRootMergeThreshold, 1000, Normal, nullptr) \
</span></span></pre>
</div>
</div>
</body>
</html>