No subject
Mon Sep 28 12:00:37 PDT 2015
only admits Tmp and is defined early (like an early clobber register) and is used late (like
what we previously called LateUse, except that this time it's also a warm use). We already
had the beginning of support for early def's because of early clobbers, and we already
supported late uses albeit cold ones. I really only needed to add one new role: "Scratch",
which means both early def and late use in much the same way as "UseDef" means both early
use and late def. But, it feels better to complete the set of roles, so I added LateColdUse
to differentiate from LateUse (which is now a warm use) and EarlyDef to differentiate from
Def (which is, and always has been, a late def). Forcing the code to deal with the full
matrix of possibilities resulted in what is probably a progression in how we handle defs in
the register and stack allocators. The new Inst::forEachDef(Inst*, Inst*, callback) fully
recognizes that a "def" is something that can come from either the preceding instruction or
the succeeding one.
This doesn't add any new functionality to FTL B3 yet, but the new scratch register mechanism
is covered by new testb3 tests.
* b3/B3CheckSpecial.cpp:
(JSC::B3::CheckSpecial::isValid):
(JSC::B3::CheckSpecial::admitsStack):
(JSC::B3::CheckSpecial::generate):
* b3/B3LowerToAir.cpp:
(JSC::B3::Air::LowerToAir::lower):
* b3/B3PatchpointSpecial.cpp:
(JSC::B3::PatchpointSpecial::forEachArg):
(JSC::B3::PatchpointSpecial::isValid):
(JSC::B3::PatchpointSpecial::admitsStack):
(JSC::B3::PatchpointSpecial::generate):
* b3/B3PatchpointValue.cpp:
(JSC::B3::PatchpointValue::dumpMeta):
(JSC::B3::PatchpointValue::PatchpointValue):
* b3/B3PatchpointValue.h:
* b3/B3StackmapGenerationParams.cpp:
(JSC::B3::StackmapGenerationParams::unavailableRegisters):
* b3/B3StackmapGenerationParams.h:
(JSC::B3::StackmapGenerationParams::gpScratch):
(JSC::B3::StackmapGenerationParams::fpScratch):
* b3/B3StackmapSpecial.cpp:
(JSC::B3::StackmapSpecial::forEachArgImpl):
(JSC::B3::StackmapSpecial::isValidImpl):
(JSC::B3::StackmapSpecial::admitsStackImpl):
(JSC::B3::StackmapSpecial::repsImpl):
(JSC::B3::StackmapSpecial::isArgValidForValue):
(JSC::B3::StackmapSpecial::appendRepsImpl): Deleted.
* b3/B3StackmapSpecial.h:
* b3/air/AirAllocateStack.cpp:
(JSC::B3::Air::allocateStack):
* b3/air/AirArg.cpp:
(WTF::printInternal):
* b3/air/AirArg.h:
(JSC::B3::Air::Arg::isAnyUse):
(JSC::B3::Air::Arg::isColdUse):
(JSC::B3::Air::Arg::isEarlyUse):
(JSC::B3::Air::Arg::isLateUse):
(JSC::B3::Air::Arg::isAnyDef):
(JSC::B3::Air::Arg::isEarlyDef):
(JSC::B3::Air::Arg::isLateDef):
(JSC::B3::Air::Arg::isZDef):
(JSC::B3::Air::Arg::Arg):
(JSC::B3::Air::Arg::imm):
(JSC::B3::Air::Arg::isDef): Deleted.
* b3/air/AirBasicBlock.h:
(JSC::B3::Air::BasicBlock::at):
(JSC::B3::Air::BasicBlock::get):
(JSC::B3::Air::BasicBlock::last):
* b3/air/AirEliminateDeadCode.cpp:
(JSC::B3::Air::eliminateDeadCode):
* b3/air/AirFixPartialRegisterStalls.cpp:
(JSC::B3::Air::fixPartialRegisterStalls):
* b3/air/AirInst.cpp:
(JSC::B3::Air::Inst::hasArgEffects):
* b3/air/AirInst.h:
* b3/air/AirInstInlines.h:
(JSC::B3::Air::Inst::extraEarlyClobberedRegs):
(JSC::B3::Air::Inst::forEachDef):
(JSC::B3::Air::Inst::forEachDefWithExtraClobberedRegs):
(JSC::B3::Air::Inst::reportUsedRegisters):
(JSC::B3::Air::Inst::forEachTmpWithExtraClobberedRegs): Deleted.
* b3/air/AirIteratedRegisterCoalescing.cpp:
* b3/air/AirLiveness.h:
(JSC::B3::Air::AbstractLiveness::AbstractLiveness):
(JSC::B3::Air::AbstractLiveness::LocalCalc::execute):
* b3/air/AirSpillEverything.cpp:
(JSC::B3::Air::spillEverything):
* b3/air/AirTmpWidth.cpp:
(JSC::B3::Air::TmpWidth::recompute):
* b3/air/AirUseCounts.h:
(JSC::B3::Air::UseCounts::UseCounts):
* b3/testb3.cpp:
(JSC::B3::testPatchpointAny):
(JSC::B3::testPatchpointGPScratch):
(JSC::B3::testPatchpointFPScratch):
(JSC::B3::testPatchpointLotsOfLateAnys):
(JSC::B3::run):</pre>
<h3>Modified Paths</h3>
<ul>
<li><a href="#trunkSourceJavaScriptCoreChangeLog">trunk/Source/JavaScriptCore/ChangeLog</a></li>
<li><a href="#trunkSourceJavaScriptCoreb3B3CheckSpecialcpp">trunk/Source/JavaScriptCore/b3/B3CheckSpecial.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoreb3B3LowerToAircpp">trunk/Source/JavaScriptCore/b3/B3LowerToAir.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoreb3B3PatchpointSpecialcpp">trunk/Source/JavaScriptCore/b3/B3PatchpointSpecial.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoreb3B3PatchpointValuecpp">trunk/Source/JavaScriptCore/b3/B3PatchpointValue.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoreb3B3PatchpointValueh">trunk/Source/JavaScriptCore/b3/B3PatchpointValue.h</a></li>
<li><a href="#trunkSourceJavaScriptCoreb3B3StackmapGenerationParamscpp">trunk/Source/JavaScriptCore/b3/B3StackmapGenerationParams.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoreb3B3StackmapGenerationParamsh">trunk/Source/JavaScriptCore/b3/B3StackmapGenerationParams.h</a></li>
<li><a href="#trunkSourceJavaScriptCoreb3B3StackmapSpecialcpp">trunk/Source/JavaScriptCore/b3/B3StackmapSpecial.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoreb3B3StackmapSpecialh">trunk/Source/JavaScriptCore/b3/B3StackmapSpecial.h</a></li>
<li><a href="#trunkSourceJavaScriptCoreb3airAirAllocateStackcpp">trunk/Source/JavaScriptCore/b3/air/AirAllocateStack.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoreb3airAirArgcpp">trunk/Source/JavaScriptCore/b3/air/AirArg.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoreb3airAirArgh">trunk/Source/JavaScriptCore/b3/air/AirArg.h</a></li>
<li><a href="#trunkSourceJavaScriptCoreb3airAirBasicBlockh">trunk/Source/JavaScriptCore/b3/air/AirBasicBlock.h</a></li>
<li><a href="#trunkSourceJavaScriptCoreb3airAirEliminateDeadCodecpp">trunk/Source/JavaScriptCore/b3/air/AirEliminateDeadCode.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoreb3airAirFixPartialRegisterStallscpp">trunk/Source/JavaScriptCore/b3/air/AirFixPartialRegisterStalls.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoreb3airAirInstcpp">trunk/Source/JavaScriptCore/b3/air/AirInst.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoreb3airAirInsth">trunk/Source/JavaScriptCore/b3/air/AirInst.h</a></li>
<li><a href="#trunkSourceJavaScriptCoreb3airAirInstInlinesh">trunk/Source/JavaScriptCore/b3/air/AirInstInlines.h</a></li>
<li><a href="#trunkSourceJavaScriptCoreb3airAirIteratedRegisterCoalescingcpp">trunk/Source/JavaScriptCore/b3/air/AirIteratedRegisterCoalescing.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoreb3airAirLivenessh">trunk/Source/JavaScriptCore/b3/air/AirLiveness.h</a></li>
<li><a href="#trunkSourceJavaScriptCoreb3airAirSpillEverythingcpp">trunk/Source/JavaScriptCore/b3/air/AirSpillEverything.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoreb3airAirTmpWidthcpp">trunk/Source/JavaScriptCore/b3/air/AirTmpWidth.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoreb3airAirUseCountsh">trunk/Source/JavaScriptCore/b3/air/AirUseCounts.h</a></li>
<li><a href="#trunkSourceJavaScriptCoreb3testb3cpp">trunk/Source/JavaScriptCore/b3/testb3.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoredfgDFGCommonh">trunk/Source/JavaScriptCore/dfg/DFGCommon.h</a></li>
</ul>
</div>
<div id="patch">
<h3>Diff</h3>
<a id="trunkSourceJavaScriptCoreChangeLog"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/ChangeLog (194541 => 194542)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/ChangeLog 2016-01-04 19:27:43 UTC (rev 194541)
+++ trunk/Source/JavaScriptCore/ChangeLog 2016-01-04 19:33:35 UTC (rev 194542)
</span><span class="lines">@@ -1,3 +1,112 @@
</span><ins>+2016-01-04 Filip Pizlo <fpizlo at apple.com>
+
+ B3 patchpoints should allow requesting scratch registers
+ https://bugs.webkit.org/show_bug.cgi?id=152669
+
+ Reviewed by Benjamin Poulain.
+
+ Scratch registers are something that we often need in many patchpoint use cases. In LLVM's
+ patchpoints, we didn't have a good way to request scratch registers. So, our current FTL code
+ often does crazy scratch register allocation madness even when it would be better to just ask
+ the backend for some registers. This patch adds a mechanism for requesting scratch registers
+ in B3, and wires it all the way to all of our register allocation and liveness
+ infrastructure.
+
+ From the standpoint of a patchpoint, a "scratch register" is an instruction argument that
+ only admits Tmp and is defined early (like an early clobber register) and is used late (like
+ what we previously called LateUse, except that this time it's also a warm use). We already
+ had the beginning of support for early def's because of early clobbers, and we already
+ supported late uses albeit cold ones. I really only needed to add one new role: "Scratch",
+ which means both early def and late use in much the same way as "UseDef" means both early
+ use and late def. But, it feels better to complete the set of roles, so I added LateColdUse
+ to differentiate from LateUse (which is now a warm use) and EarlyDef to differentiate from
+ Def (which is, and always has been, a late def). Forcing the code to deal with the full
+ matrix of possibilities resulted in what is probably a progression in how we handle defs in
+ the register and stack allocators. The new Inst::forEachDef(Inst*, Inst*, callback) fully
+ recognizes that a "def" is something that can come from either the preceding instruction or
+ the succeeding one.
+
+ This doesn't add any new functionality to FTL B3 yet, but the new scratch register mechanism
+ is covered by new testb3 tests.
+
+ * b3/B3CheckSpecial.cpp:
+ (JSC::B3::CheckSpecial::isValid):
+ (JSC::B3::CheckSpecial::admitsStack):
+ (JSC::B3::CheckSpecial::generate):
+ * b3/B3LowerToAir.cpp:
+ (JSC::B3::Air::LowerToAir::lower):
+ * b3/B3PatchpointSpecial.cpp:
+ (JSC::B3::PatchpointSpecial::forEachArg):
+ (JSC::B3::PatchpointSpecial::isValid):
+ (JSC::B3::PatchpointSpecial::admitsStack):
+ (JSC::B3::PatchpointSpecial::generate):
+ * b3/B3PatchpointValue.cpp:
+ (JSC::B3::PatchpointValue::dumpMeta):
+ (JSC::B3::PatchpointValue::PatchpointValue):
+ * b3/B3PatchpointValue.h:
+ * b3/B3StackmapGenerationParams.cpp:
+ (JSC::B3::StackmapGenerationParams::unavailableRegisters):
+ * b3/B3StackmapGenerationParams.h:
+ (JSC::B3::StackmapGenerationParams::gpScratch):
+ (JSC::B3::StackmapGenerationParams::fpScratch):
+ * b3/B3StackmapSpecial.cpp:
+ (JSC::B3::StackmapSpecial::forEachArgImpl):
+ (JSC::B3::StackmapSpecial::isValidImpl):
+ (JSC::B3::StackmapSpecial::admitsStackImpl):
+ (JSC::B3::StackmapSpecial::repsImpl):
+ (JSC::B3::StackmapSpecial::isArgValidForValue):
+ (JSC::B3::StackmapSpecial::appendRepsImpl): Deleted.
+ * b3/B3StackmapSpecial.h:
+ * b3/air/AirAllocateStack.cpp:
+ (JSC::B3::Air::allocateStack):
+ * b3/air/AirArg.cpp:
+ (WTF::printInternal):
+ * b3/air/AirArg.h:
+ (JSC::B3::Air::Arg::isAnyUse):
+ (JSC::B3::Air::Arg::isColdUse):
+ (JSC::B3::Air::Arg::isEarlyUse):
+ (JSC::B3::Air::Arg::isLateUse):
+ (JSC::B3::Air::Arg::isAnyDef):
+ (JSC::B3::Air::Arg::isEarlyDef):
+ (JSC::B3::Air::Arg::isLateDef):
+ (JSC::B3::Air::Arg::isZDef):
+ (JSC::B3::Air::Arg::Arg):
+ (JSC::B3::Air::Arg::imm):
+ (JSC::B3::Air::Arg::isDef): Deleted.
+ * b3/air/AirBasicBlock.h:
+ (JSC::B3::Air::BasicBlock::at):
+ (JSC::B3::Air::BasicBlock::get):
+ (JSC::B3::Air::BasicBlock::last):
+ * b3/air/AirEliminateDeadCode.cpp:
+ (JSC::B3::Air::eliminateDeadCode):
+ * b3/air/AirFixPartialRegisterStalls.cpp:
+ (JSC::B3::Air::fixPartialRegisterStalls):
+ * b3/air/AirInst.cpp:
+ (JSC::B3::Air::Inst::hasArgEffects):
+ * b3/air/AirInst.h:
+ * b3/air/AirInstInlines.h:
+ (JSC::B3::Air::Inst::extraEarlyClobberedRegs):
+ (JSC::B3::Air::Inst::forEachDef):
+ (JSC::B3::Air::Inst::forEachDefWithExtraClobberedRegs):
+ (JSC::B3::Air::Inst::reportUsedRegisters):
+ (JSC::B3::Air::Inst::forEachTmpWithExtraClobberedRegs): Deleted.
+ * b3/air/AirIteratedRegisterCoalescing.cpp:
+ * b3/air/AirLiveness.h:
+ (JSC::B3::Air::AbstractLiveness::AbstractLiveness):
+ (JSC::B3::Air::AbstractLiveness::LocalCalc::execute):
+ * b3/air/AirSpillEverything.cpp:
+ (JSC::B3::Air::spillEverything):
+ * b3/air/AirTmpWidth.cpp:
+ (JSC::B3::Air::TmpWidth::recompute):
+ * b3/air/AirUseCounts.h:
+ (JSC::B3::Air::UseCounts::UseCounts):
+ * b3/testb3.cpp:
+ (JSC::B3::testPatchpointAny):
+ (JSC::B3::testPatchpointGPScratch):
+ (JSC::B3::testPatchpointFPScratch):
+ (JSC::B3::testPatchpointLotsOfLateAnys):
+ (JSC::B3::run):
+
</ins><span class="cx"> 2016-01-04 Csaba Osztrogonác <ossy at webkit.org>
</span><span class="cx">
</span><span class="cx"> Fix the !ENABLE(INTL) build after r193493
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreb3B3CheckSpecialcpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/b3/B3CheckSpecial.cpp (194541 => 194542)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/b3/B3CheckSpecial.cpp 2016-01-04 19:27:43 UTC (rev 194541)
+++ trunk/Source/JavaScriptCore/b3/B3CheckSpecial.cpp 2016-01-04 19:33:35 UTC (rev 194542)
</span><span class="lines">@@ -1,5 +1,5 @@
</span><span class="cx"> /*
</span><del>- * Copyright (C) 2015 Apple Inc. All rights reserved.
</del><ins>+ * Copyright (C) 2015-2016 Apple Inc. All rights reserved.
</ins><span class="cx"> *
</span><span class="cx"> * Redistribution and use in source and binary forms, with or without
</span><span class="cx"> * modification, are permitted provided that the following conditions
</span><span class="lines">@@ -124,7 +124,8 @@
</span><span class="cx"> bool CheckSpecial::isValid(Inst& inst)
</span><span class="cx"> {
</span><span class="cx"> return hiddenBranch(inst).isValidForm()
</span><del>- && isValidImpl(numB3Args(inst), m_numCheckArgs + 1, inst);
</del><ins>+ && isValidImpl(numB3Args(inst), m_numCheckArgs + 1, inst)
+ && inst.args.size() - m_numCheckArgs - 1 == inst.origin->numChildren() - numB3Args(inst);
</ins><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> bool CheckSpecial::admitsStack(Inst& inst, unsigned argIndex)
</span><span class="lines">@@ -142,8 +143,7 @@
</span><span class="cx"> StackmapValue* value = inst.origin->as<StackmapValue>();
</span><span class="cx"> ASSERT(value);
</span><span class="cx">
</span><del>- Vector<ValueRep> reps;
- appendRepsImpl(context, m_numCheckArgs + 1, inst, reps);
</del><ins>+ Vector<ValueRep> reps = repsImpl(context, numB3Args(inst), m_numCheckArgs + 1, inst);
</ins><span class="cx">
</span><span class="cx"> // Set aside the args that are relevant to undoing the operation. This is because we don't want to
</span><span class="cx"> // capture all of inst in the closure below.
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreb3B3LowerToAircpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/b3/B3LowerToAir.cpp (194541 => 194542)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/b3/B3LowerToAir.cpp 2016-01-04 19:27:43 UTC (rev 194541)
+++ trunk/Source/JavaScriptCore/b3/B3LowerToAir.cpp 2016-01-04 19:33:35 UTC (rev 194542)
</span><span class="lines">@@ -1,5 +1,5 @@
</span><span class="cx"> /*
</span><del>- * Copyright (C) 2015 Apple Inc. All rights reserved.
</del><ins>+ * Copyright (C) 2015-2016 Apple Inc. All rights reserved.
</ins><span class="cx"> *
</span><span class="cx"> * Redistribution and use in source and binary forms, with or without
</span><span class="cx"> * modification, are permitted provided that the following conditions
</span><span class="lines">@@ -2007,6 +2007,11 @@
</span><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> fillStackmap(inst, patchpointValue, 0);
</span><ins>+
+ for (unsigned i = patchpointValue->numGPScratchRegisters; i--;)
+ inst.args.append(m_code.newTmp(Arg::GP));
+ for (unsigned i = patchpointValue->numFPScratchRegisters; i--;)
+ inst.args.append(m_code.newTmp(Arg::FP));
</ins><span class="cx">
</span><span class="cx"> m_insts.last().append(WTFMove(inst));
</span><span class="cx"> m_insts.last().appendVector(after);
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreb3B3PatchpointSpecialcpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/b3/B3PatchpointSpecial.cpp (194541 => 194542)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/b3/B3PatchpointSpecial.cpp 2016-01-04 19:27:43 UTC (rev 194541)
+++ trunk/Source/JavaScriptCore/b3/B3PatchpointSpecial.cpp 2016-01-04 19:33:35 UTC (rev 194542)
</span><span class="lines">@@ -1,5 +1,5 @@
</span><span class="cx"> /*
</span><del>- * Copyright (C) 2015 Apple Inc. All rights reserved.
</del><ins>+ * Copyright (C) 2015-2016 Apple Inc. All rights reserved.
</ins><span class="cx"> *
</span><span class="cx"> * Redistribution and use in source and binary forms, with or without
</span><span class="cx"> * modification, are permitted provided that the following conditions
</span><span class="lines">@@ -46,32 +46,56 @@
</span><span class="cx">
</span><span class="cx"> void PatchpointSpecial::forEachArg(Inst& inst, const ScopedLambda<Inst::EachArgCallback>& callback)
</span><span class="cx"> {
</span><del>- // FIXME: Allow B3 Patchpoints to specify LateUse.
- // https://bugs.webkit.org/show_bug.cgi?id=151335
-
- if (inst.origin->type() == Void) {
- forEachArgImpl(0, 1, inst, SameAsRep, callback);
- return;
- }
</del><ins>+ unsigned argIndex = 1;
</ins><span class="cx">
</span><del>- callback(inst.args[1], Arg::Def, inst.origin->airType(), inst.origin->airWidth());
- forEachArgImpl(0, 2, inst, SameAsRep, callback);
</del><ins>+ if (inst.origin->type() != Void)
+ callback(inst.args[argIndex++], Arg::Def, inst.origin->airType(), inst.origin->airWidth());
+
+ forEachArgImpl(0, argIndex, inst, SameAsRep, callback);
+ argIndex += inst.origin->numChildren();
+
+ for (unsigned i = inst.origin->as<PatchpointValue>()->numGPScratchRegisters; i--;)
+ callback(inst.args[argIndex++], Arg::Scratch, Arg::GP, Arg::conservativeWidth(Arg::GP));
+ for (unsigned i = inst.origin->as<PatchpointValue>()->numFPScratchRegisters; i--;)
+ callback(inst.args[argIndex++], Arg::Scratch, Arg::FP, Arg::conservativeWidth(Arg::FP));
</ins><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> bool PatchpointSpecial::isValid(Inst& inst)
</span><span class="cx"> {
</span><del>- if (inst.origin->type() == Void)
- return isValidImpl(0, 1, inst);
</del><ins>+ PatchpointValue* patchpoint = inst.origin->as<PatchpointValue>();
+ unsigned argIndex = 1;
</ins><span class="cx">
</span><del>- if (inst.args.size() < 2)
</del><ins>+ if (inst.origin->type() != Void) {
+ if (argIndex >= inst.args.size())
+ return false;
+
+ if (!isArgValidForValue(inst.args[argIndex], patchpoint))
+ return false;
+ if (!isArgValidForRep(code(), inst.args[argIndex], patchpoint->resultConstraint))
+ return false;
+ argIndex++;
+ }
+
+ if (!isValidImpl(0, argIndex, inst))
</ins><span class="cx"> return false;
</span><del>- PatchpointValue* patchpoint = inst.origin->as<PatchpointValue>();
- if (!isArgValidForValue(inst.args[1], patchpoint))
</del><ins>+ argIndex += patchpoint->numChildren();
+
+ if (argIndex + patchpoint->numGPScratchRegisters + patchpoint->numFPScratchRegisters
+ != inst.args.size())
</ins><span class="cx"> return false;
</span><del>- if (!isArgValidForRep(code(), inst.args[1], patchpoint->resultConstraint))
- return false;
</del><span class="cx">
</span><del>- return isValidImpl(0, 2, inst);
</del><ins>+ for (unsigned i = patchpoint->numGPScratchRegisters; i--;) {
+ Arg arg = inst.args[argIndex++];
+ if (!arg.isGPTmp())
+ return false;
+ }
+ for (unsigned i = patchpoint->numFPScratchRegisters; i--;) {
+ Arg arg = inst.args[argIndex++];
+ if (!arg.isFPTmp())
+ return false;
+ }
+
+ return true;
</ins><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> bool PatchpointSpecial::admitsStack(Inst& inst, unsigned argIndex)
</span><span class="lines">@@ -99,16 +123,24 @@
</span><span class="cx"> CCallHelpers::Jump PatchpointSpecial::generate(
</span><span class="cx"> Inst& inst, CCallHelpers& jit, GenerationContext& context)
</span><span class="cx"> {
</span><del>- StackmapValue* value = inst.origin->as<StackmapValue>();
</del><ins>+ PatchpointValue* value = inst.origin->as<PatchpointValue>();
</ins><span class="cx"> ASSERT(value);
</span><span class="cx">
</span><span class="cx"> Vector<ValueRep> reps;
</span><span class="cx"> unsigned offset = 1;
</span><span class="cx"> if (inst.origin->type() != Void)
</span><span class="cx"> reps.append(repForArg(*context.code, inst.args[offset++]));
</span><del>- appendRepsImpl(context, offset, inst, reps);
</del><ins>+ reps.appendVector(repsImpl(context, 0, offset, inst));
+ offset += value->numChildren();
+
+ StackmapGenerationParams params(value, reps, context);
+
+ for (unsigned i = value->numGPScratchRegisters; i--;)
+ params.m_gpScratch.append(inst.args[offset++].gpr());
+ for (unsigned i = value->numFPScratchRegisters; i--;)
+ params.m_fpScratch.append(inst.args[offset++].fpr());
</ins><span class="cx">
</span><del>- value->m_generator->run(jit, StackmapGenerationParams(value, reps, context));
</del><ins>+ value->m_generator->run(jit, params);
</ins><span class="cx">
</span><span class="cx"> return CCallHelpers::Jump();
</span><span class="cx"> }
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreb3B3PatchpointValuecpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/b3/B3PatchpointValue.cpp (194541 => 194542)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/b3/B3PatchpointValue.cpp 2016-01-04 19:27:43 UTC (rev 194541)
+++ trunk/Source/JavaScriptCore/b3/B3PatchpointValue.cpp 2016-01-04 19:33:35 UTC (rev 194542)
</span><span class="lines">@@ -1,5 +1,5 @@
</span><span class="cx"> /*
</span><del>- * Copyright (C) 2015 Apple Inc. All rights reserved.
</del><ins>+ * Copyright (C) 2015-2016 Apple Inc. All rights reserved.
</ins><span class="cx"> *
</span><span class="cx"> * Redistribution and use in source and binary forms, with or without
</span><span class="cx"> * modification, are permitted provided that the following conditions
</span><span class="lines">@@ -38,6 +38,10 @@
</span><span class="cx"> {
</span><span class="cx"> Base::dumpMeta(comma, out);
</span><span class="cx"> out.print(comma, "resultConstraint = ", resultConstraint);
</span><ins>+ if (numGPScratchRegisters)
+ out.print(comma, "numGPScratchRegisters = ", numGPScratchRegisters);
+ if (numFPScratchRegisters)
+ out.print(comma, "numFPScratchRegisters = ", numFPScratchRegisters);
</ins><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> PatchpointValue::PatchpointValue(unsigned index, Type type, Origin origin)
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreb3B3PatchpointValueh"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/b3/B3PatchpointValue.h (194541 => 194542)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/b3/B3PatchpointValue.h 2016-01-04 19:27:43 UTC (rev 194541)
+++ trunk/Source/JavaScriptCore/b3/B3PatchpointValue.h 2016-01-04 19:33:35 UTC (rev 194542)
</span><span class="lines">@@ -1,5 +1,5 @@
</span><span class="cx"> /*
</span><del>- * Copyright (C) 2015 Apple Inc. All rights reserved.
</del><ins>+ * Copyright (C) 2015-2016 Apple Inc. All rights reserved.
</ins><span class="cx"> *
</span><span class="cx"> * Redistribution and use in source and binary forms, with or without
</span><span class="cx"> * modification, are permitted provided that the following conditions
</span><span class="lines">@@ -56,6 +56,12 @@
</span><span class="cx"> // is Void. Otherwise you can set this to any input constraint.
</span><span class="cx"> ValueRep resultConstraint;
</span><span class="cx">
</span><ins>+ // The number of scratch registers that this patchpoint gets. The scratch register is guaranteed
+ // to be different from any input register and the destination register. It's also guaranteed not
+ // to be clobbered either early or late. These are 0 by default.
+ uint8_t numGPScratchRegisters { 0 };
+ uint8_t numFPScratchRegisters { 0 };
+
</ins><span class="cx"> protected:
</span><span class="cx"> void dumpMeta(CommaPrinter&, PrintStream&) const override;
</span><span class="cx">
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreb3B3StackmapGenerationParamscpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/b3/B3StackmapGenerationParams.cpp (194541 => 194542)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/b3/B3StackmapGenerationParams.cpp 2016-01-04 19:27:43 UTC (rev 194541)
+++ trunk/Source/JavaScriptCore/b3/B3StackmapGenerationParams.cpp 2016-01-04 19:33:35 UTC (rev 194542)
</span><span class="lines">@@ -50,6 +50,12 @@
</span><span class="cx"> unsavedCalleeSaves.clear(regAtOffset.reg());
</span><span class="cx">
</span><span class="cx"> result.merge(unsavedCalleeSaves);
</span><ins>+
+ for (GPRReg gpr : m_gpScratch)
+ result.clear(gpr);
+ for (FPRReg fpr : m_fpScratch)
+ result.clear(fpr);
+
</ins><span class="cx"> return result;
</span><span class="cx"> }
</span><span class="cx">
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreb3B3StackmapGenerationParamsh"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/b3/B3StackmapGenerationParams.h (194541 => 194542)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/b3/B3StackmapGenerationParams.h 2016-01-04 19:27:43 UTC (rev 194541)
+++ trunk/Source/JavaScriptCore/b3/B3StackmapGenerationParams.h 2016-01-04 19:33:35 UTC (rev 194542)
</span><span class="lines">@@ -1,5 +1,5 @@
</span><span class="cx"> /*
</span><del>- * Copyright (C) 2015 Apple Inc. All rights reserved.
</del><ins>+ * Copyright (C) 2015-2016 Apple Inc. All rights reserved.
</ins><span class="cx"> *
</span><span class="cx"> * Redistribution and use in source and binary forms, with or without
</span><span class="cx"> * modification, are permitted provided that the following conditions
</span><span class="lines">@@ -59,13 +59,28 @@
</span><span class="cx"> // This tells you the registers that were used.
</span><span class="cx"> const RegisterSet& usedRegisters() const;
</span><span class="cx">
</span><del>- // This is a useful helper if you want to do register allocation inside of a patchpoint. You
- // can only use callee-save registers if they were saved in the prologue. This gives you the
- // used register set that's useful for such settings by returning:
</del><ins>+ // This is a useful helper if you want to do register allocation inside of a patchpoint. The
+ // usedRegisters() set is not directly useful for this purpose because:
</ins><span class="cx"> //
</span><del>- // usedRegisters() | (RegisterSet::calleeSaveRegisters() - proc.calleeSaveRegisters())
- RegisterSet unavailableRegisters() const;
</del><ins>+ // - You can only use callee-save registers for scratch if they were saved in the prologue. So,
+ // if a register is callee-save, it's not enough that it's not in usedRegisters().
+ //
+ // - Scratch registers are going to be in usedRegisters() at the patchpoint. So, if you want to
+ // find one of your requested scratch registers using usedRegisters(), you'll have a bad time.
+ //
+ // This gives you the used register set that's useful for allocating scratch registers. This set
+ // is defined as:
+ //
+ // (usedRegisters() | (RegisterSet::calleeSaveRegisters() - proc.calleeSaveRegisters()))
+ // - gpScratchRegisters - fpScratchRegisters
+ //
+ // I.e. it is like usedRegisters() but also includes unsaved callee-saves and excludes scratch
+ // registers.
+ JS_EXPORT_PRIVATE RegisterSet unavailableRegisters() const;
</ins><span class="cx">
</span><ins>+ GPRReg gpScratch(unsigned index) const { return m_gpScratch[index]; }
+ FPRReg fpScratch(unsigned index) const { return m_fpScratch[index]; }
+
</ins><span class="cx"> // This is provided for convenience; it means that you don't have to capture it if you don't want to.
</span><span class="cx"> Procedure& proc() const;
</span><span class="cx">
</span><span class="lines">@@ -90,6 +105,8 @@
</span><span class="cx">
</span><span class="cx"> StackmapValue* m_value;
</span><span class="cx"> Vector<ValueRep> m_reps;
</span><ins>+ Vector<GPRReg> m_gpScratch;
+ Vector<FPRReg> m_fpScratch;
</ins><span class="cx"> Air::GenerationContext& m_context;
</span><span class="cx"> };
</span><span class="cx">
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreb3B3StackmapSpecialcpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/b3/B3StackmapSpecial.cpp (194541 => 194542)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/b3/B3StackmapSpecial.cpp 2016-01-04 19:27:43 UTC (rev 194541)
+++ trunk/Source/JavaScriptCore/b3/B3StackmapSpecial.cpp 2016-01-04 19:33:35 UTC (rev 194542)
</span><span class="lines">@@ -1,5 +1,5 @@
</span><span class="cx"> /*
</span><del>- * Copyright (C) 2015 Apple Inc. All rights reserved.
</del><ins>+ * Copyright (C) 2015-2016 Apple Inc. All rights reserved.
</ins><span class="cx"> *
</span><span class="cx"> * Redistribution and use in source and binary forms, with or without
</span><span class="cx"> * modification, are permitted provided that the following conditions
</span><span class="lines">@@ -81,9 +81,9 @@
</span><span class="cx"> // Check that insane things have not happened.
</span><span class="cx"> ASSERT(inst.args.size() >= numIgnoredAirArgs);
</span><span class="cx"> ASSERT(value->children().size() >= numIgnoredB3Args);
</span><del>- ASSERT(inst.args.size() - numIgnoredAirArgs == value->children().size() - numIgnoredB3Args);
</del><ins>+ ASSERT(inst.args.size() - numIgnoredAirArgs >= value->children().size() - numIgnoredB3Args);
</ins><span class="cx">
</span><del>- for (unsigned i = 0; i < inst.args.size() - numIgnoredAirArgs; ++i) {
</del><ins>+ for (unsigned i = 0; i < value->children().size() - numIgnoredB3Args; ++i) {
</ins><span class="cx"> Arg& arg = inst.args[i + numIgnoredAirArgs];
</span><span class="cx"> ConstrainedValue child = value->constrainedChild(i + numIgnoredB3Args);
</span><span class="cx">
</span><span class="lines">@@ -103,12 +103,12 @@
</span><span class="cx"> role = Arg::ColdUse;
</span><span class="cx"> break;
</span><span class="cx"> case ValueRep::LateColdAny:
</span><del>- role = Arg::LateUse;
</del><ins>+ role = Arg::LateColdUse;
</ins><span class="cx"> break;
</span><span class="cx"> }
</span><span class="cx"> break;
</span><span class="cx"> case ForceLateUse:
</span><del>- role = Arg::LateUse;
</del><ins>+ role = Arg::LateColdUse;
</ins><span class="cx"> break;
</span><span class="cx"> }
</span><span class="cx">
</span><span class="lines">@@ -129,13 +129,13 @@
</span><span class="cx"> ASSERT(value->children().size() >= numIgnoredB3Args);
</span><span class="cx">
</span><span class="cx"> // For the Inst to be valid, it needs to have the right number of arguments.
</span><del>- if (inst.args.size() - numIgnoredAirArgs != value->children().size() - numIgnoredB3Args)
</del><ins>+ if (inst.args.size() - numIgnoredAirArgs < value->children().size() - numIgnoredB3Args)
</ins><span class="cx"> return false;
</span><span class="cx">
</span><span class="cx"> // Regardless of constraints, stackmaps have some basic requirements for their arguments. For
</span><span class="cx"> // example, you can't have a non-FP-offset address. This verifies those conditions as well as the
</span><span class="cx"> // argument types.
</span><del>- for (unsigned i = 0; i < inst.args.size() - numIgnoredAirArgs; ++i) {
</del><ins>+ for (unsigned i = 0; i < value->children().size() - numIgnoredB3Args; ++i) {
</ins><span class="cx"> Value* child = value->child(i + numIgnoredB3Args);
</span><span class="cx"> Arg& arg = inst.args[i + numIgnoredAirArgs];
</span><span class="cx">
</span><span class="lines">@@ -167,6 +167,11 @@
</span><span class="cx">
</span><span class="cx"> unsigned stackmapArgIndex = argIndex - numIgnoredAirArgs + numIgnoredB3Args;
</span><span class="cx">
</span><ins>+ if (stackmapArgIndex >= value->numChildren()) {
+ // It's not a stackmap argument, so as far as we are concerned, it doesn't admit stack.
+ return false;
+ }
+
</ins><span class="cx"> if (stackmapArgIndex >= value->m_reps.size()) {
</span><span class="cx"> // This means that there was no constraint.
</span><span class="cx"> return true;
</span><span class="lines">@@ -180,11 +185,13 @@
</span><span class="cx"> return false;
</span><span class="cx"> }
</span><span class="cx">
</span><del>-void StackmapSpecial::appendRepsImpl(
- GenerationContext& context, unsigned numIgnoredArgs, Inst& inst, Vector<ValueRep>& result)
</del><ins>+Vector<ValueRep> StackmapSpecial::repsImpl(
+ GenerationContext& context, unsigned numIgnoredB3Args, unsigned numIgnoredAirArgs, Inst& inst)
</ins><span class="cx"> {
</span><del>- for (unsigned i = numIgnoredArgs; i < inst.args.size(); ++i)
- result.append(repForArg(*context.code, inst.args[i]));
</del><ins>+ Vector<ValueRep> result;
+ for (unsigned i = 0; i < inst.origin->numChildren() - numIgnoredB3Args; ++i)
+ result.append(repForArg(*context.code, inst.args[i + numIgnoredAirArgs]));
+ return result;
</ins><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> bool StackmapSpecial::isArgValidForValue(const Air::Arg& arg, Value* value)
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreb3B3StackmapSpecialh"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/b3/B3StackmapSpecial.h (194541 => 194542)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/b3/B3StackmapSpecial.h 2016-01-04 19:27:43 UTC (rev 194541)
+++ trunk/Source/JavaScriptCore/b3/B3StackmapSpecial.h 2016-01-04 19:33:35 UTC (rev 194542)
</span><span class="lines">@@ -1,5 +1,5 @@
</span><span class="cx"> /*
</span><del>- * Copyright (C) 2015 Apple Inc. All rights reserved.
</del><ins>+ * Copyright (C) 2015-2016 Apple Inc. All rights reserved.
</ins><span class="cx"> *
</span><span class="cx"> * Redistribution and use in source and binary forms, with or without
</span><span class="cx"> * modification, are permitted provided that the following conditions
</span><span class="lines">@@ -69,7 +69,8 @@
</span><span class="cx"> Air::Inst&, unsigned argIndex);
</span><span class="cx">
</span><span class="cx"> // Appends the reps for the Inst's args, starting with numIgnoredArgs, to the given vector.
</span><del>- void appendRepsImpl(Air::GenerationContext&, unsigned numIgnoredArgs, Air::Inst&, Vector<ValueRep>&);
</del><ins>+ Vector<ValueRep> repsImpl(
+ Air::GenerationContext&, unsigned numIgnoredB3Args, unsigned numIgnoredAirArgs, Air::Inst&);
</ins><span class="cx">
</span><span class="cx"> static bool isArgValidForValue(const Air::Arg&, Value*);
</span><span class="cx"> static bool isArgValidForRep(Air::Code&, const Air::Arg&, const ValueRep&);
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreb3airAirAllocateStackcpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/b3/air/AirAllocateStack.cpp (194541 => 194542)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/b3/air/AirAllocateStack.cpp 2016-01-04 19:27:43 UTC (rev 194541)
+++ trunk/Source/JavaScriptCore/b3/air/AirAllocateStack.cpp 2016-01-04 19:33:35 UTC (rev 194542)
</span><span class="lines">@@ -1,5 +1,5 @@
</span><span class="cx"> /*
</span><del>- * Copyright (C) 2015 Apple Inc. All rights reserved.
</del><ins>+ * Copyright (C) 2015-2016 Apple Inc. All rights reserved.
</ins><span class="cx"> *
</span><span class="cx"> * Redistribution and use in source and binary forms, with or without
</span><span class="cx"> * modification, are permitted provided that the following conditions
</span><span class="lines">@@ -143,14 +143,13 @@
</span><span class="cx"> for (BasicBlock* block : code) {
</span><span class="cx"> StackSlotLiveness::LocalCalc localCalc(liveness, block);
</span><span class="cx">
</span><del>- auto interfere = [&] (Inst& inst) {
</del><ins>+ auto interfere = [&] (unsigned instIndex) {
</ins><span class="cx"> if (verbose)
</span><span class="cx"> dataLog("Interfering: ", WTF::pointerListDump(localCalc.live()), "\n");
</span><span class="cx">
</span><del>- inst.forEachArg(
- [&] (Arg& arg, Arg::Role role, Arg::Type, Arg::Width) {
- if (!Arg::isDef(role))
- return;
</del><ins>+ Inst::forEachDef<Arg>(
+ block->get(instIndex), block->get(instIndex + 1),
+ [&] (Arg& arg, Arg::Role, Arg::Type, Arg::Width) {
</ins><span class="cx"> if (!arg.isStack())
</span><span class="cx"> return;
</span><span class="cx"> StackSlot* slot = arg.stackSlot();
</span><span class="lines">@@ -167,12 +166,10 @@
</span><span class="cx"> for (unsigned instIndex = block->size(); instIndex--;) {
</span><span class="cx"> if (verbose)
</span><span class="cx"> dataLog("Analyzing: ", block->at(instIndex), "\n");
</span><del>- Inst& inst = block->at(instIndex);
- interfere(inst);
</del><ins>+ interfere(instIndex);
</ins><span class="cx"> localCalc.execute(instIndex);
</span><span class="cx"> }
</span><del>- Inst nop;
- interfere(nop);
</del><ins>+ interfere(-1);
</ins><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> if (verbose) {
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreb3airAirArgcpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/b3/air/AirArg.cpp (194541 => 194542)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/b3/air/AirArg.cpp 2016-01-04 19:27:43 UTC (rev 194541)
+++ trunk/Source/JavaScriptCore/b3/air/AirArg.cpp 2016-01-04 19:33:35 UTC (rev 194542)
</span><span class="lines">@@ -1,5 +1,5 @@
</span><span class="cx"> /*
</span><del>- * Copyright (C) 2015 Apple Inc. All rights reserved.
</del><ins>+ * Copyright (C) 2015-2016 Apple Inc. All rights reserved.
</ins><span class="cx"> *
</span><span class="cx"> * Redistribution and use in source and binary forms, with or without
</span><span class="cx"> * modification, are permitted provided that the following conditions
</span><span class="lines">@@ -193,6 +193,15 @@
</span><span class="cx"> case Arg::LateUse:
</span><span class="cx"> out.print("LateUse");
</span><span class="cx"> return;
</span><ins>+ case Arg::LateColdUse:
+ out.print("LateColdUse");
+ return;
+ case Arg::EarlyDef:
+ out.print("EarlyDef");
+ return;
+ case Arg::Scratch:
+ out.print("Scratch");
+ return;
</ins><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> RELEASE_ASSERT_NOT_REACHED();
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreb3airAirArgh"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/b3/air/AirArg.h (194541 => 194542)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/b3/air/AirArg.h 2016-01-04 19:27:43 UTC (rev 194541)
+++ trunk/Source/JavaScriptCore/b3/air/AirArg.h 2016-01-04 19:33:35 UTC (rev 194542)
</span><span class="lines">@@ -1,5 +1,5 @@
</span><span class="cx"> /*
</span><del>- * Copyright (C) 2015 Apple Inc. All rights reserved.
</del><ins>+ * Copyright (C) 2015-2016 Apple Inc. All rights reserved.
</ins><span class="cx"> *
</span><span class="cx"> * Redistribution and use in source and binary forms, with or without
</span><span class="cx"> * modification, are permitted provided that the following conditions
</span><span class="lines">@@ -92,10 +92,14 @@
</span><span class="cx"> ColdUse,
</span><span class="cx">
</span><span class="cx"> // LateUse means that the Inst will read from this value after doing its Def's. Note that LateUse
</span><del>- // on an Addr or Index still means Use on the internal temporaries. LateUse also currently also
- // implies ColdUse.
</del><ins>+ // on an Addr or Index still means Use on the internal temporaries. Note that specifying the
+ // same Tmp once as Def and once as LateUse has undefined behavior: the use may happen before
+ // the def, or it may happen after it.
</ins><span class="cx"> LateUse,
</span><span class="cx">
</span><ins>+ // Combination of LateUse and ColdUse.
+ LateColdUse,
+
</ins><span class="cx"> // Def means that the Inst will write to this value after doing everything else.
</span><span class="cx"> //
</span><span class="cx"> // For Tmp: The Inst will write to this Tmp.
</span><span class="lines">@@ -127,6 +131,16 @@
</span><span class="cx"> // This is a combined Use and ZDef. It means that both things happen.
</span><span class="cx"> UseZDef,
</span><span class="cx">
</span><ins>+ // This is like Def, but implies that the assignment occurs before the start of the Inst's
+ // execution rather than after. Note that specifying the same Tmp once as EarlyDef and once
+ // as Use has undefined behavior: the use may happen before the def, or it may happen after
+ // it.
+ EarlyDef,
+
+ // Some instructions need a scratch register. We model this by saying that the temporary is
+ // defined early and used late. This role implies that.
+ Scratch,
+
</ins><span class="cx"> // This is a special kind of use that is only valid for addresses. It means that the
</span><span class="cx"> // instruction will evaluate the address expression and consume the effective address, but it
</span><span class="cx"> // will neither load nor store. This is an escaping use, because now the address may be
</span><span class="lines">@@ -171,10 +185,13 @@
</span><span class="cx"> case UseDef:
</span><span class="cx"> case UseZDef:
</span><span class="cx"> case LateUse:
</span><ins>+ case LateColdUse:
+ case Scratch:
</ins><span class="cx"> return true;
</span><span class="cx"> case Def:
</span><span class="cx"> case ZDef:
</span><span class="cx"> case UseAddr:
</span><ins>+ case EarlyDef:
</ins><span class="cx"> return false;
</span><span class="cx"> }
</span><span class="cx"> }
</span><span class="lines">@@ -183,14 +200,17 @@
</span><span class="cx"> {
</span><span class="cx"> switch (role) {
</span><span class="cx"> case ColdUse:
</span><del>- case LateUse:
</del><ins>+ case LateColdUse:
</ins><span class="cx"> return true;
</span><span class="cx"> case Use:
</span><span class="cx"> case UseDef:
</span><span class="cx"> case UseZDef:
</span><ins>+ case LateUse:
</ins><span class="cx"> case Def:
</span><span class="cx"> case ZDef:
</span><span class="cx"> case UseAddr:
</span><ins>+ case Scratch:
+ case EarlyDef:
</ins><span class="cx"> return false;
</span><span class="cx"> }
</span><span class="cx"> }
</span><span class="lines">@@ -213,6 +233,9 @@
</span><span class="cx"> case ZDef:
</span><span class="cx"> case UseAddr:
</span><span class="cx"> case LateUse:
</span><ins>+ case LateColdUse:
+ case Scratch:
+ case EarlyDef:
</ins><span class="cx"> return false;
</span><span class="cx"> }
</span><span class="cx"> }
</span><span class="lines">@@ -220,26 +243,83 @@
</span><span class="cx"> // Returns true if the Role implies that the Inst will Use the Arg after doing everything else.
</span><span class="cx"> static bool isLateUse(Role role)
</span><span class="cx"> {
</span><del>- return role == LateUse;
</del><ins>+ switch (role) {
+ case LateUse:
+ case LateColdUse:
+ case Scratch:
+ return true;
+ case ColdUse:
+ case Use:
+ case UseDef:
+ case UseZDef:
+ case Def:
+ case ZDef:
+ case UseAddr:
+ case EarlyDef:
+ return false;
+ }
</ins><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> // Returns true if the Role implies that the Inst will Def the Arg.
</span><del>- static bool isDef(Role role)
</del><ins>+ static bool isAnyDef(Role role)
</ins><span class="cx"> {
</span><span class="cx"> switch (role) {
</span><span class="cx"> case Use:
</span><span class="cx"> case ColdUse:
</span><span class="cx"> case UseAddr:
</span><span class="cx"> case LateUse:
</span><ins>+ case LateColdUse:
</ins><span class="cx"> return false;
</span><span class="cx"> case Def:
</span><span class="cx"> case UseDef:
</span><span class="cx"> case ZDef:
</span><span class="cx"> case UseZDef:
</span><ins>+ case EarlyDef:
+ case Scratch:
</ins><span class="cx"> return true;
</span><span class="cx"> }
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+ // Returns true if the Role implies that the Inst will Def the Arg before start of execution.
+ static bool isEarlyDef(Role role)
+ {
+ switch (role) {
+ case Use:
+ case ColdUse:
+ case UseAddr:
+ case LateUse:
+ case Def:
+ case UseDef:
+ case ZDef:
+ case UseZDef:
+ case LateColdUse:
+ return false;
+ case EarlyDef:
+ case Scratch:
+ return true;
+ }
+ }
+
+ // Returns true if the Role implies that the Inst will Def the Arg after the end of execution.
+ static bool isLateDef(Role role)
+ {
+ switch (role) {
+ case Use:
+ case ColdUse:
+ case UseAddr:
+ case LateUse:
+ case EarlyDef:
+ case Scratch:
+ case LateColdUse:
+ return false;
+ case Def:
+ case UseDef:
+ case ZDef:
+ case UseZDef:
+ return true;
+ }
+ }
+
</ins><span class="cx"> // Returns true if the Role implies that the Inst will ZDef the Arg.
</span><span class="cx"> static bool isZDef(Role role)
</span><span class="cx"> {
</span><span class="lines">@@ -250,6 +330,9 @@
</span><span class="cx"> case LateUse:
</span><span class="cx"> case Def:
</span><span class="cx"> case UseDef:
</span><ins>+ case EarlyDef:
+ case Scratch:
+ case LateColdUse:
</ins><span class="cx"> return false;
</span><span class="cx"> case ZDef:
</span><span class="cx"> case UseZDef:
</span><span class="lines">@@ -331,6 +414,11 @@
</span><span class="cx"> {
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+ Arg(Reg reg)
+ : Arg(Air::Tmp(reg))
+ {
+ }
+
</ins><span class="cx"> static Arg imm(int64_t value)
</span><span class="cx"> {
</span><span class="cx"> Arg result;
</span><span class="lines">@@ -845,7 +933,7 @@
</span><span class="cx"> case CallArg:
</span><span class="cx"> return isValidAddrForm(offset());
</span><span class="cx"> case Index:
</span><del>- return isValidIndexForm(offset(), scale(), width);
</del><ins>+ return isValidIndexForm(scale(), offset(), width);
</ins><span class="cx"> case RelCond:
</span><span class="cx"> case ResCond:
</span><span class="cx"> case DoubleCond:
</span><span class="lines">@@ -882,7 +970,7 @@
</span><span class="cx"> {
</span><span class="cx"> switch (m_kind) {
</span><span class="cx"> case Tmp:
</span><del>- ASSERT(isAnyUse(argRole) || isDef(argRole));
</del><ins>+ ASSERT(isAnyUse(argRole) || isAnyDef(argRole));
</ins><span class="cx"> functor(m_base, argRole, argType, argWidth);
</span><span class="cx"> break;
</span><span class="cx"> case Addr:
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreb3airAirBasicBlockh"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/b3/air/AirBasicBlock.h (194541 => 194542)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/b3/air/AirBasicBlock.h 2016-01-04 19:27:43 UTC (rev 194541)
+++ trunk/Source/JavaScriptCore/b3/air/AirBasicBlock.h 2016-01-04 19:33:35 UTC (rev 194542)
</span><span class="lines">@@ -1,5 +1,5 @@
</span><span class="cx"> /*
</span><del>- * Copyright (C) 2015 Apple Inc. All rights reserved.
</del><ins>+ * Copyright (C) 2015-2016 Apple Inc. All rights reserved.
</ins><span class="cx"> *
</span><span class="cx"> * Redistribution and use in source and binary forms, with or without
</span><span class="cx"> * modification, are permitted provided that the following conditions
</span><span class="lines">@@ -64,6 +64,11 @@
</span><span class="cx"> const Inst& at(unsigned index) const { return m_insts[index]; }
</span><span class="cx"> Inst& at(unsigned index) { return m_insts[index]; }
</span><span class="cx">
</span><ins>+ Inst* get(unsigned index)
+ {
+ return index < size() ? &at(index) : nullptr;
+ }
+
</ins><span class="cx"> const Inst& last() const { return m_insts.last(); }
</span><span class="cx"> Inst& last() { return m_insts.last(); }
</span><span class="cx">
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreb3airAirEliminateDeadCodecpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/b3/air/AirEliminateDeadCode.cpp (194541 => 194542)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/b3/air/AirEliminateDeadCode.cpp 2016-01-04 19:27:43 UTC (rev 194541)
+++ trunk/Source/JavaScriptCore/b3/air/AirEliminateDeadCode.cpp 2016-01-04 19:33:35 UTC (rev 194542)
</span><span class="lines">@@ -1,5 +1,5 @@
</span><span class="cx"> /*
</span><del>- * Copyright (C) 2015 Apple Inc. All rights reserved.
</del><ins>+ * Copyright (C) 2015-2016 Apple Inc. All rights reserved.
</ins><span class="cx"> *
</span><span class="cx"> * Redistribution and use in source and binary forms, with or without
</span><span class="cx"> * modification, are permitted provided that the following conditions
</span><span class="lines">@@ -81,8 +81,10 @@
</span><span class="cx"> bool storesToLive = false;
</span><span class="cx"> inst.forEachArg(
</span><span class="cx"> [&] (Arg& arg, Arg::Role role, Arg::Type, Arg::Width) {
</span><del>- if (!Arg::isDef(role))
</del><ins>+ if (!Arg::isAnyDef(role))
</ins><span class="cx"> return;
</span><ins>+ if (role == Arg::Scratch)
+ return;
</ins><span class="cx"> storesToLive |= isArgLive(arg);
</span><span class="cx"> });
</span><span class="cx"> return storesToLive;
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreb3airAirFixPartialRegisterStallscpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/b3/air/AirFixPartialRegisterStalls.cpp (194541 => 194542)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/b3/air/AirFixPartialRegisterStalls.cpp 2016-01-04 19:27:43 UTC (rev 194541)
+++ trunk/Source/JavaScriptCore/b3/air/AirFixPartialRegisterStalls.cpp 2016-01-04 19:33:35 UTC (rev 194542)
</span><span class="lines">@@ -1,5 +1,5 @@
</span><span class="cx"> /*
</span><del>- * Copyright (C) 2015 Apple Inc. All rights reserved.
</del><ins>+ * Copyright (C) 2015-2016 Apple Inc. All rights reserved.
</ins><span class="cx"> *
</span><span class="cx"> * Redistribution and use in source and binary forms, with or without
</span><span class="cx"> * modification, are permitted provided that the following conditions
</span><span class="lines">@@ -116,7 +116,7 @@
</span><span class="cx"> inst.forEachTmp([&] (Tmp& tmp, Arg::Role role, Arg::Type, Arg::Width) {
</span><span class="cx"> ASSERT_WITH_MESSAGE(tmp.isReg(), "This phase must be run after register allocation.");
</span><span class="cx">
</span><del>- if (tmp.isFPR() && Arg::isDef(role))
</del><ins>+ if (tmp.isFPR() && Arg::isAnyDef(role))
</ins><span class="cx"> localDistance.add(tmp.fpr(), distanceToBlockEnd);
</span><span class="cx"> });
</span><span class="cx"> }
</span><span class="lines">@@ -205,7 +205,7 @@
</span><span class="cx"> RegisterSet uses;
</span><span class="cx"> inst.forEachTmp([&] (Tmp& tmp, Arg::Role role, Arg::Type, Arg::Width) {
</span><span class="cx"> if (tmp.isFPR()) {
</span><del>- if (Arg::isDef(role))
</del><ins>+ if (Arg::isAnyDef(role))
</ins><span class="cx"> defs.set(tmp.fpr());
</span><span class="cx"> if (Arg::isAnyUse(role))
</span><span class="cx"> uses.set(tmp.fpr());
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreb3airAirInstcpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/b3/air/AirInst.cpp (194541 => 194542)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/b3/air/AirInst.cpp 2016-01-04 19:27:43 UTC (rev 194541)
+++ trunk/Source/JavaScriptCore/b3/air/AirInst.cpp 2016-01-04 19:33:35 UTC (rev 194542)
</span><span class="lines">@@ -1,5 +1,5 @@
</span><span class="cx"> /*
</span><del>- * Copyright (C) 2015 Apple Inc. All rights reserved.
</del><ins>+ * Copyright (C) 2015-2016 Apple Inc. All rights reserved.
</ins><span class="cx"> *
</span><span class="cx"> * Redistribution and use in source and binary forms, with or without
</span><span class="cx"> * modification, are permitted provided that the following conditions
</span><span class="lines">@@ -39,7 +39,7 @@
</span><span class="cx"> bool result = false;
</span><span class="cx"> forEachArg(
</span><span class="cx"> [&] (Arg&, Arg::Role role, Arg::Type, Arg::Width) {
</span><del>- if (Arg::isDef(role))
</del><ins>+ if (Arg::isAnyDef(role))
</ins><span class="cx"> result = true;
</span><span class="cx"> });
</span><span class="cx"> return result;
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreb3airAirInsth"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/b3/air/AirInst.h (194541 => 194542)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/b3/air/AirInst.h 2016-01-04 19:27:43 UTC (rev 194541)
+++ trunk/Source/JavaScriptCore/b3/air/AirInst.h 2016-01-04 19:33:35 UTC (rev 194542)
</span><span class="lines">@@ -1,5 +1,5 @@
</span><span class="cx"> /*
</span><del>- * Copyright (C) 2015 Apple Inc. All rights reserved.
</del><ins>+ * Copyright (C) 2015-2016 Apple Inc. All rights reserved.
</ins><span class="cx"> *
</span><span class="cx"> * Redistribution and use in source and binary forms, with or without
</span><span class="cx"> * modification, are permitted provided that the following conditions
</span><span class="lines">@@ -115,7 +115,7 @@
</span><span class="cx"> });
</span><span class="cx"> }
</span><span class="cx">
</span><del>- // Thing can be either Arg or Tmp.
</del><ins>+ // Thing can be either Arg, Tmp, or StackSlot*.
</ins><span class="cx"> template<typename Thing, typename Functor>
</span><span class="cx"> void forEach(const Functor&);
</span><span class="cx">
</span><span class="lines">@@ -127,11 +127,19 @@
</span><span class="cx"> const RegisterSet& extraClobberedRegs();
</span><span class="cx"> const RegisterSet& extraEarlyClobberedRegs();
</span><span class="cx">
</span><del>- // Iterate over the Defs and the extra clobbered registers. You must supply the next instruction if
- // there is one.
- template<typename Functor>
- void forEachTmpWithExtraClobberedRegs(Inst* nextInst, const Functor& functor);
</del><ins>+ // Iterate over all Def's that happen at the end of an instruction. You supply a pair
+ // instructions. The instructions must appear next to each other, in that order, in some basic
+ // block. You can pass null for the first instruction when analyzing what happens at the top of
+ // a basic block. You can pass null for the second instruction when analyzing what happens at the
+ // bottom of a basic block.
+ template<typename Thing, typename Functor>
+ static void forEachDef(Inst* prevInst, Inst* nextInst, const Functor&);
</ins><span class="cx">
</span><ins>+ // Iterate over all Def's that happen at the end of this instruction, including extra clobbered
+ // registers. Note that Thing can only be Arg or Tmp when you use this functor.
+ template<typename Thing, typename Functor>
+ static void forEachDefWithExtraClobberedRegs(Inst* prevInst, Inst* nextInst, const Functor&);
+
</ins><span class="cx"> // Use this to report which registers are live. This should be done just before codegen. Note
</span><span class="cx"> // that for efficiency, reportUsedRegisters() only works if hasSpecial() returns true.
</span><span class="cx"> void reportUsedRegisters(const RegisterSet&);
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreb3airAirInstInlinesh"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/b3/air/AirInstInlines.h (194541 => 194542)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/b3/air/AirInstInlines.h 2016-01-04 19:27:43 UTC (rev 194541)
+++ trunk/Source/JavaScriptCore/b3/air/AirInstInlines.h 2016-01-04 19:33:35 UTC (rev 194542)
</span><span class="lines">@@ -1,5 +1,5 @@
</span><span class="cx"> /*
</span><del>- * Copyright (C) 2015 Apple Inc. All rights reserved.
</del><ins>+ * Copyright (C) 2015-2016 Apple Inc. All rights reserved.
</ins><span class="cx"> *
</span><span class="cx"> * Redistribution and use in source and binary forms, with or without
</span><span class="cx"> * modification, are permitted provided that the following conditions
</span><span class="lines">@@ -95,24 +95,48 @@
</span><span class="cx"> return args[0].special()->extraEarlyClobberedRegs(*this);
</span><span class="cx"> }
</span><span class="cx">
</span><del>-template<typename Functor>
-inline void Inst::forEachTmpWithExtraClobberedRegs(Inst* nextInst, const Functor& functor)
</del><ins>+template<typename Thing, typename Functor>
+inline void Inst::forEachDef(Inst* prevInst, Inst* nextInst, const Functor& functor)
</ins><span class="cx"> {
</span><del>- forEachTmp(
- [&] (Tmp& tmpArg, Arg::Role role, Arg::Type argType, Arg::Width argWidth) {
- functor(tmpArg, role, argType, argWidth);
- });
</del><ins>+ if (prevInst) {
+ prevInst->forEach<Thing>(
+ [&] (Thing& thing, Arg::Role role, Arg::Type argType, Arg::Width argWidth) {
+ if (Arg::isLateDef(role))
+ functor(thing, role, argType, argWidth);
+ });
+ }
</ins><span class="cx">
</span><ins>+ if (nextInst) {
+ nextInst->forEach<Thing>(
+ [&] (Thing& thing, Arg::Role role, Arg::Type argType, Arg::Width argWidth) {
+ if (Arg::isEarlyDef(role))
+ functor(thing, role, argType, argWidth);
+ });
+ }
+}
+
+template<typename Thing, typename Functor>
+inline void Inst::forEachDefWithExtraClobberedRegs(
+ Inst* prevInst, Inst* nextInst, const Functor& functor)
+{
+ forEachDef<Thing>(prevInst, nextInst, functor);
+
+ Arg::Role regDefRole;
+
</ins><span class="cx"> auto reportReg = [&] (Reg reg) {
</span><span class="cx"> Arg::Type type = reg.isGPR() ? Arg::GP : Arg::FP;
</span><del>- functor(Tmp(reg), Arg::Def, type, Arg::conservativeWidth(type));
</del><ins>+ functor(Thing(reg), regDefRole, type, Arg::conservativeWidth(type));
</ins><span class="cx"> };
</span><span class="cx">
</span><del>- if (hasSpecial())
- extraClobberedRegs().forEach(reportReg);
</del><ins>+ if (prevInst && prevInst->hasSpecial()) {
+ regDefRole = Arg::Def;
+ prevInst->extraClobberedRegs().forEach(reportReg);
+ }
</ins><span class="cx">
</span><del>- if (nextInst && nextInst->hasSpecial())
</del><ins>+ if (nextInst && nextInst->hasSpecial()) {
+ regDefRole = Arg::EarlyDef;
</ins><span class="cx"> nextInst->extraEarlyClobberedRegs().forEach(reportReg);
</span><ins>+ }
</ins><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> inline void Inst::reportUsedRegisters(const RegisterSet& usedRegisters)
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreb3airAirIteratedRegisterCoalescingcpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/b3/air/AirIteratedRegisterCoalescing.cpp (194541 => 194542)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/b3/air/AirIteratedRegisterCoalescing.cpp 2016-01-04 19:27:43 UTC (rev 194541)
+++ trunk/Source/JavaScriptCore/b3/air/AirIteratedRegisterCoalescing.cpp 2016-01-04 19:33:35 UTC (rev 194542)
</span><span class="lines">@@ -770,40 +770,43 @@
</span><span class="cx"> typename TmpLiveness<type>::LocalCalc localCalc(liveness, block);
</span><span class="cx"> for (unsigned instIndex = block->size(); instIndex--;) {
</span><span class="cx"> Inst& inst = block->at(instIndex);
</span><del>- Inst* nextInst = instIndex + 1 < block->size() ? &block->at(instIndex + 1) : nullptr;
- build(inst, nextInst, localCalc);
</del><ins>+ Inst* nextInst = block->get(instIndex + 1);
+ build(&inst, nextInst, localCalc);
</ins><span class="cx"> localCalc.execute(instIndex);
</span><span class="cx"> }
</span><ins>+ build(nullptr, &block->at(0), localCalc);
</ins><span class="cx"> }
</span><span class="cx"> }
</span><span class="cx">
</span><del>- void build(Inst& inst, Inst* nextInst, const typename TmpLiveness<type>::LocalCalc& localCalc)
</del><ins>+ void build(Inst* prevInst, Inst* nextInst, const typename TmpLiveness<type>::LocalCalc& localCalc)
</ins><span class="cx"> {
</span><del>- inst.forEachTmpWithExtraClobberedRegs(
- nextInst,
- [&] (const Tmp& arg, Arg::Role role, Arg::Type argType, Arg::Width) {
- if (!Arg::isDef(role) || argType != type)
</del><ins>+ Inst::forEachDefWithExtraClobberedRegs<Tmp>(
+ prevInst, nextInst,
+ [&] (const Tmp& arg, Arg::Role, Arg::Type argType, Arg::Width) {
+ if (argType != type)
</ins><span class="cx"> return;
</span><span class="cx">
</span><span class="cx"> // All the Def()s interfere with each other and with all the extra clobbered Tmps.
</span><del>- // We should not use forEachDefAndExtraClobberedTmp() here since colored Tmps
</del><ins>+ // We should not use forEachDefWithExtraClobberedRegs() here since colored Tmps
</ins><span class="cx"> // do not need interference edges in our implementation.
</span><del>- inst.forEachTmp([&] (Tmp& otherArg, Arg::Role role, Arg::Type argType, Arg::Width) {
- if (!Arg::isDef(role) || argType != type)
- return;
-
- addEdge(arg, otherArg);
- });
</del><ins>+ Inst::forEachDef<Tmp>(
+ prevInst, nextInst,
+ [&] (Tmp& otherArg, Arg::Role, Arg::Type argType, Arg::Width) {
+ if (argType != type)
+ return;
+
+ addEdge(arg, otherArg);
+ });
</ins><span class="cx"> });
</span><span class="cx">
</span><del>- if (mayBeCoalescable(inst)) {
</del><ins>+ if (prevInst && mayBeCoalescable(*prevInst)) {
</ins><span class="cx"> // We do not want the Use() of this move to interfere with the Def(), even if it is live
</span><span class="cx"> // after the Move. If we were to add the interference edge, it would be impossible to
</span><span class="cx"> // coalesce the Move even if the two Tmp never interfere anywhere.
</span><span class="cx"> Tmp defTmp;
</span><span class="cx"> Tmp useTmp;
</span><del>- inst.forEachTmp([&defTmp, &useTmp] (Tmp& argTmp, Arg::Role role, Arg::Type, Arg::Width) {
- if (Arg::isDef(role))
</del><ins>+ prevInst->forEachTmp([&defTmp, &useTmp] (Tmp& argTmp, Arg::Role role, Arg::Type, Arg::Width) {
+ if (Arg::isLateDef(role))
</ins><span class="cx"> defTmp = argTmp;
</span><span class="cx"> else {
</span><span class="cx"> ASSERT(Arg::isEarlyUse(role));
</span><span class="lines">@@ -822,7 +825,7 @@
</span><span class="cx"> ASSERT(nextMoveIndex <= m_activeMoves.size());
</span><span class="cx"> m_activeMoves.ensureSize(nextMoveIndex + 1);
</span><span class="cx">
</span><del>- for (const Arg& arg : inst.args) {
</del><ins>+ for (const Arg& arg : prevInst->args) {
</ins><span class="cx"> auto& list = m_moveList[AbsoluteTmpMapper<type>::absoluteIndex(arg.tmp())];
</span><span class="cx"> list.add(nextMoveIndex);
</span><span class="cx"> }
</span><span class="lines">@@ -832,26 +835,20 @@
</span><span class="cx"> addEdge(defTmp, liveTmp);
</span><span class="cx"> }
</span><span class="cx">
</span><del>- // The next instruction could have early clobbers. We need to consider those now.
- if (nextInst && nextInst->hasSpecial()) {
- nextInst->extraEarlyClobberedRegs().forEach([&] (Reg reg) {
- if (reg.isGPR() == (type == Arg::GP)) {
- for (const Tmp& liveTmp : localCalc.live())
- addEdge(Tmp(reg), liveTmp);
- }
- });
- }
</del><ins>+ // The next instruction could have early clobbers or early def's. We need to consider
+ // those now.
+ addEdges(nullptr, nextInst, localCalc.live());
</ins><span class="cx"> } else
</span><del>- addEdges(inst, nextInst, localCalc.live());
</del><ins>+ addEdges(prevInst, nextInst, localCalc.live());
</ins><span class="cx"> }
</span><span class="cx">
</span><del>- void addEdges(Inst& inst, Inst* nextInst, typename TmpLiveness<type>::LocalCalc::Iterable liveTmps)
</del><ins>+ void addEdges(Inst* prevInst, Inst* nextInst, typename TmpLiveness<type>::LocalCalc::Iterable liveTmps)
</ins><span class="cx"> {
</span><span class="cx"> // All the Def()s interfere with everthing live.
</span><del>- inst.forEachTmpWithExtraClobberedRegs(
- nextInst,
- [&] (const Tmp& arg, Arg::Role role, Arg::Type argType, Arg::Width) {
- if (!Arg::isDef(role) || argType != type)
</del><ins>+ Inst::forEachDefWithExtraClobberedRegs<Tmp>(
+ prevInst, nextInst,
+ [&] (const Tmp& arg, Arg::Role, Arg::Type argType, Arg::Width) {
+ if (argType != type)
</ins><span class="cx"> return;
</span><span class="cx">
</span><span class="cx"> for (const Tmp& liveTmp : liveTmps) {
</span><span class="lines">@@ -1082,6 +1079,12 @@
</span><span class="cx"> void iteratedRegisterCoalescingOnType()
</span><span class="cx"> {
</span><span class="cx"> HashSet<unsigned> unspillableTmps;
</span><ins>+
+ // FIXME: If a Tmp is used only from a Scratch role and that argument is !admitsStack, then
+ // we should add the Tmp to unspillableTmps. That will help avoid relooping only to turn the
+ // Tmp into an unspillable Tmp.
+ // https://bugs.webkit.org/show_bug.cgi?id=152699
+
</ins><span class="cx"> while (true) {
</span><span class="cx"> ++m_numIterations;
</span><span class="cx">
</span><span class="lines">@@ -1242,15 +1245,12 @@
</span><span class="cx"> break;
</span><span class="cx"> }
</span><span class="cx">
</span><del>- if (Arg::isAnyUse(role)) {
- Tmp newTmp = m_code.newTmp(type);
- insertionSet.insert(instIndex, move, inst.origin, arg, newTmp);
- tmp = newTmp;
</del><ins>+ tmp = m_code.newTmp(type);
+ unspillableTmps.add(AbsoluteTmpMapper<type>::absoluteIndex(tmp));
</ins><span class="cx">
</span><del>- // Any new Fill() should never be spilled.
- unspillableTmps.add(AbsoluteTmpMapper<type>::absoluteIndex(tmp));
- }
- if (Arg::isDef(role))
</del><ins>+ if (Arg::isAnyUse(role) && role != Arg::Scratch)
+ insertionSet.insert(instIndex, move, inst.origin, arg, tmp);
+ if (Arg::isAnyDef(role))
</ins><span class="cx"> insertionSet.insert(instIndex + 1, move, inst.origin, tmp, arg);
</span><span class="cx"> });
</span><span class="cx"> }
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreb3airAirLivenessh"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/b3/air/AirLiveness.h (194541 => 194542)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/b3/air/AirLiveness.h 2016-01-04 19:27:43 UTC (rev 194541)
+++ trunk/Source/JavaScriptCore/b3/air/AirLiveness.h 2016-01-04 19:33:35 UTC (rev 194542)
</span><span class="lines">@@ -1,5 +1,5 @@
</span><span class="cx"> /*
</span><del>- * Copyright (C) 2015 Apple Inc. All rights reserved.
</del><ins>+ * Copyright (C) 2015-2016 Apple Inc. All rights reserved.
</ins><span class="cx"> *
</span><span class="cx"> * Redistribution and use in source and binary forms, with or without
</span><span class="cx"> * modification, are permitted provided that the following conditions
</span><span class="lines">@@ -118,6 +118,13 @@
</span><span class="cx"> for (size_t instIndex = block->size(); instIndex--;)
</span><span class="cx"> localCalc.execute(instIndex);
</span><span class="cx">
</span><ins>+ // Handle the early def's of the first instruction.
+ block->at(0).forEach<typename Adapter::Thing>(
+ [&] (typename Adapter::Thing& thing, Arg::Role role, Arg::Type type, Arg::Width) {
+ if (Arg::isEarlyDef(role) && Adapter::acceptsType(type))
+ m_workset.remove(Adapter::valueToIndex(thing));
+ });
+
</ins><span class="cx"> Vector<unsigned>& liveAtHead = m_liveAtHead[block];
</span><span class="cx">
</span><span class="cx"> // We only care about Tmps that were discovered in this iteration. It is impossible
</span><span class="lines">@@ -213,10 +220,21 @@
</span><span class="cx"> {
</span><span class="cx"> Inst& inst = m_block->at(instIndex);
</span><span class="cx"> auto& workset = m_liveness.m_workset;
</span><del>- // First handle def's.
</del><ins>+
+ // First handle the early def's of the next instruction.
+ if (instIndex + 1 < m_block->size()) {
+ Inst& nextInst = m_block->at(instIndex + 1);
+ nextInst.forEach<typename Adapter::Thing>(
+ [&] (typename Adapter::Thing& thing, Arg::Role role, Arg::Type type, Arg::Width) {
+ if (Arg::isEarlyDef(role) && Adapter::acceptsType(type))
+ workset.remove(Adapter::valueToIndex(thing));
+ });
+ }
+
+ // Then handle def's.
</ins><span class="cx"> inst.forEach<typename Adapter::Thing>(
</span><span class="cx"> [&] (typename Adapter::Thing& thing, Arg::Role role, Arg::Type type, Arg::Width) {
</span><del>- if (Arg::isDef(role) && Adapter::acceptsType(type))
</del><ins>+ if (Arg::isLateDef(role) && Adapter::acceptsType(type))
</ins><span class="cx"> workset.remove(Adapter::valueToIndex(thing));
</span><span class="cx"> });
</span><span class="cx">
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreb3airAirSpillEverythingcpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/b3/air/AirSpillEverything.cpp (194541 => 194542)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/b3/air/AirSpillEverything.cpp 2016-01-04 19:27:43 UTC (rev 194541)
+++ trunk/Source/JavaScriptCore/b3/air/AirSpillEverything.cpp 2016-01-04 19:33:35 UTC (rev 194542)
</span><span class="lines">@@ -1,5 +1,5 @@
</span><span class="cx"> /*
</span><del>- * Copyright (C) 2015 Apple Inc. All rights reserved.
</del><ins>+ * Copyright (C) 2015-2016 Apple Inc. All rights reserved.
</ins><span class="cx"> *
</span><span class="cx"> * Redistribution and use in source and binary forms, with or without
</span><span class="cx"> * modification, are permitted provided that the following conditions
</span><span class="lines">@@ -53,7 +53,7 @@
</span><span class="cx">
</span><span class="cx"> usedRegisters[block].resize(block->size() + 1);
</span><span class="cx">
</span><del>- auto setUsedRegisters = [&] (unsigned index, Inst& inst) {
</del><ins>+ auto setUsedRegisters = [&] (unsigned index) {
</ins><span class="cx"> RegisterSet& registerSet = usedRegisters[block][index];
</span><span class="cx"> for (Tmp tmp : gpLocalCalc.live()) {
</span><span class="cx"> if (tmp.isReg())
</span><span class="lines">@@ -66,23 +66,20 @@
</span><span class="cx">
</span><span class="cx"> // Gotta account for dead assignments to registers. These may happen because the input
</span><span class="cx"> // code is suboptimal.
</span><del>- inst.forEachTmpWithExtraClobberedRegs(
- index < block->size() ? &block->at(index) : nullptr,
- [&registerSet] (const Tmp& tmp, Arg::Role role, Arg::Type, Arg::Width) {
- if (tmp.isReg() && Arg::isDef(role))
</del><ins>+ Inst::forEachDefWithExtraClobberedRegs<Tmp>(
+ block->get(index - 1), block->get(index),
+ [&] (const Tmp& tmp, Arg::Role, Arg::Type, Arg::Width) {
+ if (tmp.isReg())
</ins><span class="cx"> registerSet.set(tmp.reg());
</span><span class="cx"> });
</span><span class="cx"> };
</span><span class="cx">
</span><span class="cx"> for (unsigned instIndex = block->size(); instIndex--;) {
</span><del>- Inst& inst = block->at(instIndex);
- setUsedRegisters(instIndex + 1, inst);
</del><ins>+ setUsedRegisters(instIndex + 1);
</ins><span class="cx"> gpLocalCalc.execute(instIndex);
</span><span class="cx"> fpLocalCalc.execute(instIndex);
</span><span class="cx"> }
</span><del>-
- Inst nop;
- setUsedRegisters(0, nop);
</del><ins>+ setUsedRegisters(0);
</ins><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> // Allocate a stack slot for each tmp.
</span><span class="lines">@@ -133,6 +130,7 @@
</span><span class="cx"> switch (role) {
</span><span class="cx"> case Arg::Use:
</span><span class="cx"> case Arg::ColdUse:
</span><ins>+ case Arg::EarlyDef:
</ins><span class="cx"> for (Reg reg : regsInPriorityOrder(type)) {
</span><span class="cx"> if (!setBefore.get(reg)) {
</span><span class="cx"> setBefore.set(reg);
</span><span class="lines">@@ -154,6 +152,8 @@
</span><span class="cx"> case Arg::UseDef:
</span><span class="cx"> case Arg::UseZDef:
</span><span class="cx"> case Arg::LateUse:
</span><ins>+ case Arg::LateColdUse:
+ case Arg::Scratch:
</ins><span class="cx"> for (Reg reg : regsInPriorityOrder(type)) {
</span><span class="cx"> if (!setBefore.get(reg) && !setAfter.get(reg)) {
</span><span class="cx"> setAfter.set(reg);
</span><span class="lines">@@ -174,14 +174,10 @@
</span><span class="cx">
</span><span class="cx"> Opcode move = type == Arg::GP ? Move : MoveDouble;
</span><span class="cx">
</span><del>- if (Arg::isAnyUse(role)) {
- insertionSet.insert(
- instIndex, move, inst.origin, arg, tmp);
- }
- if (Arg::isDef(role)) {
- insertionSet.insert(
- instIndex + 1, move, inst.origin, tmp, arg);
- }
</del><ins>+ if (Arg::isAnyUse(role) && role != Arg::Scratch)
+ insertionSet.insert(instIndex, move, inst.origin, arg, tmp);
+ if (Arg::isAnyDef(role))
+ insertionSet.insert(instIndex + 1, move, inst.origin, tmp, arg);
</ins><span class="cx"> });
</span><span class="cx"> }
</span><span class="cx"> insertionSet.execute(block);
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreb3airAirTmpWidthcpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/b3/air/AirTmpWidth.cpp (194541 => 194542)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/b3/air/AirTmpWidth.cpp 2016-01-04 19:27:43 UTC (rev 194541)
+++ trunk/Source/JavaScriptCore/b3/air/AirTmpWidth.cpp 2016-01-04 19:33:35 UTC (rev 194542)
</span><span class="lines">@@ -119,7 +119,7 @@
</span><span class="cx">
</span><span class="cx"> if (Arg::isZDef(role))
</span><span class="cx"> widths.def = std::max(widths.def, width);
</span><del>- else if (Arg::isDef(role))
</del><ins>+ else if (Arg::isAnyDef(role))
</ins><span class="cx"> widths.def = Arg::conservativeWidth(type);
</span><span class="cx"> });
</span><span class="cx"> }
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreb3airAirUseCountsh"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/b3/air/AirUseCounts.h (194541 => 194542)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/b3/air/AirUseCounts.h 2016-01-04 19:27:43 UTC (rev 194541)
+++ trunk/Source/JavaScriptCore/b3/air/AirUseCounts.h 2016-01-04 19:33:35 UTC (rev 194542)
</span><span class="lines">@@ -84,7 +84,7 @@
</span><span class="cx"> counts.numWarmUses += frequency;
</span><span class="cx"> if (Arg::isColdUse(role))
</span><span class="cx"> counts.numColdUses += frequency;
</span><del>- if (Arg::isDef(role))
</del><ins>+ if (Arg::isAnyDef(role))
</ins><span class="cx"> counts.numDefs += frequency;
</span><span class="cx"> });
</span><span class="cx"> }
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreb3testb3cpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/b3/testb3.cpp (194541 => 194542)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/b3/testb3.cpp 2016-01-04 19:27:43 UTC (rev 194541)
+++ trunk/Source/JavaScriptCore/b3/testb3.cpp 2016-01-04 19:33:35 UTC (rev 194542)
</span><span class="lines">@@ -1,5 +1,5 @@
</span><span class="cx"> /*
</span><del>- * Copyright (C) 2015 Apple Inc. All rights reserved.
</del><ins>+ * Copyright (C) 2015-2016 Apple Inc. All rights reserved.
</ins><span class="cx"> *
</span><span class="cx"> * Redistribution and use in source and binary forms, with or without
</span><span class="cx"> * modification, are permitted provided that the following conditions
</span><span class="lines">@@ -6157,6 +6157,72 @@
</span><span class="cx"> CHECK(compileAndRun<int>(proc, 1, 2) == 3);
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+void testPatchpointGPScratch()
+{
+ Procedure proc;
+ BasicBlock* root = proc.addBlock();
+ Value* arg1 = root->appendNew<ArgumentRegValue>(proc, Origin(), GPRInfo::argumentGPR0);
+ Value* arg2 = root->appendNew<ArgumentRegValue>(proc, Origin(), GPRInfo::argumentGPR1);
+ PatchpointValue* patchpoint = root->appendNew<PatchpointValue>(proc, Int32, Origin());
+ patchpoint->append(arg1, ValueRep::SomeRegister);
+ patchpoint->append(arg2, ValueRep::SomeRegister);
+ patchpoint->numGPScratchRegisters = 2;
+ patchpoint->setGenerator(
+ [&] (CCallHelpers& jit, const StackmapGenerationParams& params) {
+ AllowMacroScratchRegisterUsage allowScratch(jit);
+ // We shouldn't have spilled the inputs, so we assert that they're in registers.
+ CHECK(params.size() == 3);
+ CHECK(params[0].isGPR());
+ CHECK(params[1].isGPR());
+ CHECK(params[2].isGPR());
+ CHECK(params.gpScratch(0) != InvalidGPRReg);
+ CHECK(params.gpScratch(0) != params[0].gpr());
+ CHECK(params.gpScratch(0) != params[1].gpr());
+ CHECK(params.gpScratch(0) != params[2].gpr());
+ CHECK(params.gpScratch(1) != InvalidGPRReg);
+ CHECK(params.gpScratch(1) != params.gpScratch(0));
+ CHECK(params.gpScratch(1) != params[0].gpr());
+ CHECK(params.gpScratch(1) != params[1].gpr());
+ CHECK(params.gpScratch(1) != params[2].gpr());
+ CHECK(!params.unavailableRegisters().get(params.gpScratch(0)));
+ CHECK(!params.unavailableRegisters().get(params.gpScratch(1)));
+ add32(jit, params[1].gpr(), params[2].gpr(), params[0].gpr());
+ });
+ root->appendNew<ControlValue>(proc, Return, Origin(), patchpoint);
+
+ CHECK(compileAndRun<int>(proc, 1, 2) == 3);
+}
+
+void testPatchpointFPScratch()
+{
+ Procedure proc;
+ BasicBlock* root = proc.addBlock();
+ Value* arg1 = root->appendNew<ArgumentRegValue>(proc, Origin(), GPRInfo::argumentGPR0);
+ Value* arg2 = root->appendNew<ArgumentRegValue>(proc, Origin(), GPRInfo::argumentGPR1);
+ PatchpointValue* patchpoint = root->appendNew<PatchpointValue>(proc, Int32, Origin());
+ patchpoint->append(arg1, ValueRep::SomeRegister);
+ patchpoint->append(arg2, ValueRep::SomeRegister);
+ patchpoint->numFPScratchRegisters = 2;
+ patchpoint->setGenerator(
+ [&] (CCallHelpers& jit, const StackmapGenerationParams& params) {
+ AllowMacroScratchRegisterUsage allowScratch(jit);
+ // We shouldn't have spilled the inputs, so we assert that they're in registers.
+ CHECK(params.size() == 3);
+ CHECK(params[0].isGPR());
+ CHECK(params[1].isGPR());
+ CHECK(params[2].isGPR());
+ CHECK(params.fpScratch(0) != InvalidFPRReg);
+ CHECK(params.fpScratch(1) != InvalidFPRReg);
+ CHECK(params.fpScratch(1) != params.fpScratch(0));
+ CHECK(!params.unavailableRegisters().get(params.fpScratch(0)));
+ CHECK(!params.unavailableRegisters().get(params.fpScratch(1)));
+ add32(jit, params[1].gpr(), params[2].gpr(), params[0].gpr());
+ });
+ root->appendNew<ControlValue>(proc, Return, Origin(), patchpoint);
+
+ CHECK(compileAndRun<int>(proc, 1, 2) == 3);
+}
+
</ins><span class="cx"> void testPatchpointLotsOfLateAnys()
</span><span class="cx"> {
</span><span class="cx"> Procedure proc;
</span><span class="lines">@@ -9598,6 +9664,8 @@
</span><span class="cx"> RUN(testPatchpointFixedRegister());
</span><span class="cx"> RUN(testPatchpointAny(ValueRep::WarmAny));
</span><span class="cx"> RUN(testPatchpointAny(ValueRep::ColdAny));
</span><ins>+ RUN(testPatchpointGPScratch());
+ RUN(testPatchpointFPScratch());
</ins><span class="cx"> RUN(testPatchpointLotsOfLateAnys());
</span><span class="cx"> RUN(testPatchpointAnyImm(ValueRep::WarmAny));
</span><span class="cx"> RUN(testPatchpointAnyImm(ValueRep::ColdAny));
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoredfgDFGCommonh"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/dfg/DFGCommon.h (194541 => 194542)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/dfg/DFGCommon.h 2016-01-04 19:27:43 UTC (rev 194541)
+++ trunk/Source/JavaScriptCore/dfg/DFGCommon.h 2016-01-04 19:33:35 UTC (rev 194542)
</span><span class="lines">@@ -38,7 +38,7 @@
</span><span class="cx"> // We are in the middle of an experimental transition from LLVM to B3 as the backend for the FTL. We don't
</span><span class="cx"> // yet know how it will turn out. For now, this flag will control whether FTL uses B3. Remember to set this
</span><span class="cx"> // to 0 before committing!
</span><del>-#define FTL_USES_B3 0
</del><ins>+#define FTL_USES_B3 1
</ins><span class="cx">
</span><span class="cx"> struct Node;
</span><span class="cx">
</span></span></pre>
</div>
</div>
</body>
</html>
More information about the webkit-changes
mailing list