<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
"http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head><meta http-equiv="content-type" content="text/html; charset=utf-8" />
<title>[213714] trunk</title>
</head>
<body>
<style type="text/css"><!--
#msg dl.meta { border: 1px #006 solid; background: #369; padding: 6px; color: #fff; }
#msg dl.meta dt { float: left; width: 6em; font-weight: bold; }
#msg dt:after { content:':';}
#msg dl, #msg dt, #msg ul, #msg li, #header, #footer, #logmsg { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt; }
#msg dl a { font-weight: bold}
#msg dl a:link { color:#fc3; }
#msg dl a:active { color:#ff0; }
#msg dl a:visited { color:#cc6; }
h3 { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt; font-weight: bold; }
#msg pre { overflow: auto; background: #ffc; border: 1px #fa0 solid; padding: 6px; }
#logmsg { background: #ffc; border: 1px #fa0 solid; padding: 1em 1em 0 1em; }
#logmsg p, #logmsg pre, #logmsg blockquote { margin: 0 0 1em 0; }
#logmsg p, #logmsg li, #logmsg dt, #logmsg dd { line-height: 14pt; }
#logmsg h1, #logmsg h2, #logmsg h3, #logmsg h4, #logmsg h5, #logmsg h6 { margin: .5em 0; }
#logmsg h1:first-child, #logmsg h2:first-child, #logmsg h3:first-child, #logmsg h4:first-child, #logmsg h5:first-child, #logmsg h6:first-child { margin-top: 0; }
#logmsg ul, #logmsg ol { padding: 0; list-style-position: inside; margin: 0 0 0 1em; }
#logmsg ul { text-indent: -1em; padding-left: 1em; }#logmsg ol { text-indent: -1.5em; padding-left: 1.5em; }
#logmsg > ul, #logmsg > ol { margin: 0 0 1em 0; }
#logmsg pre { background: #eee; padding: 1em; }
#logmsg blockquote { border: 1px solid #fa0; border-left-width: 10px; padding: 1em 1em 0 1em; background: white;}
#logmsg dl { margin: 0; }
#logmsg dt { font-weight: bold; }
#logmsg dd { margin: 0; padding: 0 0 0.5em 0; }
#logmsg dd:before { content:'\00bb';}
#logmsg table { border-spacing: 0px; border-collapse: collapse; border-top: 4px solid #fa0; border-bottom: 1px solid #fa0; background: #fff; }
#logmsg table th { text-align: left; font-weight: normal; padding: 0.2em 0.5em; border-top: 1px dotted #fa0; }
#logmsg table td { text-align: right; border-top: 1px dotted #fa0; padding: 0.2em 0.5em; }
#logmsg table thead th { text-align: center; border-bottom: 1px solid #fa0; }
#logmsg table th.Corner { text-align: left; }
#logmsg hr { border: none 0; border-top: 2px dashed #fa0; height: 1px; }
#header, #footer { color: #fff; background: #636; border: 1px #300 solid; padding: 6px; }
#patch { width: 100%; }
#patch h4 {font-family: verdana,arial,helvetica,sans-serif;font-size:10pt;padding:8px;background:#369;color:#fff;margin:0;}
#patch .propset h4, #patch .binary h4 {margin:0;}
#patch pre {padding:0;line-height:1.2em;margin:0;}
#patch .diff {width:100%;background:#eee;padding: 0 0 10px 0;overflow:auto;}
#patch .propset .diff, #patch .binary .diff {padding:10px 0;}
#patch span {display:block;padding:0 10px;}
#patch .modfile, #patch .addfile, #patch .delfile, #patch .propset, #patch .binary, #patch .copfile {border:1px solid #ccc;margin:10px 0;}
#patch ins {background:#dfd;text-decoration:none;display:block;padding:0 10px;}
#patch del {background:#fdd;text-decoration:none;display:block;padding:0 10px;}
#patch .lines, .info {color:#888;background:#fff;}
--></style>
<div id="msg">
<dl class="meta">
<dt>Revision</dt> <dd><a href="http://trac.webkit.org/projects/webkit/changeset/213714">213714</a></dd>
<dt>Author</dt> <dd>fpizlo@apple.com</dd>
<dt>Date</dt> <dd>2017-03-10 09:49:42 -0800 (Fri, 10 Mar 2017)</dd>
</dl>
<h3>Log Message</h3>
<pre>B3 should have comprehensive support for atomic operations
https://bugs.webkit.org/show_bug.cgi?id=162349
Reviewed by Keith Miller.
Source/JavaScriptCore:
This adds the following capabilities to B3:
- Atomic weak/strong unfenced/fenced compare-and-swap
- Atomic add/sub/or/and/xor/xchg
- Acquire/release fencing on loads/stores
- Fenceless load-load dependencies
This adds lowering to the following instructions on x86:
- lock cmpxchg
- lock xadd
- lock add/sub/or/and/xor/xchg
This adds lowering to the following instructions on ARM64:
- ldar and friends
- stlr and friends
- ldxr and friends (unfenced LL)
- stxr and friends (unfended SC)
- ldaxr and friends (fenced LL)
- stlxr and friends (fenced SC)
- eor as a fenceless load-load dependency
This does instruction selection pattern matching to ensure that weak/strong CAS and all of the
variants of fences and atomic math ops get lowered to the best possible instruction sequence.
For example, we support the Equal(AtomicStrongCAS(expected, ...), expected) pattern and a bunch
of its friends. You can say Branch(Equal(AtomicStrongCAS(expected, ...), expected)) and it will
generate the best possible branch sequence on x86 and ARM64.
B3 now knows how to model all of the kinds of fencing. It knows that acq loads are ordered with
respect to each other and with respect to rel stores, creating sequential consistency that
transcends just the acq/rel fences themselves (see Effects::fence). It knows that the phantom
fence effects may only target some abstract heaps but not others, so that load elimination and
store sinking can still operate across fences if you just tell B3 that the fence does not alias
those accesses. This makes it super easy to teach B3 that some of your heap is thread-local.
Even better, it lets you express fine-grained dependencies where the atomics that affect one
property in shared memory do not clobber non-atomics that ffect some other property in shared
memory.
One of my favorite features is Depend, which allows you to express load-load dependencies. On
x86 it lowers to nothing, while on ARM64 it lowers to eor.
This also exposes a common atomicWeakCAS API to the x86_64/ARM64 MacroAssemblers. Same for
acq/rel. JSC's 64-bit JITs are now a happy concurrency playground.
This doesn't yet expose the functionality to JS or wasm. SAB still uses the non-intrinsic
implementations of the Atomics object, for now.
* CMakeLists.txt:
* JavaScriptCore.xcodeproj/project.pbxproj:
* assembler/ARM64Assembler.h:
(JSC::ARM64Assembler::ldar):
(JSC::ARM64Assembler::ldxr):
(JSC::ARM64Assembler::ldaxr):
(JSC::ARM64Assembler::stxr):
(JSC::ARM64Assembler::stlr):
(JSC::ARM64Assembler::stlxr):
(JSC::ARM64Assembler::excepnGenerationImmMask):
(JSC::ARM64Assembler::exoticLoad):
(JSC::ARM64Assembler::storeRelease):
(JSC::ARM64Assembler::exoticStore):
* assembler/AbstractMacroAssembler.cpp: Added.
(WTF::printInternal):
* assembler/AbstractMacroAssembler.h:
(JSC::AbstractMacroAssemblerBase::invert):
* assembler/MacroAssembler.h:
* assembler/MacroAssemblerARM64.h:
(JSC::MacroAssemblerARM64::loadAcq8SignedExtendTo32):
(JSC::MacroAssemblerARM64::loadAcq8):
(JSC::MacroAssemblerARM64::storeRel8):
(JSC::MacroAssemblerARM64::loadAcq16SignedExtendTo32):
(JSC::MacroAssemblerARM64::loadAcq16):
(JSC::MacroAssemblerARM64::storeRel16):
(JSC::MacroAssemblerARM64::loadAcq32):
(JSC::MacroAssemblerARM64::loadAcq64):
(JSC::MacroAssemblerARM64::storeRel32):
(JSC::MacroAssemblerARM64::storeRel64):
(JSC::MacroAssemblerARM64::loadLink8):
(JSC::MacroAssemblerARM64::loadLinkAcq8):
(JSC::MacroAssemblerARM64::storeCond8):
(JSC::MacroAssemblerARM64::storeCondRel8):
(JSC::MacroAssemblerARM64::loadLink16):
(JSC::MacroAssemblerARM64::loadLinkAcq16):
(JSC::MacroAssemblerARM64::storeCond16):
(JSC::MacroAssemblerARM64::storeCondRel16):
(JSC::MacroAssemblerARM64::loadLink32):
(JSC::MacroAssemblerARM64::loadLinkAcq32):
(JSC::MacroAssemblerARM64::storeCond32):
(JSC::MacroAssemblerARM64::storeCondRel32):
(JSC::MacroAssemblerARM64::loadLink64):
(JSC::MacroAssemblerARM64::loadLinkAcq64):
(JSC::MacroAssemblerARM64::storeCond64):
(JSC::MacroAssemblerARM64::storeCondRel64):
(JSC::MacroAssemblerARM64::atomicStrongCAS8):
(JSC::MacroAssemblerARM64::atomicStrongCAS16):
(JSC::MacroAssemblerARM64::atomicStrongCAS32):
(JSC::MacroAssemblerARM64::atomicStrongCAS64):
(JSC::MacroAssemblerARM64::atomicRelaxedStrongCAS8):
(JSC::MacroAssemblerARM64::atomicRelaxedStrongCAS16):
(JSC::MacroAssemblerARM64::atomicRelaxedStrongCAS32):
(JSC::MacroAssemblerARM64::atomicRelaxedStrongCAS64):
(JSC::MacroAssemblerARM64::branchAtomicWeakCAS8):
(JSC::MacroAssemblerARM64::branchAtomicWeakCAS16):
(JSC::MacroAssemblerARM64::branchAtomicWeakCAS32):
(JSC::MacroAssemblerARM64::branchAtomicWeakCAS64):
(JSC::MacroAssemblerARM64::branchAtomicRelaxedWeakCAS8):
(JSC::MacroAssemblerARM64::branchAtomicRelaxedWeakCAS16):
(JSC::MacroAssemblerARM64::branchAtomicRelaxedWeakCAS32):
(JSC::MacroAssemblerARM64::branchAtomicRelaxedWeakCAS64):
(JSC::MacroAssemblerARM64::depend32):
(JSC::MacroAssemblerARM64::depend64):
(JSC::MacroAssemblerARM64::loadLink):
(JSC::MacroAssemblerARM64::loadLinkAcq):
(JSC::MacroAssemblerARM64::storeCond):
(JSC::MacroAssemblerARM64::storeCondRel):
(JSC::MacroAssemblerARM64::signExtend):
(JSC::MacroAssemblerARM64::branch):
(JSC::MacroAssemblerARM64::atomicStrongCAS):
(JSC::MacroAssemblerARM64::atomicRelaxedStrongCAS):
(JSC::MacroAssemblerARM64::branchAtomicWeakCAS):
(JSC::MacroAssemblerARM64::branchAtomicRelaxedWeakCAS):
(JSC::MacroAssemblerARM64::extractSimpleAddress):
(JSC::MacroAssemblerARM64::signExtend<8>):
(JSC::MacroAssemblerARM64::signExtend<16>):
(JSC::MacroAssemblerARM64::branch<64>):
* assembler/MacroAssemblerX86Common.h:
(JSC::MacroAssemblerX86Common::add32):
(JSC::MacroAssemblerX86Common::and32):
(JSC::MacroAssemblerX86Common::and16):
(JSC::MacroAssemblerX86Common::and8):
(JSC::MacroAssemblerX86Common::neg32):
(JSC::MacroAssemblerX86Common::neg16):
(JSC::MacroAssemblerX86Common::neg8):
(JSC::MacroAssemblerX86Common::or32):
(JSC::MacroAssemblerX86Common::or16):
(JSC::MacroAssemblerX86Common::or8):
(JSC::MacroAssemblerX86Common::sub16):
(JSC::MacroAssemblerX86Common::sub8):
(JSC::MacroAssemblerX86Common::sub32):
(JSC::MacroAssemblerX86Common::xor32):
(JSC::MacroAssemblerX86Common::xor16):
(JSC::MacroAssemblerX86Common::xor8):
(JSC::MacroAssemblerX86Common::not32):
(JSC::MacroAssemblerX86Common::not16):
(JSC::MacroAssemblerX86Common::not8):
(JSC::MacroAssemblerX86Common::store16):
(JSC::MacroAssemblerX86Common::atomicStrongCAS8):
(JSC::MacroAssemblerX86Common::atomicStrongCAS16):
(JSC::MacroAssemblerX86Common::atomicStrongCAS32):
(JSC::MacroAssemblerX86Common::branchAtomicStrongCAS8):
(JSC::MacroAssemblerX86Common::branchAtomicStrongCAS16):
(JSC::MacroAssemblerX86Common::branchAtomicStrongCAS32):
(JSC::MacroAssemblerX86Common::atomicWeakCAS8):
(JSC::MacroAssemblerX86Common::atomicWeakCAS16):
(JSC::MacroAssemblerX86Common::atomicWeakCAS32):
(JSC::MacroAssemblerX86Common::branchAtomicWeakCAS8):
(JSC::MacroAssemblerX86Common::branchAtomicWeakCAS16):
(JSC::MacroAssemblerX86Common::branchAtomicWeakCAS32):
(JSC::MacroAssemblerX86Common::atomicRelaxedWeakCAS8):
(JSC::MacroAssemblerX86Common::atomicRelaxedWeakCAS16):
(JSC::MacroAssemblerX86Common::atomicRelaxedWeakCAS32):
(JSC::MacroAssemblerX86Common::branchAtomicRelaxedWeakCAS8):
(JSC::MacroAssemblerX86Common::branchAtomicRelaxedWeakCAS16):
(JSC::MacroAssemblerX86Common::branchAtomicRelaxedWeakCAS32):
(JSC::MacroAssemblerX86Common::atomicAdd8):
(JSC::MacroAssemblerX86Common::atomicAdd16):
(JSC::MacroAssemblerX86Common::atomicAdd32):
(JSC::MacroAssemblerX86Common::atomicSub8):
(JSC::MacroAssemblerX86Common::atomicSub16):
(JSC::MacroAssemblerX86Common::atomicSub32):
(JSC::MacroAssemblerX86Common::atomicAnd8):
(JSC::MacroAssemblerX86Common::atomicAnd16):
(JSC::MacroAssemblerX86Common::atomicAnd32):
(JSC::MacroAssemblerX86Common::atomicOr8):
(JSC::MacroAssemblerX86Common::atomicOr16):
(JSC::MacroAssemblerX86Common::atomicOr32):
(JSC::MacroAssemblerX86Common::atomicXor8):
(JSC::MacroAssemblerX86Common::atomicXor16):
(JSC::MacroAssemblerX86Common::atomicXor32):
(JSC::MacroAssemblerX86Common::atomicNeg8):
(JSC::MacroAssemblerX86Common::atomicNeg16):
(JSC::MacroAssemblerX86Common::atomicNeg32):
(JSC::MacroAssemblerX86Common::atomicNot8):
(JSC::MacroAssemblerX86Common::atomicNot16):
(JSC::MacroAssemblerX86Common::atomicNot32):
(JSC::MacroAssemblerX86Common::atomicXchgAdd8):
(JSC::MacroAssemblerX86Common::atomicXchgAdd16):
(JSC::MacroAssemblerX86Common::atomicXchgAdd32):
(JSC::MacroAssemblerX86Common::atomicXchg8):
(JSC::MacroAssemblerX86Common::atomicXchg16):
(JSC::MacroAssemblerX86Common::atomicXchg32):
(JSC::MacroAssemblerX86Common::loadAcq8):
(JSC::MacroAssemblerX86Common::loadAcq8SignedExtendTo32):
(JSC::MacroAssemblerX86Common::loadAcq16):
(JSC::MacroAssemblerX86Common::loadAcq16SignedExtendTo32):
(JSC::MacroAssemblerX86Common::loadAcq32):
(JSC::MacroAssemblerX86Common::storeRel8):
(JSC::MacroAssemblerX86Common::storeRel16):
(JSC::MacroAssemblerX86Common::storeRel32):
(JSC::MacroAssemblerX86Common::storeFence):
(JSC::MacroAssemblerX86Common::loadFence):
(JSC::MacroAssemblerX86Common::replaceWithJump):
(JSC::MacroAssemblerX86Common::maxJumpReplacementSize):
(JSC::MacroAssemblerX86Common::patchableJumpSize):
(JSC::MacroAssemblerX86Common::supportsFloatingPointRounding):
(JSC::MacroAssemblerX86Common::supportsAVX):
(JSC::MacroAssemblerX86Common::updateEax1EcxFlags):
(JSC::MacroAssemblerX86Common::x86Condition):
(JSC::MacroAssemblerX86Common::atomicStrongCAS):
(JSC::MacroAssemblerX86Common::branchAtomicStrongCAS):
* assembler/MacroAssemblerX86_64.h:
(JSC::MacroAssemblerX86_64::add64):
(JSC::MacroAssemblerX86_64::and64):
(JSC::MacroAssemblerX86_64::neg64):
(JSC::MacroAssemblerX86_64::or64):
(JSC::MacroAssemblerX86_64::sub64):
(JSC::MacroAssemblerX86_64::xor64):
(JSC::MacroAssemblerX86_64::not64):
(JSC::MacroAssemblerX86_64::store64):
(JSC::MacroAssemblerX86_64::atomicStrongCAS64):
(JSC::MacroAssemblerX86_64::branchAtomicStrongCAS64):
(JSC::MacroAssemblerX86_64::atomicWeakCAS64):
(JSC::MacroAssemblerX86_64::branchAtomicWeakCAS64):
(JSC::MacroAssemblerX86_64::atomicRelaxedWeakCAS64):
(JSC::MacroAssemblerX86_64::branchAtomicRelaxedWeakCAS64):
(JSC::MacroAssemblerX86_64::atomicAdd64):
(JSC::MacroAssemblerX86_64::atomicSub64):
(JSC::MacroAssemblerX86_64::atomicAnd64):
(JSC::MacroAssemblerX86_64::atomicOr64):
(JSC::MacroAssemblerX86_64::atomicXor64):
(JSC::MacroAssemblerX86_64::atomicNeg64):
(JSC::MacroAssemblerX86_64::atomicNot64):
(JSC::MacroAssemblerX86_64::atomicXchgAdd64):
(JSC::MacroAssemblerX86_64::atomicXchg64):
(JSC::MacroAssemblerX86_64::loadAcq64):
(JSC::MacroAssemblerX86_64::storeRel64):
* assembler/X86Assembler.h:
(JSC::X86Assembler::addl_mr):
(JSC::X86Assembler::addq_mr):
(JSC::X86Assembler::addq_rm):
(JSC::X86Assembler::addq_im):
(JSC::X86Assembler::andl_mr):
(JSC::X86Assembler::andl_rm):
(JSC::X86Assembler::andw_rm):
(JSC::X86Assembler::andb_rm):
(JSC::X86Assembler::andl_im):
(JSC::X86Assembler::andw_im):
(JSC::X86Assembler::andb_im):
(JSC::X86Assembler::andq_mr):
(JSC::X86Assembler::andq_rm):
(JSC::X86Assembler::andq_im):
(JSC::X86Assembler::incq_m):
(JSC::X86Assembler::negq_m):
(JSC::X86Assembler::negl_m):
(JSC::X86Assembler::negw_m):
(JSC::X86Assembler::negb_m):
(JSC::X86Assembler::notl_m):
(JSC::X86Assembler::notw_m):
(JSC::X86Assembler::notb_m):
(JSC::X86Assembler::notq_m):
(JSC::X86Assembler::orl_mr):
(JSC::X86Assembler::orl_rm):
(JSC::X86Assembler::orw_rm):
(JSC::X86Assembler::orb_rm):
(JSC::X86Assembler::orl_im):
(JSC::X86Assembler::orw_im):
(JSC::X86Assembler::orb_im):
(JSC::X86Assembler::orq_mr):
(JSC::X86Assembler::orq_rm):
(JSC::X86Assembler::orq_im):
(JSC::X86Assembler::subl_mr):
(JSC::X86Assembler::subl_rm):
(JSC::X86Assembler::subw_rm):
(JSC::X86Assembler::subb_rm):
(JSC::X86Assembler::subl_im):
(JSC::X86Assembler::subw_im):
(JSC::X86Assembler::subb_im):
(JSC::X86Assembler::subq_mr):
(JSC::X86Assembler::subq_rm):
(JSC::X86Assembler::subq_im):
(JSC::X86Assembler::xorl_mr):
(JSC::X86Assembler::xorl_rm):
(JSC::X86Assembler::xorl_im):
(JSC::X86Assembler::xorw_rm):
(JSC::X86Assembler::xorw_im):
(JSC::X86Assembler::xorb_rm):
(JSC::X86Assembler::xorb_im):
(JSC::X86Assembler::xorq_im):
(JSC::X86Assembler::xorq_rm):
(JSC::X86Assembler::xorq_mr):
(JSC::X86Assembler::xchgb_rm):
(JSC::X86Assembler::xchgw_rm):
(JSC::X86Assembler::xchgl_rm):
(JSC::X86Assembler::xchgq_rm):
(JSC::X86Assembler::movw_im):
(JSC::X86Assembler::movq_i32m):
(JSC::X86Assembler::cmpxchgb_rm):
(JSC::X86Assembler::cmpxchgw_rm):
(JSC::X86Assembler::cmpxchgl_rm):
(JSC::X86Assembler::cmpxchgq_rm):
(JSC::X86Assembler::xaddb_rm):
(JSC::X86Assembler::xaddw_rm):
(JSC::X86Assembler::xaddl_rm):
(JSC::X86Assembler::xaddq_rm):
(JSC::X86Assembler::X86InstructionFormatter::SingleInstructionBufferWriter::memoryModRM):
* b3/B3AtomicValue.cpp: Added.
(JSC::B3::AtomicValue::~AtomicValue):
(JSC::B3::AtomicValue::dumpMeta):
(JSC::B3::AtomicValue::cloneImpl):
(JSC::B3::AtomicValue::AtomicValue):
* b3/B3AtomicValue.h: Added.
* b3/B3BasicBlock.h:
* b3/B3BlockInsertionSet.cpp:
(JSC::B3::BlockInsertionSet::BlockInsertionSet):
(JSC::B3::BlockInsertionSet::insert): Deleted.
(JSC::B3::BlockInsertionSet::insertBefore): Deleted.
(JSC::B3::BlockInsertionSet::insertAfter): Deleted.
(JSC::B3::BlockInsertionSet::execute): Deleted.
* b3/B3BlockInsertionSet.h:
* b3/B3Effects.cpp:
(JSC::B3::Effects::interferes):
(JSC::B3::Effects::operator==):
(JSC::B3::Effects::dump):
* b3/B3Effects.h:
(JSC::B3::Effects::forCall):
(JSC::B3::Effects::mustExecute):
* b3/B3EliminateCommonSubexpressions.cpp:
* b3/B3Generate.cpp:
(JSC::B3::generateToAir):
* b3/B3GenericBlockInsertionSet.h: Added.
(JSC::B3::GenericBlockInsertionSet::GenericBlockInsertionSet):
(JSC::B3::GenericBlockInsertionSet::insert):
(JSC::B3::GenericBlockInsertionSet::insertBefore):
(JSC::B3::GenericBlockInsertionSet::insertAfter):
(JSC::B3::GenericBlockInsertionSet::execute):
* b3/B3HeapRange.h:
(JSC::B3::HeapRange::operator|):
* b3/B3InsertionSet.cpp:
(JSC::B3::InsertionSet::insertClone):
* b3/B3InsertionSet.h:
* b3/B3LegalizeMemoryOffsets.cpp:
* b3/B3LowerMacros.cpp:
(JSC::B3::lowerMacros):
* b3/B3LowerMacrosAfterOptimizations.cpp:
* b3/B3LowerToAir.cpp:
(JSC::B3::Air::LowerToAir::LowerToAir):
(JSC::B3::Air::LowerToAir::run):
(JSC::B3::Air::LowerToAir::effectiveAddr):
(JSC::B3::Air::LowerToAir::addr):
(JSC::B3::Air::LowerToAir::loadPromiseAnyOpcode):
(JSC::B3::Air::LowerToAir::appendShift):
(JSC::B3::Air::LowerToAir::tryAppendStoreBinOp):
(JSC::B3::Air::LowerToAir::storeOpcode):
(JSC::B3::Air::LowerToAir::createStore):
(JSC::B3::Air::LowerToAir::finishAppendingInstructions):
(JSC::B3::Air::LowerToAir::newBlock):
(JSC::B3::Air::LowerToAir::splitBlock):
(JSC::B3::Air::LowerToAir::fillStackmap):
(JSC::B3::Air::LowerToAir::appendX86Div):
(JSC::B3::Air::LowerToAir::appendX86UDiv):
(JSC::B3::Air::LowerToAir::loadLinkOpcode):
(JSC::B3::Air::LowerToAir::storeCondOpcode):
(JSC::B3::Air::LowerToAir::appendCAS):
(JSC::B3::Air::LowerToAir::appendVoidAtomic):
(JSC::B3::Air::LowerToAir::appendGeneralAtomic):
(JSC::B3::Air::LowerToAir::lower):
(JSC::B3::Air::LowerToAir::lowerX86Div): Deleted.
(JSC::B3::Air::LowerToAir::lowerX86UDiv): Deleted.
* b3/B3LowerToAir.h:
* b3/B3MemoryValue.cpp:
(JSC::B3::MemoryValue::isLegalOffset):
(JSC::B3::MemoryValue::accessType):
(JSC::B3::MemoryValue::accessBank):
(JSC::B3::MemoryValue::accessByteSize):
(JSC::B3::MemoryValue::dumpMeta):
(JSC::B3::MemoryValue::MemoryValue):
(JSC::B3::MemoryValue::accessWidth): Deleted.
* b3/B3MemoryValue.h:
* b3/B3MemoryValueInlines.h: Added.
(JSC::B3::MemoryValue::isLegalOffset):
(JSC::B3::MemoryValue::requiresSimpleAddr):
(JSC::B3::MemoryValue::accessWidth):
* b3/B3MoveConstants.cpp:
* b3/B3NativeTraits.h: Added.
* b3/B3Opcode.cpp:
(JSC::B3::storeOpcode):
(WTF::printInternal):
* b3/B3Opcode.h:
(JSC::B3::isLoad):
(JSC::B3::isStore):
(JSC::B3::isLoadStore):
(JSC::B3::isAtomic):
(JSC::B3::isAtomicCAS):
(JSC::B3::isAtomicXchg):
(JSC::B3::isMemoryAccess):
(JSC::B3::signExtendOpcode):
* b3/B3Procedure.cpp:
(JSC::B3::Procedure::dump):
* b3/B3Procedure.h:
(JSC::B3::Procedure::hasQuirks):
(JSC::B3::Procedure::setHasQuirks):
* b3/B3PureCSE.cpp:
(JSC::B3::pureCSE):
* b3/B3PureCSE.h:
* b3/B3ReduceStrength.cpp:
* b3/B3Validate.cpp:
* b3/B3Value.cpp:
(JSC::B3::Value::returnsBool):
(JSC::B3::Value::effects):
(JSC::B3::Value::key):
(JSC::B3::Value::performSubstitution):
(JSC::B3::Value::typeFor):
* b3/B3Value.h:
* b3/B3Width.cpp:
(JSC::B3::bestType):
* b3/B3Width.h:
(JSC::B3::canonicalWidth):
(JSC::B3::isCanonicalWidth):
(JSC::B3::mask):
* b3/air/AirArg.cpp:
(JSC::B3::Air::Arg::jsHash):
(JSC::B3::Air::Arg::dump):
(WTF::printInternal):
* b3/air/AirArg.h:
(JSC::B3::Air::Arg::isAnyUse):
(JSC::B3::Air::Arg::isColdUse):
(JSC::B3::Air::Arg::cooled):
(JSC::B3::Air::Arg::isEarlyUse):
(JSC::B3::Air::Arg::isLateUse):
(JSC::B3::Air::Arg::isAnyDef):
(JSC::B3::Air::Arg::isEarlyDef):
(JSC::B3::Air::Arg::isLateDef):
(JSC::B3::Air::Arg::isZDef):
(JSC::B3::Air::Arg::simpleAddr):
(JSC::B3::Air::Arg::statusCond):
(JSC::B3::Air::Arg::isSimpleAddr):
(JSC::B3::Air::Arg::isMemory):
(JSC::B3::Air::Arg::isStatusCond):
(JSC::B3::Air::Arg::isCondition):
(JSC::B3::Air::Arg::ptr):
(JSC::B3::Air::Arg::base):
(JSC::B3::Air::Arg::isGP):
(JSC::B3::Air::Arg::isFP):
(JSC::B3::Air::Arg::isValidForm):
(JSC::B3::Air::Arg::forEachTmpFast):
(JSC::B3::Air::Arg::forEachTmp):
(JSC::B3::Air::Arg::asAddress):
(JSC::B3::Air::Arg::asStatusCondition):
(JSC::B3::Air::Arg::isInvertible):
(JSC::B3::Air::Arg::inverted):
* b3/air/AirBasicBlock.cpp:
(JSC::B3::Air::BasicBlock::setSuccessors):
* b3/air/AirBasicBlock.h:
* b3/air/AirBlockInsertionSet.cpp: Added.
(JSC::B3::Air::BlockInsertionSet::BlockInsertionSet):
(JSC::B3::Air::BlockInsertionSet::~BlockInsertionSet):
* b3/air/AirBlockInsertionSet.h: Added.
* b3/air/AirDumpAsJS.cpp: Removed.
* b3/air/AirDumpAsJS.h: Removed.
* b3/air/AirEliminateDeadCode.cpp:
(JSC::B3::Air::eliminateDeadCode):
* b3/air/AirGenerate.cpp:
(JSC::B3::Air::prepareForGeneration):
* b3/air/AirInstInlines.h:
(JSC::B3::Air::isAtomicStrongCASValid):
(JSC::B3::Air::isBranchAtomicStrongCASValid):
(JSC::B3::Air::isAtomicStrongCAS8Valid):
(JSC::B3::Air::isAtomicStrongCAS16Valid):
(JSC::B3::Air::isAtomicStrongCAS32Valid):
(JSC::B3::Air::isAtomicStrongCAS64Valid):
(JSC::B3::Air::isBranchAtomicStrongCAS8Valid):
(JSC::B3::Air::isBranchAtomicStrongCAS16Valid):
(JSC::B3::Air::isBranchAtomicStrongCAS32Valid):
(JSC::B3::Air::isBranchAtomicStrongCAS64Valid):
* b3/air/AirOpcode.opcodes:
* b3/air/AirOptimizeBlockOrder.cpp:
(JSC::B3::Air::optimizeBlockOrder):
* b3/air/AirPadInterference.cpp:
(JSC::B3::Air::padInterference):
* b3/air/AirSpillEverything.cpp:
(JSC::B3::Air::spillEverything):
* b3/air/opcode_generator.rb:
* b3/testb3.cpp:
(JSC::B3::testLoadAcq42):
(JSC::B3::testStoreRelAddLoadAcq32):
(JSC::B3::testStoreRelAddLoadAcq8):
(JSC::B3::testStoreRelAddFenceLoadAcq8):
(JSC::B3::testStoreRelAddLoadAcq16):
(JSC::B3::testStoreRelAddLoadAcq64):
(JSC::B3::testTrappingStoreElimination):
(JSC::B3::testX86LeaAddAdd):
(JSC::B3::testX86LeaAddShlLeftScale1):
(JSC::B3::testAtomicWeakCAS):
(JSC::B3::testAtomicStrongCAS):
(JSC::B3::testAtomicXchg):
(JSC::B3::testDepend32):
(JSC::B3::testDepend64):
(JSC::B3::run):
* runtime/Options.h:
Websites/webkit.org:
Document the new opcodes!
* docs/b3/intermediate-representation.html:</pre>
<h3>Modified Paths</h3>
<ul>
<li><a href="#trunkSourceJavaScriptCoreCMakeListstxt">trunk/Source/JavaScriptCore/CMakeLists.txt</a></li>
<li><a href="#trunkSourceJavaScriptCoreChangeLog">trunk/Source/JavaScriptCore/ChangeLog</a></li>
<li><a href="#trunkSourceJavaScriptCoreJavaScriptCorexcodeprojprojectpbxproj">trunk/Source/JavaScriptCore/JavaScriptCore.xcodeproj/project.pbxproj</a></li>
<li><a href="#trunkSourceJavaScriptCoreassemblerARM64Assemblerh">trunk/Source/JavaScriptCore/assembler/ARM64Assembler.h</a></li>
<li><a href="#trunkSourceJavaScriptCoreassemblerAbstractMacroAssemblerh">trunk/Source/JavaScriptCore/assembler/AbstractMacroAssembler.h</a></li>
<li><a href="#trunkSourceJavaScriptCoreassemblerMacroAssemblerh">trunk/Source/JavaScriptCore/assembler/MacroAssembler.h</a></li>
<li><a href="#trunkSourceJavaScriptCoreassemblerMacroAssemblerARM64h">trunk/Source/JavaScriptCore/assembler/MacroAssemblerARM64.h</a></li>
<li><a href="#trunkSourceJavaScriptCoreassemblerMacroAssemblerX86Commonh">trunk/Source/JavaScriptCore/assembler/MacroAssemblerX86Common.h</a></li>
<li><a href="#trunkSourceJavaScriptCoreassemblerMacroAssemblerX86_64h">trunk/Source/JavaScriptCore/assembler/MacroAssemblerX86_64.h</a></li>
<li><a href="#trunkSourceJavaScriptCoreassemblerX86Assemblerh">trunk/Source/JavaScriptCore/assembler/X86Assembler.h</a></li>
<li><a href="#trunkSourceJavaScriptCoreb3B3BasicBlockh">trunk/Source/JavaScriptCore/b3/B3BasicBlock.h</a></li>
<li><a href="#trunkSourceJavaScriptCoreb3B3BlockInsertionSetcpp">trunk/Source/JavaScriptCore/b3/B3BlockInsertionSet.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoreb3B3BlockInsertionSeth">trunk/Source/JavaScriptCore/b3/B3BlockInsertionSet.h</a></li>
<li><a href="#trunkSourceJavaScriptCoreb3B3Effectscpp">trunk/Source/JavaScriptCore/b3/B3Effects.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoreb3B3Effectsh">trunk/Source/JavaScriptCore/b3/B3Effects.h</a></li>
<li><a href="#trunkSourceJavaScriptCoreb3B3EliminateCommonSubexpressionscpp">trunk/Source/JavaScriptCore/b3/B3EliminateCommonSubexpressions.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoreb3B3Generatecpp">trunk/Source/JavaScriptCore/b3/B3Generate.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoreb3B3HeapRangeh">trunk/Source/JavaScriptCore/b3/B3HeapRange.h</a></li>
<li><a href="#trunkSourceJavaScriptCoreb3B3InsertionSetcpp">trunk/Source/JavaScriptCore/b3/B3InsertionSet.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoreb3B3InsertionSeth">trunk/Source/JavaScriptCore/b3/B3InsertionSet.h</a></li>
<li><a href="#trunkSourceJavaScriptCoreb3B3LegalizeMemoryOffsetscpp">trunk/Source/JavaScriptCore/b3/B3LegalizeMemoryOffsets.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoreb3B3LowerMacroscpp">trunk/Source/JavaScriptCore/b3/B3LowerMacros.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoreb3B3LowerMacrosAfterOptimizationscpp">trunk/Source/JavaScriptCore/b3/B3LowerMacrosAfterOptimizations.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoreb3B3LowerToAircpp">trunk/Source/JavaScriptCore/b3/B3LowerToAir.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoreb3B3LowerToAirh">trunk/Source/JavaScriptCore/b3/B3LowerToAir.h</a></li>
<li><a href="#trunkSourceJavaScriptCoreb3B3MemoryValuecpp">trunk/Source/JavaScriptCore/b3/B3MemoryValue.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoreb3B3MemoryValueh">trunk/Source/JavaScriptCore/b3/B3MemoryValue.h</a></li>
<li><a href="#trunkSourceJavaScriptCoreb3B3MoveConstantscpp">trunk/Source/JavaScriptCore/b3/B3MoveConstants.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoreb3B3Opcodecpp">trunk/Source/JavaScriptCore/b3/B3Opcode.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoreb3B3Opcodeh">trunk/Source/JavaScriptCore/b3/B3Opcode.h</a></li>
<li><a href="#trunkSourceJavaScriptCoreb3B3Procedurecpp">trunk/Source/JavaScriptCore/b3/B3Procedure.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoreb3B3Procedureh">trunk/Source/JavaScriptCore/b3/B3Procedure.h</a></li>
<li><a href="#trunkSourceJavaScriptCoreb3B3PureCSEcpp">trunk/Source/JavaScriptCore/b3/B3PureCSE.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoreb3B3PureCSEh">trunk/Source/JavaScriptCore/b3/B3PureCSE.h</a></li>
<li><a href="#trunkSourceJavaScriptCoreb3B3ReduceStrengthcpp">trunk/Source/JavaScriptCore/b3/B3ReduceStrength.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoreb3B3Validatecpp">trunk/Source/JavaScriptCore/b3/B3Validate.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoreb3B3Valuecpp">trunk/Source/JavaScriptCore/b3/B3Value.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoreb3B3Valueh">trunk/Source/JavaScriptCore/b3/B3Value.h</a></li>
<li><a href="#trunkSourceJavaScriptCoreb3B3ValueReph">trunk/Source/JavaScriptCore/b3/B3ValueRep.h</a></li>
<li><a href="#trunkSourceJavaScriptCoreb3B3Widthcpp">trunk/Source/JavaScriptCore/b3/B3Width.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoreb3B3Widthh">trunk/Source/JavaScriptCore/b3/B3Width.h</a></li>
<li><a href="#trunkSourceJavaScriptCoreb3airAirArgcpp">trunk/Source/JavaScriptCore/b3/air/AirArg.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoreb3airAirArgh">trunk/Source/JavaScriptCore/b3/air/AirArg.h</a></li>
<li><a href="#trunkSourceJavaScriptCoreb3airAirBasicBlockcpp">trunk/Source/JavaScriptCore/b3/air/AirBasicBlock.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoreb3airAirBasicBlockh">trunk/Source/JavaScriptCore/b3/air/AirBasicBlock.h</a></li>
<li><a href="#trunkSourceJavaScriptCoreb3airAirEliminateDeadCodecpp">trunk/Source/JavaScriptCore/b3/air/AirEliminateDeadCode.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoreb3airAirGeneratecpp">trunk/Source/JavaScriptCore/b3/air/AirGenerate.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoreb3airAirInstInlinesh">trunk/Source/JavaScriptCore/b3/air/AirInstInlines.h</a></li>
<li><a href="#trunkSourceJavaScriptCoreb3airAirOpcodeopcodes">trunk/Source/JavaScriptCore/b3/air/AirOpcode.opcodes</a></li>
<li><a href="#trunkSourceJavaScriptCoreb3airAirOptimizeBlockOrdercpp">trunk/Source/JavaScriptCore/b3/air/AirOptimizeBlockOrder.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoreb3airAirPadInterferencecpp">trunk/Source/JavaScriptCore/b3/air/AirPadInterference.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoreb3airAirSpillEverythingcpp">trunk/Source/JavaScriptCore/b3/air/AirSpillEverything.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoreb3airopcode_generatorrb">trunk/Source/JavaScriptCore/b3/air/opcode_generator.rb</a></li>
<li><a href="#trunkSourceJavaScriptCoreb3testb3cpp">trunk/Source/JavaScriptCore/b3/testb3.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoreruntimeOptionsh">trunk/Source/JavaScriptCore/runtime/Options.h</a></li>
<li><a href="#trunkWebsiteswebkitorgChangeLog">trunk/Websites/webkit.org/ChangeLog</a></li>
<li><a href="#trunkWebsiteswebkitorgdocsb3intermediaterepresentationhtml">trunk/Websites/webkit.org/docs/b3/intermediate-representation.html</a></li>
</ul>
<h3>Added Paths</h3>
<ul>
<li><a href="#trunkSourceJavaScriptCoreassemblerAbstractMacroAssemblercpp">trunk/Source/JavaScriptCore/assembler/AbstractMacroAssembler.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoreb3B3AtomicValuecpp">trunk/Source/JavaScriptCore/b3/B3AtomicValue.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoreb3B3AtomicValueh">trunk/Source/JavaScriptCore/b3/B3AtomicValue.h</a></li>
<li><a href="#trunkSourceJavaScriptCoreb3B3GenericBlockInsertionSeth">trunk/Source/JavaScriptCore/b3/B3GenericBlockInsertionSet.h</a></li>
<li><a href="#trunkSourceJavaScriptCoreb3B3MemoryValueInlinesh">trunk/Source/JavaScriptCore/b3/B3MemoryValueInlines.h</a></li>
<li><a href="#trunkSourceJavaScriptCoreb3B3NativeTraitsh">trunk/Source/JavaScriptCore/b3/B3NativeTraits.h</a></li>
<li><a href="#trunkSourceJavaScriptCoreb3airAirBlockInsertionSetcpp">trunk/Source/JavaScriptCore/b3/air/AirBlockInsertionSet.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoreb3airAirBlockInsertionSeth">trunk/Source/JavaScriptCore/b3/air/AirBlockInsertionSet.h</a></li>
</ul>
<h3>Removed Paths</h3>
<ul>
<li><a href="#trunkSourceJavaScriptCoreb3airAirDumpAsJScpp">trunk/Source/JavaScriptCore/b3/air/AirDumpAsJS.cpp</a></li>
<li><a href="#trunkSourceJavaScriptCoreb3airAirDumpAsJSh">trunk/Source/JavaScriptCore/b3/air/AirDumpAsJS.h</a></li>
</ul>
</div>
<div id="patch">
<h3>Diff</h3>
<a id="trunkSourceJavaScriptCoreCMakeListstxt"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/CMakeLists.txt (213713 => 213714)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/CMakeLists.txt 2017-03-10 17:32:07 UTC (rev 213713)
+++ trunk/Source/JavaScriptCore/CMakeLists.txt 2017-03-10 17:49:42 UTC (rev 213714)
</span><span class="lines">@@ -64,6 +64,7 @@
</span><span class="cx"> API/OpaqueJSString.cpp
</span><span class="cx">
</span><span class="cx"> assembler/ARMAssembler.cpp
</span><ins>+ assembler/AbstractMacroAssembler.cpp
</ins><span class="cx"> assembler/LinkBuffer.cpp
</span><span class="cx"> assembler/MacroAssembler.cpp
</span><span class="cx"> assembler/MacroAssemblerARM.cpp
</span><span class="lines">@@ -76,12 +77,12 @@
</span><span class="cx"> b3/air/AirAllocateStack.cpp
</span><span class="cx"> b3/air/AirArg.cpp
</span><span class="cx"> b3/air/AirBasicBlock.cpp
</span><ins>+ b3/air/AirBlockInsertionSet.cpp
</ins><span class="cx"> b3/air/AirCCallSpecial.cpp
</span><span class="cx"> b3/air/AirCCallingConvention.cpp
</span><span class="cx"> b3/air/AirCode.cpp
</span><span class="cx"> b3/air/AirCustom.cpp
</span><span class="cx"> b3/air/AirDisassembler.cpp
</span><del>- b3/air/AirDumpAsJS.cpp
</del><span class="cx"> b3/air/AirEliminateDeadCode.cpp
</span><span class="cx"> b3/air/AirEmitShuffle.cpp
</span><span class="cx"> b3/air/AirFixObviousSpills.cpp
</span><span class="lines">@@ -110,6 +111,7 @@
</span><span class="cx"> b3/air/AirValidate.cpp
</span><span class="cx">
</span><span class="cx"> b3/B3ArgumentRegValue.cpp
</span><ins>+ b3/B3AtomicValue.cpp
</ins><span class="cx"> b3/B3Bank.cpp
</span><span class="cx"> b3/B3BasicBlock.cpp
</span><span class="cx"> b3/B3BlockInsertionSet.cpp
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreChangeLog"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/ChangeLog (213713 => 213714)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/ChangeLog 2017-03-10 17:32:07 UTC (rev 213713)
+++ trunk/Source/JavaScriptCore/ChangeLog 2017-03-10 17:49:42 UTC (rev 213714)
</span><span class="lines">@@ -1,3 +1,510 @@
</span><ins>+2017-03-04 Filip Pizlo <fpizlo@apple.com>
+
+ B3 should have comprehensive support for atomic operations
+ https://bugs.webkit.org/show_bug.cgi?id=162349
+
+ Reviewed by Keith Miller.
+
+ This adds the following capabilities to B3:
+
+ - Atomic weak/strong unfenced/fenced compare-and-swap
+ - Atomic add/sub/or/and/xor/xchg
+ - Acquire/release fencing on loads/stores
+ - Fenceless load-load dependencies
+
+ This adds lowering to the following instructions on x86:
+
+ - lock cmpxchg
+ - lock xadd
+ - lock add/sub/or/and/xor/xchg
+
+ This adds lowering to the following instructions on ARM64:
+
+ - ldar and friends
+ - stlr and friends
+ - ldxr and friends (unfenced LL)
+ - stxr and friends (unfended SC)
+ - ldaxr and friends (fenced LL)
+ - stlxr and friends (fenced SC)
+ - eor as a fenceless load-load dependency
+
+ This does instruction selection pattern matching to ensure that weak/strong CAS and all of the
+ variants of fences and atomic math ops get lowered to the best possible instruction sequence.
+ For example, we support the Equal(AtomicStrongCAS(expected, ...), expected) pattern and a bunch
+ of its friends. You can say Branch(Equal(AtomicStrongCAS(expected, ...), expected)) and it will
+ generate the best possible branch sequence on x86 and ARM64.
+
+ B3 now knows how to model all of the kinds of fencing. It knows that acq loads are ordered with
+ respect to each other and with respect to rel stores, creating sequential consistency that
+ transcends just the acq/rel fences themselves (see Effects::fence). It knows that the phantom
+ fence effects may only target some abstract heaps but not others, so that load elimination and
+ store sinking can still operate across fences if you just tell B3 that the fence does not alias
+ those accesses. This makes it super easy to teach B3 that some of your heap is thread-local.
+ Even better, it lets you express fine-grained dependencies where the atomics that affect one
+ property in shared memory do not clobber non-atomics that ffect some other property in shared
+ memory.
+
+ One of my favorite features is Depend, which allows you to express load-load dependencies. On
+ x86 it lowers to nothing, while on ARM64 it lowers to eor.
+
+ This also exposes a common atomicWeakCAS API to the x86_64/ARM64 MacroAssemblers. Same for
+ acq/rel. JSC's 64-bit JITs are now a happy concurrency playground.
+
+ This doesn't yet expose the functionality to JS or wasm. SAB still uses the non-intrinsic
+ implementations of the Atomics object, for now.
+
+ * CMakeLists.txt:
+ * JavaScriptCore.xcodeproj/project.pbxproj:
+ * assembler/ARM64Assembler.h:
+ (JSC::ARM64Assembler::ldar):
+ (JSC::ARM64Assembler::ldxr):
+ (JSC::ARM64Assembler::ldaxr):
+ (JSC::ARM64Assembler::stxr):
+ (JSC::ARM64Assembler::stlr):
+ (JSC::ARM64Assembler::stlxr):
+ (JSC::ARM64Assembler::excepnGenerationImmMask):
+ (JSC::ARM64Assembler::exoticLoad):
+ (JSC::ARM64Assembler::storeRelease):
+ (JSC::ARM64Assembler::exoticStore):
+ * assembler/AbstractMacroAssembler.cpp: Added.
+ (WTF::printInternal):
+ * assembler/AbstractMacroAssembler.h:
+ (JSC::AbstractMacroAssemblerBase::invert):
+ * assembler/MacroAssembler.h:
+ * assembler/MacroAssemblerARM64.h:
+ (JSC::MacroAssemblerARM64::loadAcq8SignedExtendTo32):
+ (JSC::MacroAssemblerARM64::loadAcq8):
+ (JSC::MacroAssemblerARM64::storeRel8):
+ (JSC::MacroAssemblerARM64::loadAcq16SignedExtendTo32):
+ (JSC::MacroAssemblerARM64::loadAcq16):
+ (JSC::MacroAssemblerARM64::storeRel16):
+ (JSC::MacroAssemblerARM64::loadAcq32):
+ (JSC::MacroAssemblerARM64::loadAcq64):
+ (JSC::MacroAssemblerARM64::storeRel32):
+ (JSC::MacroAssemblerARM64::storeRel64):
+ (JSC::MacroAssemblerARM64::loadLink8):
+ (JSC::MacroAssemblerARM64::loadLinkAcq8):
+ (JSC::MacroAssemblerARM64::storeCond8):
+ (JSC::MacroAssemblerARM64::storeCondRel8):
+ (JSC::MacroAssemblerARM64::loadLink16):
+ (JSC::MacroAssemblerARM64::loadLinkAcq16):
+ (JSC::MacroAssemblerARM64::storeCond16):
+ (JSC::MacroAssemblerARM64::storeCondRel16):
+ (JSC::MacroAssemblerARM64::loadLink32):
+ (JSC::MacroAssemblerARM64::loadLinkAcq32):
+ (JSC::MacroAssemblerARM64::storeCond32):
+ (JSC::MacroAssemblerARM64::storeCondRel32):
+ (JSC::MacroAssemblerARM64::loadLink64):
+ (JSC::MacroAssemblerARM64::loadLinkAcq64):
+ (JSC::MacroAssemblerARM64::storeCond64):
+ (JSC::MacroAssemblerARM64::storeCondRel64):
+ (JSC::MacroAssemblerARM64::atomicStrongCAS8):
+ (JSC::MacroAssemblerARM64::atomicStrongCAS16):
+ (JSC::MacroAssemblerARM64::atomicStrongCAS32):
+ (JSC::MacroAssemblerARM64::atomicStrongCAS64):
+ (JSC::MacroAssemblerARM64::atomicRelaxedStrongCAS8):
+ (JSC::MacroAssemblerARM64::atomicRelaxedStrongCAS16):
+ (JSC::MacroAssemblerARM64::atomicRelaxedStrongCAS32):
+ (JSC::MacroAssemblerARM64::atomicRelaxedStrongCAS64):
+ (JSC::MacroAssemblerARM64::branchAtomicWeakCAS8):
+ (JSC::MacroAssemblerARM64::branchAtomicWeakCAS16):
+ (JSC::MacroAssemblerARM64::branchAtomicWeakCAS32):
+ (JSC::MacroAssemblerARM64::branchAtomicWeakCAS64):
+ (JSC::MacroAssemblerARM64::branchAtomicRelaxedWeakCAS8):
+ (JSC::MacroAssemblerARM64::branchAtomicRelaxedWeakCAS16):
+ (JSC::MacroAssemblerARM64::branchAtomicRelaxedWeakCAS32):
+ (JSC::MacroAssemblerARM64::branchAtomicRelaxedWeakCAS64):
+ (JSC::MacroAssemblerARM64::depend32):
+ (JSC::MacroAssemblerARM64::depend64):
+ (JSC::MacroAssemblerARM64::loadLink):
+ (JSC::MacroAssemblerARM64::loadLinkAcq):
+ (JSC::MacroAssemblerARM64::storeCond):
+ (JSC::MacroAssemblerARM64::storeCondRel):
+ (JSC::MacroAssemblerARM64::signExtend):
+ (JSC::MacroAssemblerARM64::branch):
+ (JSC::MacroAssemblerARM64::atomicStrongCAS):
+ (JSC::MacroAssemblerARM64::atomicRelaxedStrongCAS):
+ (JSC::MacroAssemblerARM64::branchAtomicWeakCAS):
+ (JSC::MacroAssemblerARM64::branchAtomicRelaxedWeakCAS):
+ (JSC::MacroAssemblerARM64::extractSimpleAddress):
+ (JSC::MacroAssemblerARM64::signExtend<8>):
+ (JSC::MacroAssemblerARM64::signExtend<16>):
+ (JSC::MacroAssemblerARM64::branch<64>):
+ * assembler/MacroAssemblerX86Common.h:
+ (JSC::MacroAssemblerX86Common::add32):
+ (JSC::MacroAssemblerX86Common::and32):
+ (JSC::MacroAssemblerX86Common::and16):
+ (JSC::MacroAssemblerX86Common::and8):
+ (JSC::MacroAssemblerX86Common::neg32):
+ (JSC::MacroAssemblerX86Common::neg16):
+ (JSC::MacroAssemblerX86Common::neg8):
+ (JSC::MacroAssemblerX86Common::or32):
+ (JSC::MacroAssemblerX86Common::or16):
+ (JSC::MacroAssemblerX86Common::or8):
+ (JSC::MacroAssemblerX86Common::sub16):
+ (JSC::MacroAssemblerX86Common::sub8):
+ (JSC::MacroAssemblerX86Common::sub32):
+ (JSC::MacroAssemblerX86Common::xor32):
+ (JSC::MacroAssemblerX86Common::xor16):
+ (JSC::MacroAssemblerX86Common::xor8):
+ (JSC::MacroAssemblerX86Common::not32):
+ (JSC::MacroAssemblerX86Common::not16):
+ (JSC::MacroAssemblerX86Common::not8):
+ (JSC::MacroAssemblerX86Common::store16):
+ (JSC::MacroAssemblerX86Common::atomicStrongCAS8):
+ (JSC::MacroAssemblerX86Common::atomicStrongCAS16):
+ (JSC::MacroAssemblerX86Common::atomicStrongCAS32):
+ (JSC::MacroAssemblerX86Common::branchAtomicStrongCAS8):
+ (JSC::MacroAssemblerX86Common::branchAtomicStrongCAS16):
+ (JSC::MacroAssemblerX86Common::branchAtomicStrongCAS32):
+ (JSC::MacroAssemblerX86Common::atomicWeakCAS8):
+ (JSC::MacroAssemblerX86Common::atomicWeakCAS16):
+ (JSC::MacroAssemblerX86Common::atomicWeakCAS32):
+ (JSC::MacroAssemblerX86Common::branchAtomicWeakCAS8):
+ (JSC::MacroAssemblerX86Common::branchAtomicWeakCAS16):
+ (JSC::MacroAssemblerX86Common::branchAtomicWeakCAS32):
+ (JSC::MacroAssemblerX86Common::atomicRelaxedWeakCAS8):
+ (JSC::MacroAssemblerX86Common::atomicRelaxedWeakCAS16):
+ (JSC::MacroAssemblerX86Common::atomicRelaxedWeakCAS32):
+ (JSC::MacroAssemblerX86Common::branchAtomicRelaxedWeakCAS8):
+ (JSC::MacroAssemblerX86Common::branchAtomicRelaxedWeakCAS16):
+ (JSC::MacroAssemblerX86Common::branchAtomicRelaxedWeakCAS32):
+ (JSC::MacroAssemblerX86Common::atomicAdd8):
+ (JSC::MacroAssemblerX86Common::atomicAdd16):
+ (JSC::MacroAssemblerX86Common::atomicAdd32):
+ (JSC::MacroAssemblerX86Common::atomicSub8):
+ (JSC::MacroAssemblerX86Common::atomicSub16):
+ (JSC::MacroAssemblerX86Common::atomicSub32):
+ (JSC::MacroAssemblerX86Common::atomicAnd8):
+ (JSC::MacroAssemblerX86Common::atomicAnd16):
+ (JSC::MacroAssemblerX86Common::atomicAnd32):
+ (JSC::MacroAssemblerX86Common::atomicOr8):
+ (JSC::MacroAssemblerX86Common::atomicOr16):
+ (JSC::MacroAssemblerX86Common::atomicOr32):
+ (JSC::MacroAssemblerX86Common::atomicXor8):
+ (JSC::MacroAssemblerX86Common::atomicXor16):
+ (JSC::MacroAssemblerX86Common::atomicXor32):
+ (JSC::MacroAssemblerX86Common::atomicNeg8):
+ (JSC::MacroAssemblerX86Common::atomicNeg16):
+ (JSC::MacroAssemblerX86Common::atomicNeg32):
+ (JSC::MacroAssemblerX86Common::atomicNot8):
+ (JSC::MacroAssemblerX86Common::atomicNot16):
+ (JSC::MacroAssemblerX86Common::atomicNot32):
+ (JSC::MacroAssemblerX86Common::atomicXchgAdd8):
+ (JSC::MacroAssemblerX86Common::atomicXchgAdd16):
+ (JSC::MacroAssemblerX86Common::atomicXchgAdd32):
+ (JSC::MacroAssemblerX86Common::atomicXchg8):
+ (JSC::MacroAssemblerX86Common::atomicXchg16):
+ (JSC::MacroAssemblerX86Common::atomicXchg32):
+ (JSC::MacroAssemblerX86Common::loadAcq8):
+ (JSC::MacroAssemblerX86Common::loadAcq8SignedExtendTo32):
+ (JSC::MacroAssemblerX86Common::loadAcq16):
+ (JSC::MacroAssemblerX86Common::loadAcq16SignedExtendTo32):
+ (JSC::MacroAssemblerX86Common::loadAcq32):
+ (JSC::MacroAssemblerX86Common::storeRel8):
+ (JSC::MacroAssemblerX86Common::storeRel16):
+ (JSC::MacroAssemblerX86Common::storeRel32):
+ (JSC::MacroAssemblerX86Common::storeFence):
+ (JSC::MacroAssemblerX86Common::loadFence):
+ (JSC::MacroAssemblerX86Common::replaceWithJump):
+ (JSC::MacroAssemblerX86Common::maxJumpReplacementSize):
+ (JSC::MacroAssemblerX86Common::patchableJumpSize):
+ (JSC::MacroAssemblerX86Common::supportsFloatingPointRounding):
+ (JSC::MacroAssemblerX86Common::supportsAVX):
+ (JSC::MacroAssemblerX86Common::updateEax1EcxFlags):
+ (JSC::MacroAssemblerX86Common::x86Condition):
+ (JSC::MacroAssemblerX86Common::atomicStrongCAS):
+ (JSC::MacroAssemblerX86Common::branchAtomicStrongCAS):
+ * assembler/MacroAssemblerX86_64.h:
+ (JSC::MacroAssemblerX86_64::add64):
+ (JSC::MacroAssemblerX86_64::and64):
+ (JSC::MacroAssemblerX86_64::neg64):
+ (JSC::MacroAssemblerX86_64::or64):
+ (JSC::MacroAssemblerX86_64::sub64):
+ (JSC::MacroAssemblerX86_64::xor64):
+ (JSC::MacroAssemblerX86_64::not64):
+ (JSC::MacroAssemblerX86_64::store64):
+ (JSC::MacroAssemblerX86_64::atomicStrongCAS64):
+ (JSC::MacroAssemblerX86_64::branchAtomicStrongCAS64):
+ (JSC::MacroAssemblerX86_64::atomicWeakCAS64):
+ (JSC::MacroAssemblerX86_64::branchAtomicWeakCAS64):
+ (JSC::MacroAssemblerX86_64::atomicRelaxedWeakCAS64):
+ (JSC::MacroAssemblerX86_64::branchAtomicRelaxedWeakCAS64):
+ (JSC::MacroAssemblerX86_64::atomicAdd64):
+ (JSC::MacroAssemblerX86_64::atomicSub64):
+ (JSC::MacroAssemblerX86_64::atomicAnd64):
+ (JSC::MacroAssemblerX86_64::atomicOr64):
+ (JSC::MacroAssemblerX86_64::atomicXor64):
+ (JSC::MacroAssemblerX86_64::atomicNeg64):
+ (JSC::MacroAssemblerX86_64::atomicNot64):
+ (JSC::MacroAssemblerX86_64::atomicXchgAdd64):
+ (JSC::MacroAssemblerX86_64::atomicXchg64):
+ (JSC::MacroAssemblerX86_64::loadAcq64):
+ (JSC::MacroAssemblerX86_64::storeRel64):
+ * assembler/X86Assembler.h:
+ (JSC::X86Assembler::addl_mr):
+ (JSC::X86Assembler::addq_mr):
+ (JSC::X86Assembler::addq_rm):
+ (JSC::X86Assembler::addq_im):
+ (JSC::X86Assembler::andl_mr):
+ (JSC::X86Assembler::andl_rm):
+ (JSC::X86Assembler::andw_rm):
+ (JSC::X86Assembler::andb_rm):
+ (JSC::X86Assembler::andl_im):
+ (JSC::X86Assembler::andw_im):
+ (JSC::X86Assembler::andb_im):
+ (JSC::X86Assembler::andq_mr):
+ (JSC::X86Assembler::andq_rm):
+ (JSC::X86Assembler::andq_im):
+ (JSC::X86Assembler::incq_m):
+ (JSC::X86Assembler::negq_m):
+ (JSC::X86Assembler::negl_m):
+ (JSC::X86Assembler::negw_m):
+ (JSC::X86Assembler::negb_m):
+ (JSC::X86Assembler::notl_m):
+ (JSC::X86Assembler::notw_m):
+ (JSC::X86Assembler::notb_m):
+ (JSC::X86Assembler::notq_m):
+ (JSC::X86Assembler::orl_mr):
+ (JSC::X86Assembler::orl_rm):
+ (JSC::X86Assembler::orw_rm):
+ (JSC::X86Assembler::orb_rm):
+ (JSC::X86Assembler::orl_im):
+ (JSC::X86Assembler::orw_im):
+ (JSC::X86Assembler::orb_im):
+ (JSC::X86Assembler::orq_mr):
+ (JSC::X86Assembler::orq_rm):
+ (JSC::X86Assembler::orq_im):
+ (JSC::X86Assembler::subl_mr):
+ (JSC::X86Assembler::subl_rm):
+ (JSC::X86Assembler::subw_rm):
+ (JSC::X86Assembler::subb_rm):
+ (JSC::X86Assembler::subl_im):
+ (JSC::X86Assembler::subw_im):
+ (JSC::X86Assembler::subb_im):
+ (JSC::X86Assembler::subq_mr):
+ (JSC::X86Assembler::subq_rm):
+ (JSC::X86Assembler::subq_im):
+ (JSC::X86Assembler::xorl_mr):
+ (JSC::X86Assembler::xorl_rm):
+ (JSC::X86Assembler::xorl_im):
+ (JSC::X86Assembler::xorw_rm):
+ (JSC::X86Assembler::xorw_im):
+ (JSC::X86Assembler::xorb_rm):
+ (JSC::X86Assembler::xorb_im):
+ (JSC::X86Assembler::xorq_im):
+ (JSC::X86Assembler::xorq_rm):
+ (JSC::X86Assembler::xorq_mr):
+ (JSC::X86Assembler::xchgb_rm):
+ (JSC::X86Assembler::xchgw_rm):
+ (JSC::X86Assembler::xchgl_rm):
+ (JSC::X86Assembler::xchgq_rm):
+ (JSC::X86Assembler::movw_im):
+ (JSC::X86Assembler::movq_i32m):
+ (JSC::X86Assembler::cmpxchgb_rm):
+ (JSC::X86Assembler::cmpxchgw_rm):
+ (JSC::X86Assembler::cmpxchgl_rm):
+ (JSC::X86Assembler::cmpxchgq_rm):
+ (JSC::X86Assembler::xaddb_rm):
+ (JSC::X86Assembler::xaddw_rm):
+ (JSC::X86Assembler::xaddl_rm):
+ (JSC::X86Assembler::xaddq_rm):
+ (JSC::X86Assembler::X86InstructionFormatter::SingleInstructionBufferWriter::memoryModRM):
+ * b3/B3AtomicValue.cpp: Added.
+ (JSC::B3::AtomicValue::~AtomicValue):
+ (JSC::B3::AtomicValue::dumpMeta):
+ (JSC::B3::AtomicValue::cloneImpl):
+ (JSC::B3::AtomicValue::AtomicValue):
+ * b3/B3AtomicValue.h: Added.
+ * b3/B3BasicBlock.h:
+ * b3/B3BlockInsertionSet.cpp:
+ (JSC::B3::BlockInsertionSet::BlockInsertionSet):
+ (JSC::B3::BlockInsertionSet::insert): Deleted.
+ (JSC::B3::BlockInsertionSet::insertBefore): Deleted.
+ (JSC::B3::BlockInsertionSet::insertAfter): Deleted.
+ (JSC::B3::BlockInsertionSet::execute): Deleted.
+ * b3/B3BlockInsertionSet.h:
+ * b3/B3Effects.cpp:
+ (JSC::B3::Effects::interferes):
+ (JSC::B3::Effects::operator==):
+ (JSC::B3::Effects::dump):
+ * b3/B3Effects.h:
+ (JSC::B3::Effects::forCall):
+ (JSC::B3::Effects::mustExecute):
+ * b3/B3EliminateCommonSubexpressions.cpp:
+ * b3/B3Generate.cpp:
+ (JSC::B3::generateToAir):
+ * b3/B3GenericBlockInsertionSet.h: Added.
+ (JSC::B3::GenericBlockInsertionSet::GenericBlockInsertionSet):
+ (JSC::B3::GenericBlockInsertionSet::insert):
+ (JSC::B3::GenericBlockInsertionSet::insertBefore):
+ (JSC::B3::GenericBlockInsertionSet::insertAfter):
+ (JSC::B3::GenericBlockInsertionSet::execute):
+ * b3/B3HeapRange.h:
+ (JSC::B3::HeapRange::operator|):
+ * b3/B3InsertionSet.cpp:
+ (JSC::B3::InsertionSet::insertClone):
+ * b3/B3InsertionSet.h:
+ * b3/B3LegalizeMemoryOffsets.cpp:
+ * b3/B3LowerMacros.cpp:
+ (JSC::B3::lowerMacros):
+ * b3/B3LowerMacrosAfterOptimizations.cpp:
+ * b3/B3LowerToAir.cpp:
+ (JSC::B3::Air::LowerToAir::LowerToAir):
+ (JSC::B3::Air::LowerToAir::run):
+ (JSC::B3::Air::LowerToAir::effectiveAddr):
+ (JSC::B3::Air::LowerToAir::addr):
+ (JSC::B3::Air::LowerToAir::loadPromiseAnyOpcode):
+ (JSC::B3::Air::LowerToAir::appendShift):
+ (JSC::B3::Air::LowerToAir::tryAppendStoreBinOp):
+ (JSC::B3::Air::LowerToAir::storeOpcode):
+ (JSC::B3::Air::LowerToAir::createStore):
+ (JSC::B3::Air::LowerToAir::finishAppendingInstructions):
+ (JSC::B3::Air::LowerToAir::newBlock):
+ (JSC::B3::Air::LowerToAir::splitBlock):
+ (JSC::B3::Air::LowerToAir::fillStackmap):
+ (JSC::B3::Air::LowerToAir::appendX86Div):
+ (JSC::B3::Air::LowerToAir::appendX86UDiv):
+ (JSC::B3::Air::LowerToAir::loadLinkOpcode):
+ (JSC::B3::Air::LowerToAir::storeCondOpcode):
+ (JSC::B3::Air::LowerToAir::appendCAS):
+ (JSC::B3::Air::LowerToAir::appendVoidAtomic):
+ (JSC::B3::Air::LowerToAir::appendGeneralAtomic):
+ (JSC::B3::Air::LowerToAir::lower):
+ (JSC::B3::Air::LowerToAir::lowerX86Div): Deleted.
+ (JSC::B3::Air::LowerToAir::lowerX86UDiv): Deleted.
+ * b3/B3LowerToAir.h:
+ * b3/B3MemoryValue.cpp:
+ (JSC::B3::MemoryValue::isLegalOffset):
+ (JSC::B3::MemoryValue::accessType):
+ (JSC::B3::MemoryValue::accessBank):
+ (JSC::B3::MemoryValue::accessByteSize):
+ (JSC::B3::MemoryValue::dumpMeta):
+ (JSC::B3::MemoryValue::MemoryValue):
+ (JSC::B3::MemoryValue::accessWidth): Deleted.
+ * b3/B3MemoryValue.h:
+ * b3/B3MemoryValueInlines.h: Added.
+ (JSC::B3::MemoryValue::isLegalOffset):
+ (JSC::B3::MemoryValue::requiresSimpleAddr):
+ (JSC::B3::MemoryValue::accessWidth):
+ * b3/B3MoveConstants.cpp:
+ * b3/B3NativeTraits.h: Added.
+ * b3/B3Opcode.cpp:
+ (JSC::B3::storeOpcode):
+ (WTF::printInternal):
+ * b3/B3Opcode.h:
+ (JSC::B3::isLoad):
+ (JSC::B3::isStore):
+ (JSC::B3::isLoadStore):
+ (JSC::B3::isAtomic):
+ (JSC::B3::isAtomicCAS):
+ (JSC::B3::isAtomicXchg):
+ (JSC::B3::isMemoryAccess):
+ (JSC::B3::signExtendOpcode):
+ * b3/B3Procedure.cpp:
+ (JSC::B3::Procedure::dump):
+ * b3/B3Procedure.h:
+ (JSC::B3::Procedure::hasQuirks):
+ (JSC::B3::Procedure::setHasQuirks):
+ * b3/B3PureCSE.cpp:
+ (JSC::B3::pureCSE):
+ * b3/B3PureCSE.h:
+ * b3/B3ReduceStrength.cpp:
+ * b3/B3Validate.cpp:
+ * b3/B3Value.cpp:
+ (JSC::B3::Value::returnsBool):
+ (JSC::B3::Value::effects):
+ (JSC::B3::Value::key):
+ (JSC::B3::Value::performSubstitution):
+ (JSC::B3::Value::typeFor):
+ * b3/B3Value.h:
+ * b3/B3Width.cpp:
+ (JSC::B3::bestType):
+ * b3/B3Width.h:
+ (JSC::B3::canonicalWidth):
+ (JSC::B3::isCanonicalWidth):
+ (JSC::B3::mask):
+ * b3/air/AirArg.cpp:
+ (JSC::B3::Air::Arg::jsHash):
+ (JSC::B3::Air::Arg::dump):
+ (WTF::printInternal):
+ * b3/air/AirArg.h:
+ (JSC::B3::Air::Arg::isAnyUse):
+ (JSC::B3::Air::Arg::isColdUse):
+ (JSC::B3::Air::Arg::cooled):
+ (JSC::B3::Air::Arg::isEarlyUse):
+ (JSC::B3::Air::Arg::isLateUse):
+ (JSC::B3::Air::Arg::isAnyDef):
+ (JSC::B3::Air::Arg::isEarlyDef):
+ (JSC::B3::Air::Arg::isLateDef):
+ (JSC::B3::Air::Arg::isZDef):
+ (JSC::B3::Air::Arg::simpleAddr):
+ (JSC::B3::Air::Arg::statusCond):
+ (JSC::B3::Air::Arg::isSimpleAddr):
+ (JSC::B3::Air::Arg::isMemory):
+ (JSC::B3::Air::Arg::isStatusCond):
+ (JSC::B3::Air::Arg::isCondition):
+ (JSC::B3::Air::Arg::ptr):
+ (JSC::B3::Air::Arg::base):
+ (JSC::B3::Air::Arg::isGP):
+ (JSC::B3::Air::Arg::isFP):
+ (JSC::B3::Air::Arg::isValidForm):
+ (JSC::B3::Air::Arg::forEachTmpFast):
+ (JSC::B3::Air::Arg::forEachTmp):
+ (JSC::B3::Air::Arg::asAddress):
+ (JSC::B3::Air::Arg::asStatusCondition):
+ (JSC::B3::Air::Arg::isInvertible):
+ (JSC::B3::Air::Arg::inverted):
+ * b3/air/AirBasicBlock.cpp:
+ (JSC::B3::Air::BasicBlock::setSuccessors):
+ * b3/air/AirBasicBlock.h:
+ * b3/air/AirBlockInsertionSet.cpp: Added.
+ (JSC::B3::Air::BlockInsertionSet::BlockInsertionSet):
+ (JSC::B3::Air::BlockInsertionSet::~BlockInsertionSet):
+ * b3/air/AirBlockInsertionSet.h: Added.
+ * b3/air/AirDumpAsJS.cpp: Removed.
+ * b3/air/AirDumpAsJS.h: Removed.
+ * b3/air/AirEliminateDeadCode.cpp:
+ (JSC::B3::Air::eliminateDeadCode):
+ * b3/air/AirGenerate.cpp:
+ (JSC::B3::Air::prepareForGeneration):
+ * b3/air/AirInstInlines.h:
+ (JSC::B3::Air::isAtomicStrongCASValid):
+ (JSC::B3::Air::isBranchAtomicStrongCASValid):
+ (JSC::B3::Air::isAtomicStrongCAS8Valid):
+ (JSC::B3::Air::isAtomicStrongCAS16Valid):
+ (JSC::B3::Air::isAtomicStrongCAS32Valid):
+ (JSC::B3::Air::isAtomicStrongCAS64Valid):
+ (JSC::B3::Air::isBranchAtomicStrongCAS8Valid):
+ (JSC::B3::Air::isBranchAtomicStrongCAS16Valid):
+ (JSC::B3::Air::isBranchAtomicStrongCAS32Valid):
+ (JSC::B3::Air::isBranchAtomicStrongCAS64Valid):
+ * b3/air/AirOpcode.opcodes:
+ * b3/air/AirOptimizeBlockOrder.cpp:
+ (JSC::B3::Air::optimizeBlockOrder):
+ * b3/air/AirPadInterference.cpp:
+ (JSC::B3::Air::padInterference):
+ * b3/air/AirSpillEverything.cpp:
+ (JSC::B3::Air::spillEverything):
+ * b3/air/opcode_generator.rb:
+ * b3/testb3.cpp:
+ (JSC::B3::testLoadAcq42):
+ (JSC::B3::testStoreRelAddLoadAcq32):
+ (JSC::B3::testStoreRelAddLoadAcq8):
+ (JSC::B3::testStoreRelAddFenceLoadAcq8):
+ (JSC::B3::testStoreRelAddLoadAcq16):
+ (JSC::B3::testStoreRelAddLoadAcq64):
+ (JSC::B3::testTrappingStoreElimination):
+ (JSC::B3::testX86LeaAddAdd):
+ (JSC::B3::testX86LeaAddShlLeftScale1):
+ (JSC::B3::testAtomicWeakCAS):
+ (JSC::B3::testAtomicStrongCAS):
+ (JSC::B3::testAtomicXchg):
+ (JSC::B3::testDepend32):
+ (JSC::B3::testDepend64):
+ (JSC::B3::run):
+ * runtime/Options.h:
+
</ins><span class="cx"> 2017-03-10 Csaba Osztrogonác <ossy@webkit.org>
</span><span class="cx">
</span><span class="cx"> Unreviewed typo fixes after r213652.
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreJavaScriptCorexcodeprojprojectpbxproj"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/JavaScriptCore.xcodeproj/project.pbxproj (213713 => 213714)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/JavaScriptCore.xcodeproj/project.pbxproj 2017-03-10 17:32:07 UTC (rev 213713)
+++ trunk/Source/JavaScriptCore/JavaScriptCore.xcodeproj/project.pbxproj 2017-03-10 17:49:42 UTC (rev 213714)
</span><span class="lines">@@ -274,6 +274,14 @@
</span><span class="cx"> 0F2C63B01E60AE4300C13839 /* B3Bank.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F2C63AC1E60AE3C00C13839 /* B3Bank.h */; };
</span><span class="cx"> 0F2C63B11E60AE4500C13839 /* B3Width.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 0F2C63AD1E60AE3C00C13839 /* B3Width.cpp */; };
</span><span class="cx"> 0F2C63B21E60AE4700C13839 /* B3Width.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F2C63AE1E60AE3D00C13839 /* B3Width.h */; };
</span><ins>+ 0F2C63B61E6343EA00C13839 /* B3AtomicValue.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 0F2C63B31E6343E800C13839 /* B3AtomicValue.cpp */; };
+ 0F2C63B71E6343ED00C13839 /* B3AtomicValue.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F2C63B41E6343E800C13839 /* B3AtomicValue.h */; };
+ 0F2C63B81E6343F700C13839 /* B3GenericBlockInsertionSet.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F2C63B51E6343E800C13839 /* B3GenericBlockInsertionSet.h */; };
+ 0F2C63BB1E63440A00C13839 /* AirBlockInsertionSet.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 0F2C63B91E63440800C13839 /* AirBlockInsertionSet.cpp */; };
+ 0F2C63BC1E63440C00C13839 /* AirBlockInsertionSet.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F2C63BA1E63440800C13839 /* AirBlockInsertionSet.h */; };
+ 0F2C63C01E660EA700C13839 /* AbstractMacroAssembler.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 0F2C63BF1E660EA500C13839 /* AbstractMacroAssembler.cpp */; };
+ 0F2C63C21E664A5C00C13839 /* B3NativeTraits.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F2C63C11E664A5A00C13839 /* B3NativeTraits.h */; };
+ 0F2C63C41E69EF9400C13839 /* B3MemoryValueInlines.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F2C63C31E69EF9200C13839 /* B3MemoryValueInlines.h */; };
</ins><span class="cx"> 0F2D4DDD19832D34007D4B19 /* DebuggerScope.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 0F2D4DDB19832D34007D4B19 /* DebuggerScope.cpp */; };
</span><span class="cx"> 0F2D4DDE19832D34007D4B19 /* DebuggerScope.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F2D4DDC19832D34007D4B19 /* DebuggerScope.h */; settings = {ATTRIBUTES = (Private, ); }; };
</span><span class="cx"> 0F2D4DE819832DAC007D4B19 /* ToThisStatus.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 0F2D4DE519832DAC007D4B19 /* ToThisStatus.cpp */; };
</span><span class="lines">@@ -2249,8 +2257,6 @@
</span><span class="cx"> DC2143081CA32E58000A8869 /* ICStats.cpp in Sources */ = {isa = PBXBuildFile; fileRef = DC2143051CA32E52000A8869 /* ICStats.cpp */; };
</span><span class="cx"> DC3D2B0A1D34316200BA918C /* HeapCell.h in Headers */ = {isa = PBXBuildFile; fileRef = DC3D2B091D34316100BA918C /* HeapCell.h */; settings = {ATTRIBUTES = (Private, ); }; };
</span><span class="cx"> DC3D2B0C1D34377000BA918C /* HeapCell.cpp in Sources */ = {isa = PBXBuildFile; fileRef = DC3D2B0B1D34376E00BA918C /* HeapCell.cpp */; };
</span><del>- DC454B8C1D00E822004C18AF /* AirDumpAsJS.cpp in Sources */ = {isa = PBXBuildFile; fileRef = DC454B8A1D00E81F004C18AF /* AirDumpAsJS.cpp */; };
- DC454B8D1D00E824004C18AF /* AirDumpAsJS.h in Headers */ = {isa = PBXBuildFile; fileRef = DC454B8B1D00E81F004C18AF /* AirDumpAsJS.h */; };
</del><span class="cx"> DC605B5D1CE26EA000593718 /* ProfilerEvent.cpp in Sources */ = {isa = PBXBuildFile; fileRef = DC605B591CE26E9800593718 /* ProfilerEvent.cpp */; };
</span><span class="cx"> DC605B5E1CE26EA200593718 /* ProfilerEvent.h in Headers */ = {isa = PBXBuildFile; fileRef = DC605B5A1CE26E9800593718 /* ProfilerEvent.h */; settings = {ATTRIBUTES = (Private, ); }; };
</span><span class="cx"> DC605B5F1CE26EA500593718 /* ProfilerUID.cpp in Sources */ = {isa = PBXBuildFile; fileRef = DC605B5B1CE26E9800593718 /* ProfilerUID.cpp */; };
</span><span class="lines">@@ -2775,6 +2781,14 @@
</span><span class="cx"> 0F2C63AC1E60AE3C00C13839 /* B3Bank.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = B3Bank.h; path = b3/B3Bank.h; sourceTree = "<group>"; };
</span><span class="cx"> 0F2C63AD1E60AE3C00C13839 /* B3Width.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; name = B3Width.cpp; path = b3/B3Width.cpp; sourceTree = "<group>"; };
</span><span class="cx"> 0F2C63AE1E60AE3D00C13839 /* B3Width.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = B3Width.h; path = b3/B3Width.h; sourceTree = "<group>"; };
</span><ins>+ 0F2C63B31E6343E800C13839 /* B3AtomicValue.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; name = B3AtomicValue.cpp; path = b3/B3AtomicValue.cpp; sourceTree = "<group>"; };
+ 0F2C63B41E6343E800C13839 /* B3AtomicValue.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = B3AtomicValue.h; path = b3/B3AtomicValue.h; sourceTree = "<group>"; };
+ 0F2C63B51E6343E800C13839 /* B3GenericBlockInsertionSet.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = B3GenericBlockInsertionSet.h; path = b3/B3GenericBlockInsertionSet.h; sourceTree = "<group>"; };
+ 0F2C63B91E63440800C13839 /* AirBlockInsertionSet.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; name = AirBlockInsertionSet.cpp; path = b3/air/AirBlockInsertionSet.cpp; sourceTree = "<group>"; };
+ 0F2C63BA1E63440800C13839 /* AirBlockInsertionSet.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = AirBlockInsertionSet.h; path = b3/air/AirBlockInsertionSet.h; sourceTree = "<group>"; };
+ 0F2C63BF1E660EA500C13839 /* AbstractMacroAssembler.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = AbstractMacroAssembler.cpp; sourceTree = "<group>"; };
+ 0F2C63C11E664A5A00C13839 /* B3NativeTraits.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = B3NativeTraits.h; path = b3/B3NativeTraits.h; sourceTree = "<group>"; };
+ 0F2C63C31E69EF9200C13839 /* B3MemoryValueInlines.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = B3MemoryValueInlines.h; path = b3/B3MemoryValueInlines.h; sourceTree = "<group>"; };
</ins><span class="cx"> 0F2D4DDB19832D34007D4B19 /* DebuggerScope.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = DebuggerScope.cpp; sourceTree = "<group>"; };
</span><span class="cx"> 0F2D4DDC19832D34007D4B19 /* DebuggerScope.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = DebuggerScope.h; sourceTree = "<group>"; };
</span><span class="cx"> 0F2D4DDF19832D91007D4B19 /* TypeProfilerLog.cpp */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.cpp.cpp; path = TypeProfilerLog.cpp; sourceTree = "<group>"; };
</span><span class="lines">@@ -4795,8 +4809,6 @@
</span><span class="cx"> DC2143061CA32E52000A8869 /* ICStats.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = ICStats.h; sourceTree = "<group>"; };
</span><span class="cx"> DC3D2B091D34316100BA918C /* HeapCell.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = HeapCell.h; sourceTree = "<group>"; };
</span><span class="cx"> DC3D2B0B1D34376E00BA918C /* HeapCell.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = HeapCell.cpp; sourceTree = "<group>"; };
</span><del>- DC454B8A1D00E81F004C18AF /* AirDumpAsJS.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; name = AirDumpAsJS.cpp; path = b3/air/AirDumpAsJS.cpp; sourceTree = "<group>"; };
- DC454B8B1D00E81F004C18AF /* AirDumpAsJS.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = AirDumpAsJS.h; path = b3/air/AirDumpAsJS.h; sourceTree = "<group>"; };
</del><span class="cx"> DC605B591CE26E9800593718 /* ProfilerEvent.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; name = ProfilerEvent.cpp; path = profiler/ProfilerEvent.cpp; sourceTree = "<group>"; };
</span><span class="cx"> DC605B5A1CE26E9800593718 /* ProfilerEvent.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = ProfilerEvent.h; path = profiler/ProfilerEvent.h; sourceTree = "<group>"; };
</span><span class="cx"> DC605B5B1CE26E9800593718 /* ProfilerUID.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; name = ProfilerUID.cpp; path = profiler/ProfilerUID.cpp; sourceTree = "<group>"; };
</span><span class="lines">@@ -5314,6 +5326,8 @@
</span><span class="cx"> 0FEC84B31BDACD880080FF74 /* air */,
</span><span class="cx"> 0FEC84B41BDACDAC0080FF74 /* B3ArgumentRegValue.cpp */,
</span><span class="cx"> 0FEC84B51BDACDAC0080FF74 /* B3ArgumentRegValue.h */,
</span><ins>+ 0F2C63B31E6343E800C13839 /* B3AtomicValue.cpp */,
+ 0F2C63B41E6343E800C13839 /* B3AtomicValue.h */,
</ins><span class="cx"> 0F2C63AB1E60AE3C00C13839 /* B3Bank.cpp */,
</span><span class="cx"> 0F2C63AC1E60AE3C00C13839 /* B3Bank.h */,
</span><span class="cx"> 0FEC84B61BDACDAC0080FF74 /* B3BasicBlock.cpp */,
</span><span class="lines">@@ -5376,6 +5390,7 @@
</span><span class="cx"> 0FEC84CD1BDACDAC0080FF74 /* B3FrequentedBlock.h */,
</span><span class="cx"> 0FEC84CE1BDACDAC0080FF74 /* B3Generate.cpp */,
</span><span class="cx"> 0FEC84CF1BDACDAC0080FF74 /* B3Generate.h */,
</span><ins>+ 0F2C63B51E6343E800C13839 /* B3GenericBlockInsertionSet.h */,
</ins><span class="cx"> 0FEC84D01BDACDAC0080FF74 /* B3GenericFrequentedBlock.h */,
</span><span class="cx"> 0FEC85BF1BE167A00080FF74 /* B3HeapRange.cpp */,
</span><span class="cx"> 0FEC85C01BE167A00080FF74 /* B3HeapRange.h */,
</span><span class="lines">@@ -5398,8 +5413,10 @@
</span><span class="cx"> 43AB26C51C1A52F700D82AE6 /* B3MathExtras.h */,
</span><span class="cx"> 0FEC84D51BDACDAC0080FF74 /* B3MemoryValue.cpp */,
</span><span class="cx"> 0FEC84D61BDACDAC0080FF74 /* B3MemoryValue.h */,
</span><ins>+ 0F2C63C31E69EF9200C13839 /* B3MemoryValueInlines.h */,
</ins><span class="cx"> 0F338E031BF0276C0013C88F /* B3MoveConstants.cpp */,
</span><span class="cx"> 0F338E041BF0276C0013C88F /* B3MoveConstants.h */,
</span><ins>+ 0F2C63C11E664A5A00C13839 /* B3NativeTraits.h */,
</ins><span class="cx"> 0F338E051BF0276C0013C88F /* B3OpaqueByproduct.h */,
</span><span class="cx"> 0F338E061BF0276C0013C88F /* B3OpaqueByproducts.cpp */,
</span><span class="cx"> 0F338E071BF0276C0013C88F /* B3OpaqueByproducts.h */,
</span><span class="lines">@@ -5491,6 +5508,8 @@
</span><span class="cx"> 0F64EAF21C4ECD0600621E9B /* AirArgInlines.h */,
</span><span class="cx"> 0FEC854C1BDACDC70080FF74 /* AirBasicBlock.cpp */,
</span><span class="cx"> 0FEC854D1BDACDC70080FF74 /* AirBasicBlock.h */,
</span><ins>+ 0F2C63B91E63440800C13839 /* AirBlockInsertionSet.cpp */,
+ 0F2C63BA1E63440800C13839 /* AirBlockInsertionSet.h */,
</ins><span class="cx"> 0FB3878B1BFBC44D00E3AB1E /* AirBlockWorklist.h */,
</span><span class="cx"> 0F6183201C45BF070072450B /* AirCCallingConvention.cpp */,
</span><span class="cx"> 0F6183211C45BF070072450B /* AirCCallingConvention.h */,
</span><span class="lines">@@ -5502,8 +5521,6 @@
</span><span class="cx"> 0F10F1A21C420BF0001C07D2 /* AirCustom.h */,
</span><span class="cx"> 79ABB17B1E5CCB570045B9A6 /* AirDisassembler.cpp */,
</span><span class="cx"> 79ABB17C1E5CCB570045B9A6 /* AirDisassembler.h */,
</span><del>- DC454B8A1D00E81F004C18AF /* AirDumpAsJS.cpp */,
- DC454B8B1D00E81F004C18AF /* AirDumpAsJS.h */,
</del><span class="cx"> 0F4570361BE44C910062A629 /* AirEliminateDeadCode.cpp */,
</span><span class="cx"> 0F4570371BE44C910062A629 /* AirEliminateDeadCode.h */,
</span><span class="cx"> 0F6183231C45BF070072450B /* AirEmitShuffle.cpp */,
</span><span class="lines">@@ -7355,6 +7372,7 @@
</span><span class="cx"> isa = PBXGroup;
</span><span class="cx"> children = (
</span><span class="cx"> 0F1FE51B1922A3BC006987C5 /* AbortReason.h */,
</span><ins>+ 0F2C63BF1E660EA500C13839 /* AbstractMacroAssembler.cpp */,
</ins><span class="cx"> 860161DF0F3A83C100F84710 /* AbstractMacroAssembler.h */,
</span><span class="cx"> 0F3730901C0CD70C00052BFA /* AllowMacroScratchRegisterUsage.h */,
</span><span class="cx"> 8640923B156EED3B00566CB2 /* ARM64Assembler.h */,
</span><span class="lines">@@ -7383,8 +7401,8 @@
</span><span class="cx"> 86C568DE11A213EE0007F7F0 /* MacroAssemblerMIPS.h */,
</span><span class="cx"> FE68C6351B90DDD90042BCB3 /* MacroAssemblerPrinter.cpp */,
</span><span class="cx"> FE68C6361B90DDD90042BCB3 /* MacroAssemblerPrinter.h */,
</span><ins>+ 860161E10F3A83C100F84710 /* MacroAssemblerX86_64.h */,
</ins><span class="cx"> 860161E00F3A83C100F84710 /* MacroAssemblerX86.h */,
</span><del>- 860161E10F3A83C100F84710 /* MacroAssemblerX86_64.h */,
</del><span class="cx"> A7A4AE0717973B26005612B1 /* MacroAssemblerX86Common.cpp */,
</span><span class="cx"> 860161E20F3A83C100F84710 /* MacroAssemblerX86Common.h */,
</span><span class="cx"> 65860177185A8F5E00030EEE /* MaxFrameExtentForSlowPathCall.h */,
</span><span class="lines">@@ -7978,7 +7996,6 @@
</span><span class="cx"> 658824AF1E5CFDB000FB7359 /* ConfigFile.h in Headers */,
</span><span class="cx"> 0FEC85761BDACDC70080FF74 /* AirCode.h in Headers */,
</span><span class="cx"> 0F10F1A31C420BF0001C07D2 /* AirCustom.h in Headers */,
</span><del>- DC454B8D1D00E824004C18AF /* AirDumpAsJS.h in Headers */,
</del><span class="cx"> 0F4570391BE44C910062A629 /* AirEliminateDeadCode.h in Headers */,
</span><span class="cx"> 0F61832D1C45BF070072450B /* AirEmitShuffle.h in Headers */,
</span><span class="cx"> 0F4DE1CF1C4C1B54004D6C11 /* AirFixObviousSpills.h in Headers */,
</span><span class="lines">@@ -7991,6 +8008,7 @@
</span><span class="cx"> 0FEC85801BDACDC70080FF74 /* AirInst.h in Headers */,
</span><span class="cx"> 0FEC85811BDACDC70080FF74 /* AirInstInlines.h in Headers */,
</span><span class="cx"> 0FDF67D71D9DC442001B9825 /* AirKind.h in Headers */,
</span><ins>+ 0F2C63BC1E63440C00C13839 /* AirBlockInsertionSet.h in Headers */,
</ins><span class="cx"> 2684D4381C00161C0081D663 /* AirLiveness.h in Headers */,
</span><span class="cx"> 0FE34C1A1C4B39AE0003A512 /* AirLogRegisterPressure.h in Headers */,
</span><span class="cx"> 0F61832F1C45BF070072450B /* AirLowerAfterRegAlloc.h in Headers */,
</span><span class="lines">@@ -8082,6 +8100,7 @@
</span><span class="cx"> 43422A631C158E6D00E2EB98 /* B3ConstFloatValue.h in Headers */,
</span><span class="cx"> 0FEC85B31BDED9570080FF74 /* B3ConstPtrValue.h in Headers */,
</span><span class="cx"> 0F338DF61BE93D550013C88F /* B3ConstrainedValue.h in Headers */,
</span><ins>+ 0F2C63B81E6343F700C13839 /* B3GenericBlockInsertionSet.h in Headers */,
</ins><span class="cx"> 0F338E0E1BF0276C0013C88F /* B3DataSection.h in Headers */,
</span><span class="cx"> 0F33FCFC1C1625BE00323F67 /* B3Dominators.h in Headers */,
</span><span class="cx"> 0F6B8AD91C4EDDA200969052 /* B3DuplicateTails.h in Headers */,
</span><span class="lines">@@ -8272,6 +8291,7 @@
</span><span class="cx"> A77A423E17A0BBFD00A8DB81 /* DFGAbstractHeap.h in Headers */,
</span><span class="cx"> A704D90317A0BAA8006BA554 /* DFGAbstractInterpreter.h in Headers */,
</span><span class="cx"> A704D90417A0BAA8006BA554 /* DFGAbstractInterpreterInlines.h in Headers */,
</span><ins>+ 0F2C63C21E664A5C00C13839 /* B3NativeTraits.h in Headers */,
</ins><span class="cx"> 0F620177143FCD3F0068B77C /* DFGAbstractValue.h in Headers */,
</span><span class="cx"> 0FD3E4021B618AAF00C80E1E /* DFGAdaptiveInferredPropertyValueWatchpoint.h in Headers */,
</span><span class="cx"> 0F18D3D01B55A6E0002C5C9F /* DFGAdaptiveStructureWatchpoint.h in Headers */,
</span><span class="lines">@@ -8404,6 +8424,7 @@
</span><span class="cx"> 0F392C8A1B46188400844728 /* DFGOSRExitFuzz.h in Headers */,
</span><span class="cx"> 0FEFC9AB1681A3B600567F53 /* DFGOSRExitJumpPlaceholder.h in Headers */,
</span><span class="cx"> 0F235BEE17178E7300690C7F /* DFGOSRExitPreparation.h in Headers */,
</span><ins>+ 0F2C63B71E6343ED00C13839 /* B3AtomicValue.h in Headers */,
</ins><span class="cx"> 0F6237981AE45CA700D402EA /* DFGPhantomInsertionPhase.h in Headers */,
</span><span class="cx"> 0FFFC95C14EF90AF00C72532 /* DFGPhase.h in Headers */,
</span><span class="cx"> 0F2B9CEB19D0BA7D00B1D1B5 /* DFGPhiChildren.h in Headers */,
</span><span class="lines">@@ -8821,6 +8842,7 @@
</span><span class="cx"> 79B819931DD25CF500DDC714 /* JSGlobalObjectInlines.h in Headers */,
</span><span class="cx"> A51007C1187CC3C600B38879 /* JSGlobalObjectInspectorController.h in Headers */,
</span><span class="cx"> A50E4B6418809DD50068A46D /* JSGlobalObjectRuntimeAgent.h in Headers */,
</span><ins>+ 0F2C63C41E69EF9400C13839 /* B3MemoryValueInlines.h in Headers */,
</ins><span class="cx"> A503FA2A188F105900110F14 /* JSGlobalObjectScriptDebugServer.h in Headers */,
</span><span class="cx"> A513E5C0185BFACC007E95AD /* JSInjectedScriptHost.h in Headers */,
</span><span class="cx"> A513E5C2185BFACC007E95AD /* JSInjectedScriptHostPrototype.h in Headers */,
</span><span class="lines">@@ -9831,6 +9853,7 @@
</span><span class="cx"> buildActionMask = 2147483647;
</span><span class="cx"> files = (
</span><span class="cx"> 0FFA549716B8835000B3A982 /* A64DOpcode.cpp in Sources */,
</span><ins>+ 0F2C63C01E660EA700C13839 /* AbstractMacroAssembler.cpp in Sources */,
</ins><span class="cx"> AD4937C31DDBE6140077C807 /* AbstractModuleRecord.cpp in Sources */,
</span><span class="cx"> 0F55F0F414D1063900AC7649 /* AbstractPC.cpp in Sources */,
</span><span class="cx"> 5370B4F51BF26202005C40FC /* AdaptiveInferredPropertyValueWatchpointBase.cpp in Sources */,
</span><span class="lines">@@ -9841,7 +9864,6 @@
</span><span class="cx"> 0FEC85731BDACDC70080FF74 /* AirCCallSpecial.cpp in Sources */,
</span><span class="cx"> 0FEC85751BDACDC70080FF74 /* AirCode.cpp in Sources */,
</span><span class="cx"> 0F61832B1C45BF070072450B /* AirCustom.cpp in Sources */,
</span><del>- DC454B8C1D00E822004C18AF /* AirDumpAsJS.cpp in Sources */,
</del><span class="cx"> 0F4570381BE44C910062A629 /* AirEliminateDeadCode.cpp in Sources */,
</span><span class="cx"> 0F61832C1C45BF070072450B /* AirEmitShuffle.cpp in Sources */,
</span><span class="cx"> 0F4DE1CE1C4C1B54004D6C11 /* AirFixObviousSpills.cpp in Sources */,
</span><span class="lines">@@ -10288,6 +10310,7 @@
</span><span class="cx"> 9E729407190F01A5001A91B5 /* InitializeThreading.cpp in Sources */,
</span><span class="cx"> A513E5B7185B8BD3007E95AD /* InjectedScript.cpp in Sources */,
</span><span class="cx"> A514B2C2185A684400F3C7CB /* InjectedScriptBase.cpp in Sources */,
</span><ins>+ 0F2C63B61E6343EA00C13839 /* B3AtomicValue.cpp in Sources */,
</ins><span class="cx"> A58E35911860DECF001F24FE /* InjectedScriptHost.cpp in Sources */,
</span><span class="cx"> A513E5CA185F9624007E95AD /* InjectedScriptManager.cpp in Sources */,
</span><span class="cx"> A5840E20187B7B8600843B10 /* InjectedScriptModule.cpp in Sources */,
</span><span class="lines">@@ -10306,6 +10329,7 @@
</span><span class="cx"> E35E035F1B7AB43E0073AD2A /* InspectorInstrumentationObject.cpp in Sources */,
</span><span class="cx"> A532438B18568335002ED692 /* InspectorProtocolObjects.cpp in Sources */,
</span><span class="cx"> A50E4B6118809DD50068A46D /* InspectorRuntimeAgent.cpp in Sources */,
</span><ins>+ 0F2C63BB1E63440A00C13839 /* AirBlockInsertionSet.cpp in Sources */,
</ins><span class="cx"> A55165D21BDF0B98003B75C1 /* InspectorScriptProfilerAgent.cpp in Sources */,
</span><span class="cx"> A593CF821840377100BFCE27 /* InspectorValues.cpp in Sources */,
</span><span class="cx"> 147F39CF107EC37600427A48 /* InternalFunction.cpp in Sources */,
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreassemblerARM64Assemblerh"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/assembler/ARM64Assembler.h (213713 => 213714)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/assembler/ARM64Assembler.h 2017-03-10 17:32:07 UTC (rev 213713)
+++ trunk/Source/JavaScriptCore/assembler/ARM64Assembler.h 2017-03-10 17:49:42 UTC (rev 213714)
</span><span class="lines">@@ -694,7 +694,22 @@
</span><span class="cx"> LdrLiteralOp_LDRSW = 2,
</span><span class="cx"> LdrLiteralOp_128BIT = 2
</span><span class="cx"> };
</span><ins>+
+ enum ExoticLoadFence {
+ ExoticLoadFence_None,
+ ExoticLoadFence_Acquire
+ };
+
+ enum ExoticLoadAtomic {
+ ExoticLoadAtomic_Link,
+ ExoticLoadAtomic_None
+ };
</ins><span class="cx">
</span><ins>+ enum ExoticStoreFence {
+ ExoticStoreFence_None,
+ ExoticStoreFence_Release,
+ };
+
</ins><span class="cx"> static unsigned memPairOffsetShift(bool V, MemPairOpSize size)
</span><span class="cx"> {
</span><span class="cx"> // return the log2 of the size in bytes, e.g. 64 bit size returns 3
</span><span class="lines">@@ -1531,8 +1546,50 @@
</span><span class="cx"> {
</span><span class="cx"> insn(0xd5033abf);
</span><span class="cx"> }
</span><ins>+
+ template<int datasize>
+ void ldar(RegisterID dst, RegisterID src)
+ {
+ CHECK_DATASIZE();
+ insn(exoticLoad(MEMOPSIZE, ExoticLoadFence_Acquire, ExoticLoadAtomic_None, dst, src));
+ }
</ins><span class="cx">
</span><span class="cx"> template<int datasize>
</span><ins>+ void ldxr(RegisterID dst, RegisterID src)
+ {
+ CHECK_DATASIZE();
+ insn(exoticLoad(MEMOPSIZE, ExoticLoadFence_None, ExoticLoadAtomic_Link, dst, src));
+ }
+
+ template<int datasize>
+ void ldaxr(RegisterID dst, RegisterID src)
+ {
+ CHECK_DATASIZE();
+ insn(exoticLoad(MEMOPSIZE, ExoticLoadFence_Acquire, ExoticLoadAtomic_Link, dst, src));
+ }
+
+ template<int datasize>
+ void stxr(RegisterID result, RegisterID src, RegisterID dst)
+ {
+ CHECK_DATASIZE();
+ insn(exoticStore(MEMOPSIZE, ExoticStoreFence_None, result, src, dst));
+ }
+
+ template<int datasize>
+ void stlr(RegisterID src, RegisterID dst)
+ {
+ CHECK_DATASIZE();
+ insn(storeRelease(MEMOPSIZE, src, dst));
+ }
+
+ template<int datasize>
+ void stlxr(RegisterID result, RegisterID src, RegisterID dst)
+ {
+ CHECK_DATASIZE();
+ insn(exoticStore(MEMOPSIZE, ExoticStoreFence_Release, result, src, dst));
+ }
+
+ template<int datasize>
</ins><span class="cx"> ALWAYS_INLINE void orn(RegisterID rd, RegisterID rn, RegisterID rm)
</span><span class="cx"> {
</span><span class="cx"> orn<datasize>(rd, rn, rm, LSL, 0);
</span><span class="lines">@@ -3608,7 +3665,22 @@
</span><span class="cx"> const int op4 = 0;
</span><span class="cx"> return (0xd6000000 | opc << 21 | op2 << 16 | op3 << 10 | xOrZr(rn) << 5 | op4);
</span><span class="cx"> }
</span><del>-
</del><ins>+
+ static int exoticLoad(MemOpSize size, ExoticLoadFence fence, ExoticLoadAtomic atomic, RegisterID dst, RegisterID src)
+ {
+ return 0x085f7c00 | size << 30 | fence << 15 | atomic << 23 | src << 5 | dst;
+ }
+
+ static int storeRelease(MemOpSize size, RegisterID src, RegisterID dst)
+ {
+ return 0x089ffc00 | size << 30 | dst << 5 | src;
+ }
+
+ static int exoticStore(MemOpSize size, ExoticStoreFence fence, RegisterID result, RegisterID src, RegisterID dst)
+ {
+ return 0x08007c00 | size << 30 | result << 16 | fence << 15 | dst << 5 | src;
+ }
+
</ins><span class="cx"> // Workaround for Cortex-A53 erratum (835769). Emit an extra nop if the
</span><span class="cx"> // last instruction in the buffer is a load, store or prefetch. Needed
</span><span class="cx"> // before 64-bit multiply-accumulate instructions.
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreassemblerAbstractMacroAssemblercpp"></a>
<div class="addfile"><h4>Added: trunk/Source/JavaScriptCore/assembler/AbstractMacroAssembler.cpp (0 => 213714)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/assembler/AbstractMacroAssembler.cpp (rev 0)
+++ trunk/Source/JavaScriptCore/assembler/AbstractMacroAssembler.cpp 2017-03-10 17:49:42 UTC (rev 213714)
</span><span class="lines">@@ -0,0 +1,52 @@
</span><ins>+/*
+ * Copyright (C) 2017 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+ * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+ * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+ * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include "config.h"
+#include "MacroAssembler.h" // Have to break with style because AbstractMacroAssembler.h is a shady header.
+
+#if ENABLE(ASSEMBLER)
+
+#include <wtf/PrintStream.h>
+
+using namespace JSC;
+
+namespace WTF {
+
+void printInternal(PrintStream& out, AbstractMacroAssemblerBase::StatusCondition condition)
+{
+ switch (condition) {
+ case AbstractMacroAssemblerBase::Success:
+ out.print("Success");
+ return;
+ case AbstractMacroAssemblerBase::Failure:
+ out.print("Failure");
+ return;
+ }
+ RELEASE_ASSERT_NOT_REACHED();
+}
+
+} // namespace WTF
+
+#endif // ENABLE(ASSEMBLER)
</ins></span></pre></div>
<a id="trunkSourceJavaScriptCoreassemblerAbstractMacroAssemblerh"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/assembler/AbstractMacroAssembler.h (213713 => 213714)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/assembler/AbstractMacroAssembler.h 2017-03-10 17:32:07 UTC (rev 213713)
+++ trunk/Source/JavaScriptCore/assembler/AbstractMacroAssembler.h 2017-03-10 17:49:42 UTC (rev 213714)
</span><span class="lines">@@ -1,5 +1,5 @@
</span><span class="cx"> /*
</span><del>- * Copyright (C) 2008, 2012, 2014-2016 Apple Inc. All rights reserved.
</del><ins>+ * Copyright (C) 2008-2017 Apple Inc. All rights reserved.
</ins><span class="cx"> *
</span><span class="cx"> * Redistribution and use in source and binary forms, with or without
</span><span class="cx"> * modification, are permitted provided that the following conditions
</span><span class="lines">@@ -36,6 +36,7 @@
</span><span class="cx"> #include <wtf/CryptographicallyRandomNumber.h>
</span><span class="cx"> #include <wtf/Noncopyable.h>
</span><span class="cx"> #include <wtf/SharedTask.h>
</span><ins>+#include <wtf/Vector.h>
</ins><span class="cx"> #include <wtf/WeakRandom.h>
</span><span class="cx">
</span><span class="cx"> namespace JSC {
</span><span class="lines">@@ -50,8 +51,28 @@
</span><span class="cx"> struct OSRExit;
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+class AbstractMacroAssemblerBase {
+public:
+ enum StatusCondition {
+ Success,
+ Failure
+ };
+
+ static StatusCondition invert(StatusCondition condition)
+ {
+ switch (condition) {
+ case Success:
+ return Failure;
+ case Failure:
+ return Success;
+ }
+ RELEASE_ASSERT_NOT_REACHED();
+ return Success;
+ }
+};
+
</ins><span class="cx"> template <class AssemblerType, class MacroAssemblerType>
</span><del>-class AbstractMacroAssembler {
</del><ins>+class AbstractMacroAssembler : public AbstractMacroAssemblerBase {
</ins><span class="cx"> public:
</span><span class="cx"> typedef AbstractMacroAssembler<AssemblerType, MacroAssemblerType> AbstractMacroAssemblerType;
</span><span class="cx"> typedef AssemblerType AssemblerType_T;
</span><span class="lines">@@ -1108,3 +1129,16 @@
</span><span class="cx"> #endif // ENABLE(ASSEMBLER)
</span><span class="cx">
</span><span class="cx"> } // namespace JSC
</span><ins>+
+#if ENABLE(ASSEMBLER)
+
+namespace WTF {
+
+class PrintStream;
+
+void printInternal(PrintStream& out, JSC::AbstractMacroAssemblerBase::StatusCondition);
+
+} // namespace WTF
+
+#endif // ENABLE(ASSEMBLER)
+
</ins></span></pre></div>
<a id="trunkSourceJavaScriptCoreassemblerMacroAssemblerh"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/assembler/MacroAssembler.h (213713 => 213714)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/assembler/MacroAssembler.h 2017-03-10 17:32:07 UTC (rev 213713)
+++ trunk/Source/JavaScriptCore/assembler/MacroAssembler.h 2017-03-10 17:49:42 UTC (rev 213714)
</span><span class="lines">@@ -1,5 +1,5 @@
</span><span class="cx"> /*
</span><del>- * Copyright (C) 2008, 2012-2015 Apple Inc. All rights reserved.
</del><ins>+ * Copyright (C) 2008-2017 Apple Inc. All rights reserved.
</ins><span class="cx"> *
</span><span class="cx"> * Redistribution and use in source and binary forms, with or without
</span><span class="cx"> * modification, are permitted provided that the following conditions
</span><span class="lines">@@ -136,6 +136,7 @@
</span><span class="cx"> static const double twoToThe32; // This is super useful for some double code.
</span><span class="cx">
</span><span class="cx"> // Utilities used by the DFG JIT.
</span><ins>+ using AbstractMacroAssemblerBase::invert;
</ins><span class="cx"> using MacroAssemblerBase::invert;
</span><span class="cx">
</span><span class="cx"> static DoubleCondition invert(DoubleCondition cond)
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreassemblerMacroAssemblerARM64h"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/assembler/MacroAssemblerARM64.h (213713 => 213714)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/assembler/MacroAssemblerARM64.h 2017-03-10 17:32:07 UTC (rev 213713)
+++ trunk/Source/JavaScriptCore/assembler/MacroAssemblerARM64.h 2017-03-10 17:49:42 UTC (rev 213714)
</span><span class="lines">@@ -3375,7 +3375,229 @@
</span><span class="cx"> {
</span><span class="cx"> m_assembler.dmbISH();
</span><span class="cx"> }
</span><del>-
</del><ins>+
+ void loadAcq8SignedExtendTo32(ImplicitAddress address, RegisterID dest)
+ {
+ m_assembler.ldar<8>(dest, extractSimpleAddress(address));
+ }
+
+ void loadAcq8(ImplicitAddress address, RegisterID dest)
+ {
+ loadAcq8SignedExtendTo32(address, dest);
+ and32(TrustedImm32(0xff), dest);
+ }
+
+ void storeRel8(RegisterID src, ImplicitAddress address)
+ {
+ m_assembler.stlr<8>(src, extractSimpleAddress(address));
+ }
+
+ void loadAcq16SignedExtendTo32(ImplicitAddress address, RegisterID dest)
+ {
+ m_assembler.ldar<16>(dest, extractSimpleAddress(address));
+ }
+
+ void loadAcq16(ImplicitAddress address, RegisterID dest)
+ {
+ loadAcq16SignedExtendTo32(address, dest);
+ and32(TrustedImm32(0xffff), dest);
+ }
+
+ void storeRel16(RegisterID src, ImplicitAddress address)
+ {
+ m_assembler.stlr<16>(src, extractSimpleAddress(address));
+ }
+
+ void loadAcq32(ImplicitAddress address, RegisterID dest)
+ {
+ m_assembler.ldar<32>(dest, extractSimpleAddress(address));
+ }
+
+ void loadAcq64(ImplicitAddress address, RegisterID dest)
+ {
+ m_assembler.ldar<64>(dest, extractSimpleAddress(address));
+ }
+
+ void storeRel32(RegisterID dest, ImplicitAddress address)
+ {
+ m_assembler.stlr<32>(dest, extractSimpleAddress(address));
+ }
+
+ void storeRel64(RegisterID dest, ImplicitAddress address)
+ {
+ m_assembler.stlr<64>(dest, extractSimpleAddress(address));
+ }
+
+ void loadLink8(ImplicitAddress address, RegisterID dest)
+ {
+ m_assembler.ldxr<8>(dest, extractSimpleAddress(address));
+ }
+
+ void loadLinkAcq8(ImplicitAddress address, RegisterID dest)
+ {
+ m_assembler.ldaxr<8>(dest, extractSimpleAddress(address));
+ }
+
+ void storeCond8(RegisterID src, ImplicitAddress address, RegisterID result)
+ {
+ m_assembler.stxr<8>(result, src, extractSimpleAddress(address));
+ }
+
+ void storeCondRel8(RegisterID src, ImplicitAddress address, RegisterID result)
+ {
+ m_assembler.stlxr<8>(result, src, extractSimpleAddress(address));
+ }
+
+ void loadLink16(ImplicitAddress address, RegisterID dest)
+ {
+ m_assembler.ldxr<16>(dest, extractSimpleAddress(address));
+ }
+
+ void loadLinkAcq16(ImplicitAddress address, RegisterID dest)
+ {
+ m_assembler.ldaxr<16>(dest, extractSimpleAddress(address));
+ }
+
+ void storeCond16(RegisterID src, ImplicitAddress address, RegisterID result)
+ {
+ m_assembler.stxr<16>(result, src, extractSimpleAddress(address));
+ }
+
+ void storeCondRel16(RegisterID src, ImplicitAddress address, RegisterID result)
+ {
+ m_assembler.stlxr<16>(result, src, extractSimpleAddress(address));
+ }
+
+ void loadLink32(ImplicitAddress address, RegisterID dest)
+ {
+ m_assembler.ldxr<32>(dest, extractSimpleAddress(address));
+ }
+
+ void loadLinkAcq32(ImplicitAddress address, RegisterID dest)
+ {
+ m_assembler.ldaxr<32>(dest, extractSimpleAddress(address));
+ }
+
+ void storeCond32(RegisterID src, ImplicitAddress address, RegisterID result)
+ {
+ m_assembler.stxr<32>(result, src, extractSimpleAddress(address));
+ }
+
+ void storeCondRel32(RegisterID src, ImplicitAddress address, RegisterID result)
+ {
+ m_assembler.stlxr<32>(result, src, extractSimpleAddress(address));
+ }
+
+ void loadLink64(ImplicitAddress address, RegisterID dest)
+ {
+ m_assembler.ldxr<64>(dest, extractSimpleAddress(address));
+ }
+
+ void loadLinkAcq64(ImplicitAddress address, RegisterID dest)
+ {
+ m_assembler.ldaxr<64>(dest, extractSimpleAddress(address));
+ }
+
+ void storeCond64(RegisterID src, ImplicitAddress address, RegisterID result)
+ {
+ m_assembler.stxr<64>(result, src, extractSimpleAddress(address));
+ }
+
+ void storeCondRel64(RegisterID src, ImplicitAddress address, RegisterID result)
+ {
+ m_assembler.stlxr<64>(result, src, extractSimpleAddress(address));
+ }
+
+ void atomicStrongCAS8(StatusCondition cond, RegisterID expectedAndResult, RegisterID newValue, Address address, RegisterID result)
+ {
+ atomicStrongCAS<8>(cond, expectedAndResult, newValue, address, result);
+ }
+
+ void atomicStrongCAS16(StatusCondition cond, RegisterID expectedAndResult, RegisterID newValue, Address address, RegisterID result)
+ {
+ atomicStrongCAS<16>(cond, expectedAndResult, newValue, address, result);
+ }
+
+ void atomicStrongCAS32(StatusCondition cond, RegisterID expectedAndResult, RegisterID newValue, Address address, RegisterID result)
+ {
+ atomicStrongCAS<32>(cond, expectedAndResult, newValue, address, result);
+ }
+
+ void atomicStrongCAS64(StatusCondition cond, RegisterID expectedAndResult, RegisterID newValue, Address address, RegisterID result)
+ {
+ atomicStrongCAS<64>(cond, expectedAndResult, newValue, address, result);
+ }
+
+ void atomicRelaxedStrongCAS8(StatusCondition cond, RegisterID expectedAndResult, RegisterID newValue, Address address, RegisterID result)
+ {
+ atomicRelaxedStrongCAS<8>(cond, expectedAndResult, newValue, address, result);
+ }
+
+ void atomicRelaxedStrongCAS16(StatusCondition cond, RegisterID expectedAndResult, RegisterID newValue, Address address, RegisterID result)
+ {
+ atomicRelaxedStrongCAS<16>(cond, expectedAndResult, newValue, address, result);
+ }
+
+ void atomicRelaxedStrongCAS32(StatusCondition cond, RegisterID expectedAndResult, RegisterID newValue, Address address, RegisterID result)
+ {
+ atomicRelaxedStrongCAS<32>(cond, expectedAndResult, newValue, address, result);
+ }
+
+ void atomicRelaxedStrongCAS64(StatusCondition cond, RegisterID expectedAndResult, RegisterID newValue, Address address, RegisterID result)
+ {
+ atomicRelaxedStrongCAS<64>(cond, expectedAndResult, newValue, address, result);
+ }
+
+ JumpList branchAtomicWeakCAS8(StatusCondition cond, RegisterID expectedAndClobbered, RegisterID newValue, Address address)
+ {
+ return branchAtomicWeakCAS<8>(cond, expectedAndClobbered, newValue, address);
+ }
+
+ JumpList branchAtomicWeakCAS16(StatusCondition cond, RegisterID expectedAndClobbered, RegisterID newValue, Address address)
+ {
+ return branchAtomicWeakCAS<16>(cond, expectedAndClobbered, newValue, address);
+ }
+
+ JumpList branchAtomicWeakCAS32(StatusCondition cond, RegisterID expectedAndClobbered, RegisterID newValue, Address address)
+ {
+ return branchAtomicWeakCAS<32>(cond, expectedAndClobbered, newValue, address);
+ }
+
+ JumpList branchAtomicWeakCAS64(StatusCondition cond, RegisterID expectedAndClobbered, RegisterID newValue, Address address)
+ {
+ return branchAtomicWeakCAS<64>(cond, expectedAndClobbered, newValue, address);
+ }
+
+ JumpList branchAtomicRelaxedWeakCAS8(StatusCondition cond, RegisterID expectedAndClobbered, RegisterID newValue, Address address)
+ {
+ return branchAtomicRelaxedWeakCAS<8>(cond, expectedAndClobbered, newValue, address);
+ }
+
+ JumpList branchAtomicRelaxedWeakCAS16(StatusCondition cond, RegisterID expectedAndClobbered, RegisterID newValue, Address address)
+ {
+ return branchAtomicRelaxedWeakCAS<16>(cond, expectedAndClobbered, newValue, address);
+ }
+
+ JumpList branchAtomicRelaxedWeakCAS32(StatusCondition cond, RegisterID expectedAndClobbered, RegisterID newValue, Address address)
+ {
+ return branchAtomicRelaxedWeakCAS<32>(cond, expectedAndClobbered, newValue, address);
+ }
+
+ JumpList branchAtomicRelaxedWeakCAS64(StatusCondition cond, RegisterID expectedAndClobbered, RegisterID newValue, Address address)
+ {
+ return branchAtomicRelaxedWeakCAS<64>(cond, expectedAndClobbered, newValue, address);
+ }
+
+ void depend32(RegisterID src, RegisterID dest)
+ {
+ m_assembler.eor<32>(dest, src, src);
+ }
+
+ void depend64(RegisterID src, RegisterID dest)
+ {
+ m_assembler.eor<64>(dest, src, src);
+ }
+
</ins><span class="cx"> // Misc helper functions.
</span><span class="cx">
</span><span class="cx"> // Invert a relational condition, e.g. == becomes !=, < becomes >=, etc.
</span><span class="lines">@@ -3882,7 +4104,159 @@
</span><span class="cx"> }
</span><span class="cx"> return false;
</span><span class="cx"> }
</span><ins>+
+ template<int datasize>
+ void loadLink(RegisterID src, RegisterID dest)
+ {
+ m_assembler.ldxr<datasize>(dest, src);
+ }
+
+ template<int datasize>
+ void loadLinkAcq(RegisterID src, RegisterID dest)
+ {
+ m_assembler.ldaxr<datasize>(dest, src);
+ }
+
+ template<int datasize>
+ void storeCond(RegisterID src, RegisterID dest, RegisterID result)
+ {
+ m_assembler.stxr<datasize>(src, dest, result);
+ }
+
+ template<int datasize>
+ void storeCondRel(RegisterID src, RegisterID dest, RegisterID result)
+ {
+ m_assembler.stlxr<datasize>(src, dest, result);
+ }
+
+ template<int datasize>
+ void signExtend(RegisterID src, RegisterID dest)
+ {
+ move(src, dest);
+ }
+
+ template<int datasize>
+ Jump branch(RelationalCondition cond, RegisterID left, RegisterID right)
+ {
+ return branch32(cond, left, right);
+ }
+
+ template<int datasize>
+ void atomicStrongCAS(StatusCondition cond, RegisterID expectedAndResult, RegisterID newValue, Address address, RegisterID result)
+ {
+ signExtend<datasize>(expectedAndResult, expectedAndResult);
+
+ RegisterID simpleAddress = extractSimpleAddress(address);
+ RegisterID tmp = getCachedDataTempRegisterIDAndInvalidate();
+
+ Label reloop = label();
+ loadLinkAcq<datasize>(simpleAddress, tmp);
+ Jump failure = branch<datasize>(NotEqual, expectedAndResult, tmp);
+
+ storeCondRel<datasize>(newValue, simpleAddress, result);
+ branchTest32(NonZero, result).linkTo(reloop, this);
+ move(TrustedImm32(cond == Success), result);
+ Jump done = jump();
+
+ failure.link(this);
+ move(tmp, expectedAndResult);
+ storeCondRel<datasize>(tmp, simpleAddress, result);
+ branchTest32(NonZero, result).linkTo(reloop, this);
+ move(TrustedImm32(cond == Failure), result);
+
+ done.link(this);
+ }
+
+ template<int datasize>
+ void atomicRelaxedStrongCAS(StatusCondition cond, RegisterID expectedAndResult, RegisterID newValue, Address address, RegisterID result)
+ {
+ signExtend<datasize>(expectedAndResult, expectedAndResult);
+
+ RegisterID simpleAddress = extractSimpleAddress(address);
+ RegisterID tmp = getCachedDataTempRegisterIDAndInvalidate();
+
+ Label reloop = label();
+ loadLink<datasize>(simpleAddress, tmp);
+ Jump failure = branch<datasize>(NotEqual, expectedAndResult, tmp);
+
+ storeCond<datasize>(newValue, simpleAddress, result);
+ branchTest32(NonZero, result).linkTo(reloop, this);
+ move(TrustedImm32(cond == Success), result);
+ Jump done = jump();
+
+ failure.link(this);
+ move(tmp, expectedAndResult);
+ move(TrustedImm32(cond == Failure), result);
+
+ done.link(this);
+ }
+
+ template<int datasize>
+ JumpList branchAtomicWeakCAS(StatusCondition cond, RegisterID expectedAndClobbered, RegisterID newValue, Address address)
+ {
+ signExtend<datasize>(expectedAndClobbered, expectedAndClobbered);
+
+ RegisterID simpleAddress = extractSimpleAddress(address);
+ RegisterID tmp = getCachedDataTempRegisterIDAndInvalidate();
+
+ JumpList success;
+ JumpList failure;
</ins><span class="cx">
</span><ins>+ loadLinkAcq<datasize>(simpleAddress, tmp);
+ failure.append(branch<datasize>(NotEqual, expectedAndClobbered, tmp));
+ storeCondRel<datasize>(newValue, simpleAddress, expectedAndClobbered);
+
+ switch (cond) {
+ case Success:
+ success.append(branchTest32(Zero, expectedAndClobbered));
+ failure.link(this);
+ return success;
+ case Failure:
+ failure.append(branchTest32(NonZero, expectedAndClobbered));
+ return failure;
+ }
+
+ RELEASE_ASSERT_NOT_REACHED();
+ }
+
+ template<int datasize>
+ JumpList branchAtomicRelaxedWeakCAS(StatusCondition cond, RegisterID expectedAndClobbered, RegisterID newValue, Address address)
+ {
+ signExtend<datasize>(expectedAndClobbered, expectedAndClobbered);
+
+ RegisterID simpleAddress = extractSimpleAddress(address);
+ RegisterID tmp = getCachedDataTempRegisterIDAndInvalidate();
+
+ JumpList success;
+ JumpList failure;
+
+ loadLink<datasize>(simpleAddress, tmp);
+ failure.append(branch<datasize>(NotEqual, expectedAndClobbered, tmp));
+ storeCond<datasize>(newValue, simpleAddress, expectedAndClobbered);
+
+ switch (cond) {
+ case Success:
+ success.append(branchTest32(Zero, expectedAndClobbered));
+ failure.link(this);
+ return success;
+ case Failure:
+ failure.append(branchTest32(NonZero, expectedAndClobbered));
+ return failure;
+ }
+
+ RELEASE_ASSERT_NOT_REACHED();
+ }
+
+ RegisterID extractSimpleAddress(ImplicitAddress address)
+ {
+ if (!address.offset)
+ return address.base;
+
+ signExtend32ToPtr(TrustedImm32(address.offset), getCachedMemoryTempRegisterIDAndInvalidate());
+ add64(address.base, memoryTempRegister);
+ return memoryTempRegister;
+ }
+
</ins><span class="cx"> Jump jumpAfterFloatingPointCompare(DoubleCondition cond)
</span><span class="cx"> {
</span><span class="cx"> if (cond == DoubleNotEqual) {
</span><span class="lines">@@ -3994,6 +4368,24 @@
</span><span class="cx"> m_assembler.sturh(rt, rn, simm);
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+template<>
+inline void MacroAssemblerARM64::signExtend<8>(RegisterID src, RegisterID dest)
+{
+ signExtend8To32(src, dest);
+}
+
+template<>
+inline void MacroAssemblerARM64::signExtend<16>(RegisterID src, RegisterID dest)
+{
+ signExtend16To32(src, dest);
+}
+
+template<>
+inline MacroAssemblerARM64::Jump MacroAssemblerARM64::branch<64>(RelationalCondition cond, RegisterID left, RegisterID right)
+{
+ return branch64(cond, left, right);
+}
+
</ins><span class="cx"> } // namespace JSC
</span><span class="cx">
</span><span class="cx"> #endif // ENABLE(ASSEMBLER)
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreassemblerMacroAssemblerX86Commonh"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/assembler/MacroAssemblerX86Common.h (213713 => 213714)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/assembler/MacroAssemblerX86Common.h 2017-03-10 17:32:07 UTC (rev 213713)
+++ trunk/Source/JavaScriptCore/assembler/MacroAssemblerX86Common.h 2017-03-10 17:49:42 UTC (rev 213714)
</span><span class="lines">@@ -171,6 +171,11 @@
</span><span class="cx"> m_assembler.addl_mr(src.offset, src.base, dest);
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+ void add32(BaseIndex src, RegisterID dest)
+ {
+ m_assembler.addl_mr(src.offset, src.base, src.index, src.scale, dest);
+ }
+
</ins><span class="cx"> void add32(RegisterID src, Address dest)
</span><span class="cx"> {
</span><span class="cx"> m_assembler.addl_rm(src, dest.offset, dest.base);
</span><span class="lines">@@ -251,16 +256,71 @@
</span><span class="cx"> m_assembler.andl_rm(src, dest.offset, dest.base);
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+ void and32(RegisterID src, BaseIndex dest)
+ {
+ m_assembler.andl_rm(src, dest.offset, dest.base, dest.index, dest.scale);
+ }
+
+ void and16(RegisterID src, Address dest)
+ {
+ m_assembler.andw_rm(src, dest.offset, dest.base);
+ }
+
+ void and16(RegisterID src, BaseIndex dest)
+ {
+ m_assembler.andw_rm(src, dest.offset, dest.base, dest.index, dest.scale);
+ }
+
+ void and8(RegisterID src, Address dest)
+ {
+ m_assembler.andb_rm(src, dest.offset, dest.base);
+ }
+
+ void and8(RegisterID src, BaseIndex dest)
+ {
+ m_assembler.andb_rm(src, dest.offset, dest.base, dest.index, dest.scale);
+ }
+
</ins><span class="cx"> void and32(Address src, RegisterID dest)
</span><span class="cx"> {
</span><span class="cx"> m_assembler.andl_mr(src.offset, src.base, dest);
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+ void and32(BaseIndex src, RegisterID dest)
+ {
+ m_assembler.andl_mr(src.offset, src.base, src.index, src.scale, dest);
+ }
+
</ins><span class="cx"> void and32(TrustedImm32 imm, Address address)
</span><span class="cx"> {
</span><span class="cx"> m_assembler.andl_im(imm.m_value, address.offset, address.base);
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+ void and32(TrustedImm32 imm, BaseIndex address)
+ {
+ m_assembler.andl_im(imm.m_value, address.offset, address.base, address.index, address.scale);
+ }
+
+ void and16(TrustedImm32 imm, Address address)
+ {
+ m_assembler.andw_im(static_cast<int16_t>(imm.m_value), address.offset, address.base);
+ }
+
+ void and16(TrustedImm32 imm, BaseIndex address)
+ {
+ m_assembler.andw_im(static_cast<int16_t>(imm.m_value), address.offset, address.base, address.index, address.scale);
+ }
+
+ void and8(TrustedImm32 imm, Address address)
+ {
+ m_assembler.andb_im(static_cast<int8_t>(imm.m_value), address.offset, address.base);
+ }
+
+ void and8(TrustedImm32 imm, BaseIndex address)
+ {
+ m_assembler.andb_im(static_cast<int8_t>(imm.m_value), address.offset, address.base, address.index, address.scale);
+ }
+
</ins><span class="cx"> void and32(RegisterID op1, RegisterID op2, RegisterID dest)
</span><span class="cx"> {
</span><span class="cx"> if (op1 == op2)
</span><span class="lines">@@ -456,6 +516,31 @@
</span><span class="cx"> m_assembler.negl_m(srcDest.offset, srcDest.base);
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+ void neg32(BaseIndex srcDest)
+ {
+ m_assembler.negl_m(srcDest.offset, srcDest.base, srcDest.index, srcDest.scale);
+ }
+
+ void neg16(Address srcDest)
+ {
+ m_assembler.negw_m(srcDest.offset, srcDest.base);
+ }
+
+ void neg16(BaseIndex srcDest)
+ {
+ m_assembler.negw_m(srcDest.offset, srcDest.base, srcDest.index, srcDest.scale);
+ }
+
+ void neg8(Address srcDest)
+ {
+ m_assembler.negb_m(srcDest.offset, srcDest.base);
+ }
+
+ void neg8(BaseIndex srcDest)
+ {
+ m_assembler.negb_m(srcDest.offset, srcDest.base, srcDest.index, srcDest.scale);
+ }
+
</ins><span class="cx"> void or32(RegisterID src, RegisterID dest)
</span><span class="cx"> {
</span><span class="cx"> m_assembler.orl_rr(src, dest);
</span><span class="lines">@@ -471,16 +556,71 @@
</span><span class="cx"> m_assembler.orl_rm(src, dest.offset, dest.base);
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+ void or32(RegisterID src, BaseIndex dest)
+ {
+ m_assembler.orl_rm(src, dest.offset, dest.base, dest.index, dest.scale);
+ }
+
+ void or16(RegisterID src, Address dest)
+ {
+ m_assembler.orw_rm(src, dest.offset, dest.base);
+ }
+
+ void or16(RegisterID src, BaseIndex dest)
+ {
+ m_assembler.orw_rm(src, dest.offset, dest.base, dest.index, dest.scale);
+ }
+
+ void or8(RegisterID src, Address dest)
+ {
+ m_assembler.orb_rm(src, dest.offset, dest.base);
+ }
+
+ void or8(RegisterID src, BaseIndex dest)
+ {
+ m_assembler.orb_rm(src, dest.offset, dest.base, dest.index, dest.scale);
+ }
+
</ins><span class="cx"> void or32(Address src, RegisterID dest)
</span><span class="cx"> {
</span><span class="cx"> m_assembler.orl_mr(src.offset, src.base, dest);
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+ void or32(BaseIndex src, RegisterID dest)
+ {
+ m_assembler.orl_mr(src.offset, src.base, src.index, src.scale, dest);
+ }
+
</ins><span class="cx"> void or32(TrustedImm32 imm, Address address)
</span><span class="cx"> {
</span><span class="cx"> m_assembler.orl_im(imm.m_value, address.offset, address.base);
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+ void or32(TrustedImm32 imm, BaseIndex address)
+ {
+ m_assembler.orl_im(imm.m_value, address.offset, address.base, address.index, address.scale);
+ }
+
+ void or16(TrustedImm32 imm, Address address)
+ {
+ m_assembler.orw_im(static_cast<int16_t>(imm.m_value), address.offset, address.base);
+ }
+
+ void or16(TrustedImm32 imm, BaseIndex address)
+ {
+ m_assembler.orw_im(static_cast<int16_t>(imm.m_value), address.offset, address.base, address.index, address.scale);
+ }
+
+ void or8(TrustedImm32 imm, Address address)
+ {
+ m_assembler.orb_im(static_cast<int8_t>(imm.m_value), address.offset, address.base);
+ }
+
+ void or8(TrustedImm32 imm, BaseIndex address)
+ {
+ m_assembler.orb_im(static_cast<int8_t>(imm.m_value), address.offset, address.base, address.index, address.scale);
+ }
+
</ins><span class="cx"> void or32(RegisterID op1, RegisterID op2, RegisterID dest)
</span><span class="cx"> {
</span><span class="cx"> if (op1 == op2)
</span><span class="lines">@@ -654,16 +794,71 @@
</span><span class="cx"> m_assembler.subl_im(imm.m_value, address.offset, address.base);
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+ void sub16(TrustedImm32 imm, Address address)
+ {
+ m_assembler.subw_im(static_cast<int16_t>(imm.m_value), address.offset, address.base);
+ }
+
+ void sub8(TrustedImm32 imm, Address address)
+ {
+ m_assembler.subb_im(static_cast<int8_t>(imm.m_value), address.offset, address.base);
+ }
+
+ void sub32(TrustedImm32 imm, BaseIndex address)
+ {
+ m_assembler.subl_im(imm.m_value, address.offset, address.base, address.index, address.scale);
+ }
+
+ void sub16(TrustedImm32 imm, BaseIndex address)
+ {
+ m_assembler.subw_im(static_cast<int16_t>(imm.m_value), address.offset, address.base, address.index, address.scale);
+ }
+
+ void sub8(TrustedImm32 imm, BaseIndex address)
+ {
+ m_assembler.subb_im(static_cast<int8_t>(imm.m_value), address.offset, address.base, address.index, address.scale);
+ }
+
</ins><span class="cx"> void sub32(Address src, RegisterID dest)
</span><span class="cx"> {
</span><span class="cx"> m_assembler.subl_mr(src.offset, src.base, dest);
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+ void sub32(BaseIndex src, RegisterID dest)
+ {
+ m_assembler.subl_mr(src.offset, src.base, src.index, src.scale, dest);
+ }
+
</ins><span class="cx"> void sub32(RegisterID src, Address dest)
</span><span class="cx"> {
</span><span class="cx"> m_assembler.subl_rm(src, dest.offset, dest.base);
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+ void sub16(RegisterID src, Address dest)
+ {
+ m_assembler.subw_rm(src, dest.offset, dest.base);
+ }
+
+ void sub8(RegisterID src, Address dest)
+ {
+ m_assembler.subb_rm(src, dest.offset, dest.base);
+ }
+
+ void sub32(RegisterID src, BaseIndex dest)
+ {
+ m_assembler.subl_rm(src, dest.offset, dest.base, dest.index, dest.scale);
+ }
+
+ void sub16(RegisterID src, BaseIndex dest)
+ {
+ m_assembler.subw_rm(src, dest.offset, dest.base, dest.index, dest.scale);
+ }
+
+ void sub8(RegisterID src, BaseIndex dest)
+ {
+ m_assembler.subb_rm(src, dest.offset, dest.base, dest.index, dest.scale);
+ }
+
</ins><span class="cx"> void xor32(RegisterID src, RegisterID dest)
</span><span class="cx"> {
</span><span class="cx"> m_assembler.xorl_rr(src, dest);
</span><span class="lines">@@ -677,6 +872,50 @@
</span><span class="cx"> m_assembler.xorl_im(imm.m_value, dest.offset, dest.base);
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+ void xor32(TrustedImm32 imm, BaseIndex dest)
+ {
+ if (imm.m_value == -1)
+ m_assembler.notl_m(dest.offset, dest.base, dest.index, dest.scale);
+ else
+ m_assembler.xorl_im(imm.m_value, dest.offset, dest.base, dest.index, dest.scale);
+ }
+
+ void xor16(TrustedImm32 imm, Address dest)
+ {
+ imm.m_value = static_cast<int16_t>(imm.m_value);
+ if (imm.m_value == -1)
+ m_assembler.notw_m(dest.offset, dest.base);
+ else
+ m_assembler.xorw_im(imm.m_value, dest.offset, dest.base);
+ }
+
+ void xor16(TrustedImm32 imm, BaseIndex dest)
+ {
+ imm.m_value = static_cast<int16_t>(imm.m_value);
+ if (imm.m_value == -1)
+ m_assembler.notw_m(dest.offset, dest.base, dest.index, dest.scale);
+ else
+ m_assembler.xorw_im(imm.m_value, dest.offset, dest.base, dest.index, dest.scale);
+ }
+
+ void xor8(TrustedImm32 imm, Address dest)
+ {
+ imm.m_value = static_cast<int8_t>(imm.m_value);
+ if (imm.m_value == -1)
+ m_assembler.notb_m(dest.offset, dest.base);
+ else
+ m_assembler.xorb_im(imm.m_value, dest.offset, dest.base);
+ }
+
+ void xor8(TrustedImm32 imm, BaseIndex dest)
+ {
+ imm.m_value = static_cast<int8_t>(imm.m_value);
+ if (imm.m_value == -1)
+ m_assembler.notb_m(dest.offset, dest.base, dest.index, dest.scale);
+ else
+ m_assembler.xorb_im(imm.m_value, dest.offset, dest.base, dest.index, dest.scale);
+ }
+
</ins><span class="cx"> void xor32(TrustedImm32 imm, RegisterID dest)
</span><span class="cx"> {
</span><span class="cx"> if (imm.m_value == -1)
</span><span class="lines">@@ -690,11 +929,41 @@
</span><span class="cx"> m_assembler.xorl_rm(src, dest.offset, dest.base);
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+ void xor32(RegisterID src, BaseIndex dest)
+ {
+ m_assembler.xorl_rm(src, dest.offset, dest.base, dest.index, dest.scale);
+ }
+
+ void xor16(RegisterID src, Address dest)
+ {
+ m_assembler.xorw_rm(src, dest.offset, dest.base);
+ }
+
+ void xor16(RegisterID src, BaseIndex dest)
+ {
+ m_assembler.xorw_rm(src, dest.offset, dest.base, dest.index, dest.scale);
+ }
+
+ void xor8(RegisterID src, Address dest)
+ {
+ m_assembler.xorb_rm(src, dest.offset, dest.base);
+ }
+
+ void xor8(RegisterID src, BaseIndex dest)
+ {
+ m_assembler.xorb_rm(src, dest.offset, dest.base, dest.index, dest.scale);
+ }
+
</ins><span class="cx"> void xor32(Address src, RegisterID dest)
</span><span class="cx"> {
</span><span class="cx"> m_assembler.xorl_mr(src.offset, src.base, dest);
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+ void xor32(BaseIndex src, RegisterID dest)
+ {
+ m_assembler.xorl_mr(src.offset, src.base, src.index, src.scale, dest);
+ }
+
</ins><span class="cx"> void xor32(RegisterID op1, RegisterID op2, RegisterID dest)
</span><span class="cx"> {
</span><span class="cx"> if (op1 == op2)
</span><span class="lines">@@ -741,6 +1010,31 @@
</span><span class="cx"> m_assembler.notl_m(dest.offset, dest.base);
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+ void not32(BaseIndex dest)
+ {
+ m_assembler.notl_m(dest.offset, dest.base, dest.index, dest.scale);
+ }
+
+ void not16(Address dest)
+ {
+ m_assembler.notw_m(dest.offset, dest.base);
+ }
+
+ void not16(BaseIndex dest)
+ {
+ m_assembler.notw_m(dest.offset, dest.base, dest.index, dest.scale);
+ }
+
+ void not8(Address dest)
+ {
+ m_assembler.notb_m(dest.offset, dest.base);
+ }
+
+ void not8(BaseIndex dest)
+ {
+ m_assembler.notb_m(dest.offset, dest.base, dest.index, dest.scale);
+ }
+
</ins><span class="cx"> void sqrtDouble(FPRegisterID src, FPRegisterID dst)
</span><span class="cx"> {
</span><span class="cx"> m_assembler.sqrtsd_rr(src, dst);
</span><span class="lines">@@ -1071,43 +1365,24 @@
</span><span class="cx">
</span><span class="cx"> void store16(RegisterID src, BaseIndex address)
</span><span class="cx"> {
</span><del>-#if CPU(X86)
- // On 32-bit x86 we can only store from the first 4 registers;
- // esp..edi are mapped to the 'h' registers!
- if (src >= 4) {
- // Pick a temporary register.
- RegisterID temp = getUnusedRegister(address);
-
- // Swap to the temporary register to perform the store.
- swap(src, temp);
- m_assembler.movw_rm(temp, address.offset, address.base, address.index, address.scale);
- swap(src, temp);
- return;
- }
-#endif
</del><span class="cx"> m_assembler.movw_rm(src, address.offset, address.base, address.index, address.scale);
</span><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> void store16(RegisterID src, Address address)
</span><span class="cx"> {
</span><del>-#if CPU(X86)
- // On 32-bit x86 we can only store from the first 4 registers;
- // esp..edi are mapped to the 'h' registers!
- if (src >= 4) {
- // Pick a temporary register.
- RegisterID temp = getUnusedRegister(address);
-
- // Swap to the temporary register to perform the store.
- swap(src, temp);
- m_assembler.movw_rm(temp, address.offset, address.base);
- swap(src, temp);
- return;
- }
-#endif
</del><span class="cx"> m_assembler.movw_rm(src, address.offset, address.base);
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+ void store16(TrustedImm32 imm, BaseIndex address)
+ {
+ m_assembler.movw_im(static_cast<int16_t>(imm.m_value), address.offset, address.base, address.index, address.scale);
+ }
</ins><span class="cx">
</span><ins>+ void store16(TrustedImm32 imm, Address address)
+ {
+ m_assembler.movw_im(static_cast<int16_t>(imm.m_value), address.offset, address.base);
+ }
+
</ins><span class="cx"> // Floating-point operation:
</span><span class="cx"> //
</span><span class="cx"> // Presently only supports SSE, not x87 floating point.
</span><span class="lines">@@ -2745,7 +3020,834 @@
</span><span class="cx"> m_assembler.lock();
</span><span class="cx"> m_assembler.orl_im(0, 0, X86Registers::esp);
</span><span class="cx"> }
</span><ins>+
+ void atomicStrongCAS8(StatusCondition cond, RegisterID expectedAndResult, RegisterID newValue, Address address, RegisterID result)
+ {
+ atomicStrongCAS(cond, expectedAndResult, result, [&] { m_assembler.cmpxchgb_rm(newValue, address.offset, address.base); });
+ }
</ins><span class="cx">
</span><ins>+ void atomicStrongCAS8(StatusCondition cond, RegisterID expectedAndResult, RegisterID newValue, BaseIndex address, RegisterID result)
+ {
+ atomicStrongCAS(cond, expectedAndResult, result, [&] { m_assembler.cmpxchgb_rm(newValue, address.offset, address.base, address.index, address.scale); });
+ }
+
+ void atomicStrongCAS16(StatusCondition cond, RegisterID expectedAndResult, RegisterID newValue, Address address, RegisterID result)
+ {
+ atomicStrongCAS(cond, expectedAndResult, result, [&] { m_assembler.cmpxchgw_rm(newValue, address.offset, address.base); });
+ }
+
+ void atomicStrongCAS16(StatusCondition cond, RegisterID expectedAndResult, RegisterID newValue, BaseIndex address, RegisterID result)
+ {
+ atomicStrongCAS(cond, expectedAndResult, result, [&] { m_assembler.cmpxchgw_rm(newValue, address.offset, address.base, address.index, address.scale); });
+ }
+
+ void atomicStrongCAS32(StatusCondition cond, RegisterID expectedAndResult, RegisterID newValue, Address address, RegisterID result)
+ {
+ atomicStrongCAS(cond, expectedAndResult, result, [&] { m_assembler.cmpxchgl_rm(newValue, address.offset, address.base); });
+ }
+
+ void atomicStrongCAS32(StatusCondition cond, RegisterID expectedAndResult, RegisterID newValue, BaseIndex address, RegisterID result)
+ {
+ atomicStrongCAS(cond, expectedAndResult, result, [&] { m_assembler.cmpxchgl_rm(newValue, address.offset, address.base, address.index, address.scale); });
+ }
+
+ void atomicStrongCAS8(RegisterID expectedAndResult, RegisterID newValue, Address address)
+ {
+ atomicStrongCAS(expectedAndResult, [&] { m_assembler.cmpxchgb_rm(newValue, address.offset, address.base); });
+ }
+
+ void atomicStrongCAS8(RegisterID expectedAndResult, RegisterID newValue, BaseIndex address)
+ {
+ atomicStrongCAS(expectedAndResult, [&] { m_assembler.cmpxchgb_rm(newValue, address.offset, address.base, address.index, address.scale); });
+ }
+
+ void atomicStrongCAS16(RegisterID expectedAndResult, RegisterID newValue, Address address)
+ {
+ atomicStrongCAS(expectedAndResult, [&] { m_assembler.cmpxchgw_rm(newValue, address.offset, address.base); });
+ }
+
+ void atomicStrongCAS16(RegisterID expectedAndResult, RegisterID newValue, BaseIndex address)
+ {
+ atomicStrongCAS(expectedAndResult, [&] { m_assembler.cmpxchgw_rm(newValue, address.offset, address.base, address.index, address.scale); });
+ }
+
+ void atomicStrongCAS32(RegisterID expectedAndResult, RegisterID newValue, Address address)
+ {
+ atomicStrongCAS(expectedAndResult, [&] { m_assembler.cmpxchgl_rm(newValue, address.offset, address.base); });
+ }
+
+ void atomicStrongCAS32(RegisterID expectedAndResult, RegisterID newValue, BaseIndex address)
+ {
+ atomicStrongCAS(expectedAndResult, [&] { m_assembler.cmpxchgl_rm(newValue, address.offset, address.base, address.index, address.scale); });
+ }
+
+ Jump branchAtomicStrongCAS8(StatusCondition cond, RegisterID expectedAndResult, RegisterID newValue, Address address)
+ {
+ return branchAtomicStrongCAS(cond, expectedAndResult, [&] { m_assembler.cmpxchgb_rm(newValue, address.offset, address.base); });
+ }
+
+ Jump branchAtomicStrongCAS8(StatusCondition cond, RegisterID expectedAndResult, RegisterID newValue, BaseIndex address)
+ {
+ return branchAtomicStrongCAS(cond, expectedAndResult, [&] { m_assembler.cmpxchgb_rm(newValue, address.offset, address.base, address.index, address.scale); });
+ }
+
+ Jump branchAtomicStrongCAS16(StatusCondition cond, RegisterID expectedAndResult, RegisterID newValue, Address address)
+ {
+ return branchAtomicStrongCAS(cond, expectedAndResult, [&] { m_assembler.cmpxchgw_rm(newValue, address.offset, address.base); });
+ }
+
+ Jump branchAtomicStrongCAS16(StatusCondition cond, RegisterID expectedAndResult, RegisterID newValue, BaseIndex address)
+ {
+ return branchAtomicStrongCAS(cond, expectedAndResult, [&] { m_assembler.cmpxchgw_rm(newValue, address.offset, address.base, address.index, address.scale); });
+ }
+
+ Jump branchAtomicStrongCAS32(StatusCondition cond, RegisterID expectedAndResult, RegisterID newValue, Address address)
+ {
+ return branchAtomicStrongCAS(cond, expectedAndResult, [&] { m_assembler.cmpxchgl_rm(newValue, address.offset, address.base); });
+ }
+
+ Jump branchAtomicStrongCAS32(StatusCondition cond, RegisterID expectedAndResult, RegisterID newValue, BaseIndex address)
+ {
+ return branchAtomicStrongCAS(cond, expectedAndResult, [&] { m_assembler.cmpxchgl_rm(newValue, address.offset, address.base, address.index, address.scale); });
+ }
+
+ // If you use weak CAS, you cannot rely on expectedAndClobbered to have any particular value after
+ // this completes. On x86, it will contain the result of the strong CAS. On ARM, it will still have
+ // the expected value.
+ void atomicWeakCAS8(StatusCondition cond, RegisterID expectedAndClobbered, RegisterID newValue, Address address, RegisterID result)
+ {
+ atomicStrongCAS8(cond, expectedAndClobbered, newValue, address, result);
+ }
+
+ void atomicWeakCAS8(StatusCondition cond, RegisterID expectedAndClobbered, RegisterID newValue, BaseIndex address, RegisterID result)
+ {
+ atomicStrongCAS8(cond, expectedAndClobbered, newValue, address, result);
+ }
+
+ void atomicWeakCAS16(StatusCondition cond, RegisterID expectedAndClobbered, RegisterID newValue, Address address, RegisterID result)
+ {
+ atomicStrongCAS16(cond, expectedAndClobbered, newValue, address, result);
+ }
+
+ void atomicWeakCAS16(StatusCondition cond, RegisterID expectedAndClobbered, RegisterID newValue, BaseIndex address, RegisterID result)
+ {
+ atomicStrongCAS16(cond, expectedAndClobbered, newValue, address, result);
+ }
+
+ void atomicWeakCAS32(StatusCondition cond, RegisterID expectedAndClobbered, RegisterID newValue, Address address, RegisterID result)
+ {
+ atomicStrongCAS32(cond, expectedAndClobbered, newValue, address, result);
+ }
+
+ void atomicWeakCAS32(StatusCondition cond, RegisterID expectedAndClobbered, RegisterID newValue, BaseIndex address, RegisterID result)
+ {
+ atomicStrongCAS32(cond, expectedAndClobbered, newValue, address, result);
+ }
+
+ Jump branchAtomicWeakCAS8(StatusCondition cond, RegisterID expectedAndClobbered, RegisterID newValue, Address address)
+ {
+ return branchAtomicStrongCAS8(cond, expectedAndClobbered, newValue, address);
+ }
+
+ Jump branchAtomicWeakCAS8(StatusCondition cond, RegisterID expectedAndClobbered, RegisterID newValue, BaseIndex address)
+ {
+ return branchAtomicStrongCAS8(cond, expectedAndClobbered, newValue, address);
+ }
+
+ Jump branchAtomicWeakCAS16(StatusCondition cond, RegisterID expectedAndClobbered, RegisterID newValue, Address address)
+ {
+ return branchAtomicStrongCAS16(cond, expectedAndClobbered, newValue, address);
+ }
+
+ Jump branchAtomicWeakCAS16(StatusCondition cond, RegisterID expectedAndClobbered, RegisterID newValue, BaseIndex address)
+ {
+ return branchAtomicStrongCAS16(cond, expectedAndClobbered, newValue, address);
+ }
+
+ Jump branchAtomicWeakCAS32(StatusCondition cond, RegisterID expectedAndClobbered, RegisterID newValue, Address address)
+ {
+ return branchAtomicStrongCAS32(cond, expectedAndClobbered, newValue, address);
+ }
+
+ Jump branchAtomicWeakCAS32(StatusCondition cond, RegisterID expectedAndClobbered, RegisterID newValue, BaseIndex address)
+ {
+ return branchAtomicStrongCAS32(cond, expectedAndClobbered, newValue, address);
+ }
+
+ void atomicRelaxedWeakCAS8(StatusCondition cond, RegisterID expectedAndClobbered, RegisterID newValue, Address address, RegisterID result)
+ {
+ atomicStrongCAS8(cond, expectedAndClobbered, newValue, address, result);
+ }
+
+ void atomicRelaxedWeakCAS8(StatusCondition cond, RegisterID expectedAndClobbered, RegisterID newValue, BaseIndex address, RegisterID result)
+ {
+ atomicStrongCAS8(cond, expectedAndClobbered, newValue, address, result);
+ }
+
+ void atomicRelaxedWeakCAS16(StatusCondition cond, RegisterID expectedAndClobbered, RegisterID newValue, Address address, RegisterID result)
+ {
+ atomicStrongCAS16(cond, expectedAndClobbered, newValue, address, result);
+ }
+
+ void atomicRelaxedWeakCAS16(StatusCondition cond, RegisterID expectedAndClobbered, RegisterID newValue, BaseIndex address, RegisterID result)
+ {
+ atomicStrongCAS16(cond, expectedAndClobbered, newValue, address, result);
+ }
+
+ void atomicRelaxedWeakCAS32(StatusCondition cond, RegisterID expectedAndClobbered, RegisterID newValue, Address address, RegisterID result)
+ {
+ atomicStrongCAS32(cond, expectedAndClobbered, newValue, address, result);
+ }
+
+ void atomicRelaxedWeakCAS32(StatusCondition cond, RegisterID expectedAndClobbered, RegisterID newValue, BaseIndex address, RegisterID result)
+ {
+ atomicStrongCAS32(cond, expectedAndClobbered, newValue, address, result);
+ }
+
+ Jump branchAtomicRelaxedWeakCAS8(StatusCondition cond, RegisterID expectedAndClobbered, RegisterID newValue, Address address)
+ {
+ return branchAtomicStrongCAS8(cond, expectedAndClobbered, newValue, address);
+ }
+
+ Jump branchAtomicRelaxedWeakCAS8(StatusCondition cond, RegisterID expectedAndClobbered, RegisterID newValue, BaseIndex address)
+ {
+ return branchAtomicStrongCAS8(cond, expectedAndClobbered, newValue, address);
+ }
+
+ Jump branchAtomicRelaxedWeakCAS16(StatusCondition cond, RegisterID expectedAndClobbered, RegisterID newValue, Address address)
+ {
+ return branchAtomicStrongCAS16(cond, expectedAndClobbered, newValue, address);
+ }
+
+ Jump branchAtomicRelaxedWeakCAS16(StatusCondition cond, RegisterID expectedAndClobbered, RegisterID newValue, BaseIndex address)
+ {
+ return branchAtomicStrongCAS16(cond, expectedAndClobbered, newValue, address);
+ }
+
+ Jump branchAtomicRelaxedWeakCAS32(StatusCondition cond, RegisterID expectedAndClobbered, RegisterID newValue, Address address)
+ {
+ return branchAtomicStrongCAS32(cond, expectedAndClobbered, newValue, address);
+ }
+
+ Jump branchAtomicRelaxedWeakCAS32(StatusCondition cond, RegisterID expectedAndClobbered, RegisterID newValue, BaseIndex address)
+ {
+ return branchAtomicStrongCAS32(cond, expectedAndClobbered, newValue, address);
+ }
+
+ void atomicAdd8(TrustedImm32 imm, Address address)
+ {
+ m_assembler.lock();
+ add8(imm, address);
+ }
+
+ void atomicAdd8(TrustedImm32 imm, BaseIndex address)
+ {
+ m_assembler.lock();
+ add8(imm, address);
+ }
+
+ void atomicAdd8(RegisterID reg, Address address)
+ {
+ m_assembler.lock();
+ add8(reg, address);
+ }
+
+ void atomicAdd8(RegisterID reg, BaseIndex address)
+ {
+ m_assembler.lock();
+ add8(reg, address);
+ }
+
+ void atomicAdd16(TrustedImm32 imm, Address address)
+ {
+ m_assembler.lock();
+ add16(imm, address);
+ }
+
+ void atomicAdd16(TrustedImm32 imm, BaseIndex address)
+ {
+ m_assembler.lock();
+ add16(imm, address);
+ }
+
+ void atomicAdd16(RegisterID reg, Address address)
+ {
+ m_assembler.lock();
+ add16(reg, address);
+ }
+
+ void atomicAdd16(RegisterID reg, BaseIndex address)
+ {
+ m_assembler.lock();
+ add16(reg, address);
+ }
+
+ void atomicAdd32(TrustedImm32 imm, Address address)
+ {
+ m_assembler.lock();
+ add32(imm, address);
+ }
+
+ void atomicAdd32(TrustedImm32 imm, BaseIndex address)
+ {
+ m_assembler.lock();
+ add32(imm, address);
+ }
+
+ void atomicAdd32(RegisterID reg, Address address)
+ {
+ m_assembler.lock();
+ add32(reg, address);
+ }
+
+ void atomicAdd32(RegisterID reg, BaseIndex address)
+ {
+ m_assembler.lock();
+ add32(reg, address);
+ }
+
+ void atomicSub8(TrustedImm32 imm, Address address)
+ {
+ m_assembler.lock();
+ sub8(imm, address);
+ }
+
+ void atomicSub8(TrustedImm32 imm, BaseIndex address)
+ {
+ m_assembler.lock();
+ sub8(imm, address);
+ }
+
+ void atomicSub8(RegisterID reg, Address address)
+ {
+ m_assembler.lock();
+ sub8(reg, address);
+ }
+
+ void atomicSub8(RegisterID reg, BaseIndex address)
+ {
+ m_assembler.lock();
+ sub8(reg, address);
+ }
+
+ void atomicSub16(TrustedImm32 imm, Address address)
+ {
+ m_assembler.lock();
+ sub16(imm, address);
+ }
+
+ void atomicSub16(TrustedImm32 imm, BaseIndex address)
+ {
+ m_assembler.lock();
+ sub16(imm, address);
+ }
+
+ void atomicSub16(RegisterID reg, Address address)
+ {
+ m_assembler.lock();
+ sub16(reg, address);
+ }
+
+ void atomicSub16(RegisterID reg, BaseIndex address)
+ {
+ m_assembler.lock();
+ sub16(reg, address);
+ }
+
+ void atomicSub32(TrustedImm32 imm, Address address)
+ {
+ m_assembler.lock();
+ sub32(imm, address);
+ }
+
+ void atomicSub32(TrustedImm32 imm, BaseIndex address)
+ {
+ m_assembler.lock();
+ sub32(imm, address);
+ }
+
+ void atomicSub32(RegisterID reg, Address address)
+ {
+ m_assembler.lock();
+ sub32(reg, address);
+ }
+
+ void atomicSub32(RegisterID reg, BaseIndex address)
+ {
+ m_assembler.lock();
+ sub32(reg, address);
+ }
+
+ void atomicAnd8(TrustedImm32 imm, Address address)
+ {
+ m_assembler.lock();
+ and8(imm, address);
+ }
+
+ void atomicAnd8(TrustedImm32 imm, BaseIndex address)
+ {
+ m_assembler.lock();
+ and8(imm, address);
+ }
+
+ void atomicAnd8(RegisterID reg, Address address)
+ {
+ m_assembler.lock();
+ and8(reg, address);
+ }
+
+ void atomicAnd8(RegisterID reg, BaseIndex address)
+ {
+ m_assembler.lock();
+ and8(reg, address);
+ }
+
+ void atomicAnd16(TrustedImm32 imm, Address address)
+ {
+ m_assembler.lock();
+ and16(imm, address);
+ }
+
+ void atomicAnd16(TrustedImm32 imm, BaseIndex address)
+ {
+ m_assembler.lock();
+ and16(imm, address);
+ }
+
+ void atomicAnd16(RegisterID reg, Address address)
+ {
+ m_assembler.lock();
+ and16(reg, address);
+ }
+
+ void atomicAnd16(RegisterID reg, BaseIndex address)
+ {
+ m_assembler.lock();
+ and16(reg, address);
+ }
+
+ void atomicAnd32(TrustedImm32 imm, Address address)
+ {
+ m_assembler.lock();
+ and32(imm, address);
+ }
+
+ void atomicAnd32(TrustedImm32 imm, BaseIndex address)
+ {
+ m_assembler.lock();
+ and32(imm, address);
+ }
+
+ void atomicAnd32(RegisterID reg, Address address)
+ {
+ m_assembler.lock();
+ and32(reg, address);
+ }
+
+ void atomicAnd32(RegisterID reg, BaseIndex address)
+ {
+ m_assembler.lock();
+ and32(reg, address);
+ }
+
+ void atomicOr8(TrustedImm32 imm, Address address)
+ {
+ m_assembler.lock();
+ or8(imm, address);
+ }
+
+ void atomicOr8(TrustedImm32 imm, BaseIndex address)
+ {
+ m_assembler.lock();
+ or8(imm, address);
+ }
+
+ void atomicOr8(RegisterID reg, Address address)
+ {
+ m_assembler.lock();
+ or8(reg, address);
+ }
+
+ void atomicOr8(RegisterID reg, BaseIndex address)
+ {
+ m_assembler.lock();
+ or8(reg, address);
+ }
+
+ void atomicOr16(TrustedImm32 imm, Address address)
+ {
+ m_assembler.lock();
+ or16(imm, address);
+ }
+
+ void atomicOr16(TrustedImm32 imm, BaseIndex address)
+ {
+ m_assembler.lock();
+ or16(imm, address);
+ }
+
+ void atomicOr16(RegisterID reg, Address address)
+ {
+ m_assembler.lock();
+ or16(reg, address);
+ }
+
+ void atomicOr16(RegisterID reg, BaseIndex address)
+ {
+ m_assembler.lock();
+ or16(reg, address);
+ }
+
+ void atomicOr32(TrustedImm32 imm, Address address)
+ {
+ m_assembler.lock();
+ or32(imm, address);
+ }
+
+ void atomicOr32(TrustedImm32 imm, BaseIndex address)
+ {
+ m_assembler.lock();
+ or32(imm, address);
+ }
+
+ void atomicOr32(RegisterID reg, Address address)
+ {
+ m_assembler.lock();
+ or32(reg, address);
+ }
+
+ void atomicOr32(RegisterID reg, BaseIndex address)
+ {
+ m_assembler.lock();
+ or32(reg, address);
+ }
+
+ void atomicXor8(TrustedImm32 imm, Address address)
+ {
+ m_assembler.lock();
+ xor8(imm, address);
+ }
+
+ void atomicXor8(TrustedImm32 imm, BaseIndex address)
+ {
+ m_assembler.lock();
+ xor8(imm, address);
+ }
+
+ void atomicXor8(RegisterID reg, Address address)
+ {
+ m_assembler.lock();
+ xor8(reg, address);
+ }
+
+ void atomicXor8(RegisterID reg, BaseIndex address)
+ {
+ m_assembler.lock();
+ xor8(reg, address);
+ }
+
+ void atomicXor16(TrustedImm32 imm, Address address)
+ {
+ m_assembler.lock();
+ xor16(imm, address);
+ }
+
+ void atomicXor16(TrustedImm32 imm, BaseIndex address)
+ {
+ m_assembler.lock();
+ xor16(imm, address);
+ }
+
+ void atomicXor16(RegisterID reg, Address address)
+ {
+ m_assembler.lock();
+ xor16(reg, address);
+ }
+
+ void atomicXor16(RegisterID reg, BaseIndex address)
+ {
+ m_assembler.lock();
+ xor16(reg, address);
+ }
+
+ void atomicXor32(TrustedImm32 imm, Address address)
+ {
+ m_assembler.lock();
+ xor32(imm, address);
+ }
+
+ void atomicXor32(TrustedImm32 imm, BaseIndex address)
+ {
+ m_assembler.lock();
+ xor32(imm, address);
+ }
+
+ void atomicXor32(RegisterID reg, Address address)
+ {
+ m_assembler.lock();
+ xor32(reg, address);
+ }
+
+ void atomicXor32(RegisterID reg, BaseIndex address)
+ {
+ m_assembler.lock();
+ xor32(reg, address);
+ }
+
+ void atomicNeg8(Address address)
+ {
+ m_assembler.lock();
+ neg8(address);
+ }
+
+ void atomicNeg8(BaseIndex address)
+ {
+ m_assembler.lock();
+ neg8(address);
+ }
+
+ void atomicNeg16(Address address)
+ {
+ m_assembler.lock();
+ neg16(address);
+ }
+
+ void atomicNeg16(BaseIndex address)
+ {
+ m_assembler.lock();
+ neg16(address);
+ }
+
+ void atomicNeg32(Address address)
+ {
+ m_assembler.lock();
+ neg32(address);
+ }
+
+ void atomicNeg32(BaseIndex address)
+ {
+ m_assembler.lock();
+ neg32(address);
+ }
+
+ void atomicNot8(Address address)
+ {
+ m_assembler.lock();
+ not8(address);
+ }
+
+ void atomicNot8(BaseIndex address)
+ {
+ m_assembler.lock();
+ not8(address);
+ }
+
+ void atomicNot16(Address address)
+ {
+ m_assembler.lock();
+ not16(address);
+ }
+
+ void atomicNot16(BaseIndex address)
+ {
+ m_assembler.lock();
+ not16(address);
+ }
+
+ void atomicNot32(Address address)
+ {
+ m_assembler.lock();
+ not32(address);
+ }
+
+ void atomicNot32(BaseIndex address)
+ {
+ m_assembler.lock();
+ not32(address);
+ }
+
+ void atomicXchgAdd8(RegisterID reg, Address address)
+ {
+ m_assembler.lock();
+ m_assembler.xaddb_rm(reg, address.offset, address.base);
+ }
+
+ void atomicXchgAdd8(RegisterID reg, BaseIndex address)
+ {
+ m_assembler.lock();
+ m_assembler.xaddb_rm(reg, address.offset, address.base, address.index, address.scale);
+ }
+
+ void atomicXchgAdd16(RegisterID reg, Address address)
+ {
+ m_assembler.lock();
+ m_assembler.xaddw_rm(reg, address.offset, address.base);
+ }
+
+ void atomicXchgAdd16(RegisterID reg, BaseIndex address)
+ {
+ m_assembler.lock();
+ m_assembler.xaddw_rm(reg, address.offset, address.base, address.index, address.scale);
+ }
+
+ void atomicXchgAdd32(RegisterID reg, Address address)
+ {
+ m_assembler.lock();
+ m_assembler.xaddl_rm(reg, address.offset, address.base);
+ }
+
+ void atomicXchgAdd32(RegisterID reg, BaseIndex address)
+ {
+ m_assembler.lock();
+ m_assembler.xaddl_rm(reg, address.offset, address.base, address.index, address.scale);
+ }
+
+ void atomicXchg8(RegisterID reg, Address address)
+ {
+ m_assembler.lock();
+ m_assembler.xchgb_rm(reg, address.offset, address.base);
+ }
+
+ void atomicXchg8(RegisterID reg, BaseIndex address)
+ {
+ m_assembler.lock();
+ m_assembler.xchgb_rm(reg, address.offset, address.base, address.index, address.scale);
+ }
+
+ void atomicXchg16(RegisterID reg, Address address)
+ {
+ m_assembler.lock();
+ m_assembler.xchgw_rm(reg, address.offset, address.base);
+ }
+
+ void atomicXchg16(RegisterID reg, BaseIndex address)
+ {
+ m_assembler.lock();
+ m_assembler.xchgw_rm(reg, address.offset, address.base, address.index, address.scale);
+ }
+
+ void atomicXchg32(RegisterID reg, Address address)
+ {
+ m_assembler.lock();
+ m_assembler.xchgl_rm(reg, address.offset, address.base);
+ }
+
+ void atomicXchg32(RegisterID reg, BaseIndex address)
+ {
+ m_assembler.lock();
+ m_assembler.xchgl_rm(reg, address.offset, address.base, address.index, address.scale);
+ }
+
+ void loadAcq8(Address src, RegisterID dest)
+ {
+ load8(src, dest);
+ }
+
+ void loadAcq8(BaseIndex src, RegisterID dest)
+ {
+ load8(src, dest);
+ }
+
+ void loadAcq8SignedExtendTo32(Address src, RegisterID dest)
+ {
+ load8SignedExtendTo32(src, dest);
+ }
+
+ void loadAcq8SignedExtendTo32(BaseIndex src, RegisterID dest)
+ {
+ load8SignedExtendTo32(src, dest);
+ }
+
+ void loadAcq16(Address src, RegisterID dest)
+ {
+ load16(src, dest);
+ }
+
+ void loadAcq16(BaseIndex src, RegisterID dest)
+ {
+ load16(src, dest);
+ }
+
+ void loadAcq16SignedExtendTo32(Address src, RegisterID dest)
+ {
+ load16SignedExtendTo32(src, dest);
+ }
+
+ void loadAcq16SignedExtendTo32(BaseIndex src, RegisterID dest)
+ {
+ load16SignedExtendTo32(src, dest);
+ }
+
+ void loadAcq32(Address src, RegisterID dest)
+ {
+ load32(src, dest);
+ }
+
+ void loadAcq32(BaseIndex src, RegisterID dest)
+ {
+ load32(src, dest);
+ }
+
+ void storeRel8(RegisterID src, Address dest)
+ {
+ store8(src, dest);
+ }
+
+ void storeRel8(RegisterID src, BaseIndex dest)
+ {
+ store8(src, dest);
+ }
+
+ void storeRel8(TrustedImm32 imm, Address dest)
+ {
+ store8(imm, dest);
+ }
+
+ void storeRel8(TrustedImm32 imm, BaseIndex dest)
+ {
+ store8(imm, dest);
+ }
+
+ void storeRel16(RegisterID src, Address dest)
+ {
+ store16(src, dest);
+ }
+
+ void storeRel16(RegisterID src, BaseIndex dest)
+ {
+ store16(src, dest);
+ }
+
+ void storeRel16(TrustedImm32 imm, Address dest)
+ {
+ store16(imm, dest);
+ }
+
+ void storeRel16(TrustedImm32 imm, BaseIndex dest)
+ {
+ store16(imm, dest);
+ }
+
+ void storeRel32(RegisterID src, Address dest)
+ {
+ store32(src, dest);
+ }
+
+ void storeRel32(RegisterID src, BaseIndex dest)
+ {
+ store32(src, dest);
+ }
+
+ void storeRel32(TrustedImm32 imm, Address dest)
+ {
+ store32(imm, dest);
+ }
+
+ void storeRel32(TrustedImm32 imm, BaseIndex dest)
+ {
+ store32(imm, dest);
+ }
+
</ins><span class="cx"> // We take this to mean that it prevents motion of normal stores. So, it's a no-op on x86.
</span><span class="cx"> void storeFence()
</span><span class="cx"> {
</span><span class="lines">@@ -2838,6 +3940,18 @@
</span><span class="cx"> return static_cast<X86Assembler::Condition>(cond);
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+ X86Assembler::Condition x86Condition(StatusCondition cond)
+ {
+ switch (cond) {
+ case Success:
+ return X86Assembler::ConditionE;
+ case Failure:
+ return X86Assembler::ConditionNE;
+ }
+ RELEASE_ASSERT_NOT_REACHED();
+ return X86Assembler::ConditionE;
+ }
+
</ins><span class="cx"> void set32(X86Assembler::Condition cond, RegisterID dest)
</span><span class="cx"> {
</span><span class="cx"> #if CPU(X86)
</span><span class="lines">@@ -2932,7 +4046,36 @@
</span><span class="cx"> move(TrustedImm32(sizeOfRegister), dst);
</span><span class="cx"> srcIsNonZero.link(this);
</span><span class="cx"> }
</span><ins>+
+ template<typename Func>
+ void atomicStrongCAS(StatusCondition cond, RegisterID expectedAndResult, RegisterID result, const Func& func)
+ {
+ swap(expectedAndResult, X86Registers::eax);
+ m_assembler.lock();
+ func();
+ swap(expectedAndResult, X86Registers::eax);
+ set32(x86Condition(cond), result);
+ }
</ins><span class="cx">
</span><ins>+ template<typename Func>
+ void atomicStrongCAS(RegisterID expectedAndResult, const Func& func)
+ {
+ swap(expectedAndResult, X86Registers::eax);
+ m_assembler.lock();
+ func();
+ swap(expectedAndResult, X86Registers::eax);
+ }
+
+ template<typename Func>
+ Jump branchAtomicStrongCAS(StatusCondition cond, RegisterID expectedAndResult, const Func& func)
+ {
+ swap(expectedAndResult, X86Registers::eax);
+ m_assembler.lock();
+ func();
+ swap(expectedAndResult, X86Registers::eax);
+ return Jump(m_assembler.jCC(x86Condition(cond)));
+ }
+
</ins><span class="cx"> private:
</span><span class="cx"> // Only MacroAssemblerX86 should be using the following method; SSE2 is always available on
</span><span class="cx"> // x86_64, and clients & subclasses of MacroAssembler should be using 'supportsFloatingPoint()'.
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreassemblerMacroAssemblerX86_64h"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/assembler/MacroAssemblerX86_64.h (213713 => 213714)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/assembler/MacroAssemblerX86_64.h 2017-03-10 17:32:07 UTC (rev 213713)
+++ trunk/Source/JavaScriptCore/assembler/MacroAssemblerX86_64.h 2017-03-10 17:49:42 UTC (rev 213714)
</span><span class="lines">@@ -1,5 +1,5 @@
</span><span class="cx"> /*
</span><del>- * Copyright (C) 2008, 2012, 2014-2016 Apple Inc. All rights reserved.
</del><ins>+ * Copyright (C) 2008-2017 Apple Inc. All rights reserved.
</ins><span class="cx"> *
</span><span class="cx"> * Redistribution and use in source and binary forms, with or without
</span><span class="cx"> * modification, are permitted provided that the following conditions
</span><span class="lines">@@ -264,11 +264,21 @@
</span><span class="cx"> m_assembler.addq_mr(src.offset, src.base, dest);
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+ void add64(BaseIndex src, RegisterID dest)
+ {
+ m_assembler.addq_mr(src.offset, src.base, src.index, src.scale, dest);
+ }
+
</ins><span class="cx"> void add64(RegisterID src, Address dest)
</span><span class="cx"> {
</span><span class="cx"> m_assembler.addq_rm(src, dest.offset, dest.base);
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+ void add64(RegisterID src, BaseIndex dest)
+ {
+ m_assembler.addq_rm(src, dest.offset, dest.base, dest.index, dest.scale);
+ }
+
</ins><span class="cx"> void add64(AbsoluteAddress src, RegisterID dest)
</span><span class="cx"> {
</span><span class="cx"> move(TrustedImmPtr(src.m_ptr), scratchRegister());
</span><span class="lines">@@ -306,6 +316,14 @@
</span><span class="cx"> m_assembler.addq_im(imm.m_value, address.offset, address.base);
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+ void add64(TrustedImm32 imm, BaseIndex address)
+ {
+ if (imm.m_value == 1)
+ m_assembler.incq_m(address.offset, address.base, address.index, address.scale);
+ else
+ m_assembler.addq_im(imm.m_value, address.offset, address.base, address.index, address.scale);
+ }
+
</ins><span class="cx"> void add64(TrustedImm32 imm, AbsoluteAddress address)
</span><span class="cx"> {
</span><span class="cx"> move(TrustedImmPtr(address.m_ptr), scratchRegister());
</span><span class="lines">@@ -342,11 +360,41 @@
</span><span class="cx"> m_assembler.andq_rr(src, dest);
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+ void and64(RegisterID src, Address dest)
+ {
+ m_assembler.andq_rm(src, dest.offset, dest.base);
+ }
+
+ void and64(RegisterID src, BaseIndex dest)
+ {
+ m_assembler.andq_rm(src, dest.offset, dest.base, dest.index, dest.scale);
+ }
+
+ void and64(Address src, RegisterID dest)
+ {
+ m_assembler.andq_mr(src.offset, src.base, dest);
+ }
+
+ void and64(BaseIndex src, RegisterID dest)
+ {
+ m_assembler.andq_mr(src.offset, src.base, src.index, src.scale, dest);
+ }
+
</ins><span class="cx"> void and64(TrustedImm32 imm, RegisterID srcDest)
</span><span class="cx"> {
</span><span class="cx"> m_assembler.andq_ir(imm.m_value, srcDest);
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+ void and64(TrustedImm32 imm, Address dest)
+ {
+ m_assembler.andq_im(imm.m_value, dest.offset, dest.base);
+ }
+
+ void and64(TrustedImm32 imm, BaseIndex dest)
+ {
+ m_assembler.andq_im(imm.m_value, dest.offset, dest.base, dest.index, dest.scale);
+ }
+
</ins><span class="cx"> void and64(TrustedImmPtr imm, RegisterID srcDest)
</span><span class="cx"> {
</span><span class="cx"> intptr_t intValue = imm.asIntptr();
</span><span class="lines">@@ -552,11 +600,51 @@
</span><span class="cx"> m_assembler.negq_r(dest);
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+ void neg64(Address dest)
+ {
+ m_assembler.negq_m(dest.offset, dest.base);
+ }
+
+ void neg64(BaseIndex dest)
+ {
+ m_assembler.negq_m(dest.offset, dest.base, dest.index, dest.scale);
+ }
+
</ins><span class="cx"> void or64(RegisterID src, RegisterID dest)
</span><span class="cx"> {
</span><span class="cx"> m_assembler.orq_rr(src, dest);
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+ void or64(RegisterID src, Address dest)
+ {
+ m_assembler.orq_rm(src, dest.offset, dest.base);
+ }
+
+ void or64(RegisterID src, BaseIndex dest)
+ {
+ m_assembler.orq_rm(src, dest.offset, dest.base, dest.index, dest.scale);
+ }
+
+ void or64(Address src, RegisterID dest)
+ {
+ m_assembler.orq_mr(src.offset, src.base, dest);
+ }
+
+ void or64(BaseIndex src, RegisterID dest)
+ {
+ m_assembler.orq_mr(src.offset, src.base, src.index, src.scale, dest);
+ }
+
+ void or64(TrustedImm32 imm, Address dest)
+ {
+ m_assembler.orq_im(imm.m_value, dest.offset, dest.base);
+ }
+
+ void or64(TrustedImm32 imm, BaseIndex dest)
+ {
+ m_assembler.orq_im(imm.m_value, dest.offset, dest.base, dest.index, dest.scale);
+ }
+
</ins><span class="cx"> void or64(TrustedImm64 imm, RegisterID srcDest)
</span><span class="cx"> {
</span><span class="cx"> if (imm.m_value <= std::numeric_limits<int32_t>::max()
</span><span class="lines">@@ -619,16 +707,31 @@
</span><span class="cx"> m_assembler.subq_im(imm.m_value, address.offset, address.base);
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+ void sub64(TrustedImm32 imm, BaseIndex address)
+ {
+ m_assembler.subq_im(imm.m_value, address.offset, address.base, address.index, address.scale);
+ }
+
</ins><span class="cx"> void sub64(Address src, RegisterID dest)
</span><span class="cx"> {
</span><span class="cx"> m_assembler.subq_mr(src.offset, src.base, dest);
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+ void sub64(BaseIndex src, RegisterID dest)
+ {
+ m_assembler.subq_mr(src.offset, src.base, src.index, src.scale, dest);
+ }
+
</ins><span class="cx"> void sub64(RegisterID src, Address dest)
</span><span class="cx"> {
</span><span class="cx"> m_assembler.subq_rm(src, dest.offset, dest.base);
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+ void sub64(RegisterID src, BaseIndex dest)
+ {
+ m_assembler.subq_rm(src, dest.offset, dest.base, dest.index, dest.scale);
+ }
+
</ins><span class="cx"> void xor64(RegisterID src, RegisterID dest)
</span><span class="cx"> {
</span><span class="cx"> m_assembler.xorq_rr(src, dest);
</span><span class="lines">@@ -651,6 +754,31 @@
</span><span class="cx"> m_assembler.xorq_rm(src, dest.offset, dest.base);
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+ void xor64(RegisterID src, BaseIndex dest)
+ {
+ m_assembler.xorq_rm(src, dest.offset, dest.base, dest.index, dest.scale);
+ }
+
+ void xor64(Address src, RegisterID dest)
+ {
+ m_assembler.xorq_mr(src.offset, src.base, dest);
+ }
+
+ void xor64(BaseIndex src, RegisterID dest)
+ {
+ m_assembler.xorq_mr(src.offset, src.base, src.index, src.scale, dest);
+ }
+
+ void xor64(TrustedImm32 imm, Address dest)
+ {
+ m_assembler.xorq_im(imm.m_value, dest.offset, dest.base);
+ }
+
+ void xor64(TrustedImm32 imm, BaseIndex dest)
+ {
+ m_assembler.xorq_im(imm.m_value, dest.offset, dest.base, dest.index, dest.scale);
+ }
+
</ins><span class="cx"> void xor64(TrustedImm32 imm, RegisterID srcDest)
</span><span class="cx"> {
</span><span class="cx"> m_assembler.xorq_ir(imm.m_value, srcDest);
</span><span class="lines">@@ -666,6 +794,11 @@
</span><span class="cx"> m_assembler.notq_m(dest.offset, dest.base);
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+ void not64(BaseIndex dest)
+ {
+ m_assembler.notq_m(dest.offset, dest.base, dest.index, dest.scale);
+ }
+
</ins><span class="cx"> void load64(ImplicitAddress address, RegisterID dest)
</span><span class="cx"> {
</span><span class="cx"> m_assembler.movq_mr(address.offset, address.base, dest);
</span><span class="lines">@@ -725,6 +858,11 @@
</span><span class="cx"> m_assembler.movq_i32m(imm.m_value, address.offset, address.base);
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+ void store64(TrustedImm32 imm, BaseIndex address)
+ {
+ m_assembler.movq_i32m(imm.m_value, address.offset, address.base, address.index, address.scale);
+ }
+
</ins><span class="cx"> void store64(TrustedImm64 imm, ImplicitAddress address)
</span><span class="cx"> {
</span><span class="cx"> if (CAN_SIGN_EXTEND_32_64(imm.m_value)) {
</span><span class="lines">@@ -1294,7 +1432,275 @@
</span><span class="cx"> MacroAssemblerX86Common::move(TrustedImmPtr(address.m_ptr), scratchRegister());
</span><span class="cx"> return MacroAssemblerX86Common::branchTest8(cond, Address(scratchRegister()), mask8);
</span><span class="cx"> }
</span><ins>+
+ void atomicStrongCAS64(StatusCondition cond, RegisterID expectedAndResult, RegisterID newValue, Address address, RegisterID result)
+ {
+ atomicStrongCAS(cond, expectedAndResult, result, [&] { m_assembler.cmpxchgq_rm(newValue, address.offset, address.base); });
+ }
</ins><span class="cx">
</span><ins>+ void atomicStrongCAS64(StatusCondition cond, RegisterID expectedAndResult, RegisterID newValue, BaseIndex address, RegisterID result)
+ {
+ atomicStrongCAS(cond, expectedAndResult, result, [&] { m_assembler.cmpxchgq_rm(newValue, address.offset, address.base, address.index, address.scale); });
+ }
+
+ void atomicStrongCAS64(RegisterID expectedAndResult, RegisterID newValue, Address address)
+ {
+ atomicStrongCAS(expectedAndResult, [&] { m_assembler.cmpxchgq_rm(newValue, address.offset, address.base); });
+ }
+
+ void atomicStrongCAS64(RegisterID expectedAndResult, RegisterID newValue, BaseIndex address)
+ {
+ atomicStrongCAS(expectedAndResult, [&] { m_assembler.cmpxchgq_rm(newValue, address.offset, address.base, address.index, address.scale); });
+ }
+
+ Jump branchAtomicStrongCAS64(StatusCondition cond, RegisterID expectedAndResult, RegisterID newValue, Address address)
+ {
+ return branchAtomicStrongCAS(cond, expectedAndResult, [&] { m_assembler.cmpxchgq_rm(newValue, address.offset, address.base); });
+ }
+
+ Jump branchAtomicStrongCAS64(StatusCondition cond, RegisterID expectedAndResult, RegisterID newValue, BaseIndex address)
+ {
+ return branchAtomicStrongCAS(cond, expectedAndResult, [&] { m_assembler.cmpxchgq_rm(newValue, address.offset, address.base, address.index, address.scale); });
+ }
+
+ void atomicWeakCAS64(StatusCondition cond, RegisterID expectedAndClobbered, RegisterID newValue, Address address, RegisterID result)
+ {
+ atomicStrongCAS64(cond, expectedAndClobbered, newValue, address, result);
+ }
+
+ void atomicWeakCAS64(StatusCondition cond, RegisterID expectedAndClobbered, RegisterID newValue, BaseIndex address, RegisterID result)
+ {
+ atomicStrongCAS64(cond, expectedAndClobbered, newValue, address, result);
+ }
+
+ Jump branchAtomicWeakCAS64(StatusCondition cond, RegisterID expectedAndClobbered, RegisterID newValue, Address address)
+ {
+ return branchAtomicStrongCAS64(cond, expectedAndClobbered, newValue, address);
+ }
+
+ Jump branchAtomicWeakCAS64(StatusCondition cond, RegisterID expectedAndClobbered, RegisterID newValue, BaseIndex address)
+ {
+ return branchAtomicStrongCAS64(cond, expectedAndClobbered, newValue, address);
+ }
+
+ void atomicRelaxedWeakCAS64(StatusCondition cond, RegisterID expectedAndClobbered, RegisterID newValue, Address address, RegisterID result)
+ {
+ atomicStrongCAS64(cond, expectedAndClobbered, newValue, address, result);
+ }
+
+ void atomicRelaxedWeakCAS64(StatusCondition cond, RegisterID expectedAndClobbered, RegisterID newValue, BaseIndex address, RegisterID result)
+ {
+ atomicStrongCAS64(cond, expectedAndClobbered, newValue, address, result);
+ }
+
+ Jump branchAtomicRelaxedWeakCAS64(StatusCondition cond, RegisterID expectedAndClobbered, RegisterID newValue, Address address)
+ {
+ return branchAtomicStrongCAS64(cond, expectedAndClobbered, newValue, address);
+ }
+
+ Jump branchAtomicRelaxedWeakCAS64(StatusCondition cond, RegisterID expectedAndClobbered, RegisterID newValue, BaseIndex address)
+ {
+ return branchAtomicStrongCAS64(cond, expectedAndClobbered, newValue, address);
+ }
+
+ void atomicAdd64(TrustedImm32 imm, Address address)
+ {
+ m_assembler.lock();
+ add64(imm, address);
+ }
+
+ void atomicAdd64(TrustedImm32 imm, BaseIndex address)
+ {
+ m_assembler.lock();
+ add64(imm, address);
+ }
+
+ void atomicAdd64(RegisterID reg, Address address)
+ {
+ m_assembler.lock();
+ add64(reg, address);
+ }
+
+ void atomicAdd64(RegisterID reg, BaseIndex address)
+ {
+ m_assembler.lock();
+ add64(reg, address);
+ }
+
+ void atomicSub64(TrustedImm32 imm, Address address)
+ {
+ m_assembler.lock();
+ sub64(imm, address);
+ }
+
+ void atomicSub64(TrustedImm32 imm, BaseIndex address)
+ {
+ m_assembler.lock();
+ sub64(imm, address);
+ }
+
+ void atomicSub64(RegisterID reg, Address address)
+ {
+ m_assembler.lock();
+ sub64(reg, address);
+ }
+
+ void atomicSub64(RegisterID reg, BaseIndex address)
+ {
+ m_assembler.lock();
+ sub64(reg, address);
+ }
+
+ void atomicAnd64(TrustedImm32 imm, Address address)
+ {
+ m_assembler.lock();
+ and64(imm, address);
+ }
+
+ void atomicAnd64(TrustedImm32 imm, BaseIndex address)
+ {
+ m_assembler.lock();
+ and64(imm, address);
+ }
+
+ void atomicAnd64(RegisterID reg, Address address)
+ {
+ m_assembler.lock();
+ and64(reg, address);
+ }
+
+ void atomicAnd64(RegisterID reg, BaseIndex address)
+ {
+ m_assembler.lock();
+ and64(reg, address);
+ }
+
+ void atomicOr64(TrustedImm32 imm, Address address)
+ {
+ m_assembler.lock();
+ or64(imm, address);
+ }
+
+ void atomicOr64(TrustedImm32 imm, BaseIndex address)
+ {
+ m_assembler.lock();
+ or64(imm, address);
+ }
+
+ void atomicOr64(RegisterID reg, Address address)
+ {
+ m_assembler.lock();
+ or64(reg, address);
+ }
+
+ void atomicOr64(RegisterID reg, BaseIndex address)
+ {
+ m_assembler.lock();
+ or64(reg, address);
+ }
+
+ void atomicXor64(TrustedImm32 imm, Address address)
+ {
+ m_assembler.lock();
+ xor64(imm, address);
+ }
+
+ void atomicXor64(TrustedImm32 imm, BaseIndex address)
+ {
+ m_assembler.lock();
+ xor64(imm, address);
+ }
+
+ void atomicXor64(RegisterID reg, Address address)
+ {
+ m_assembler.lock();
+ xor64(reg, address);
+ }
+
+ void atomicXor64(RegisterID reg, BaseIndex address)
+ {
+ m_assembler.lock();
+ xor64(reg, address);
+ }
+
+ void atomicNeg64(Address address)
+ {
+ m_assembler.lock();
+ neg64(address);
+ }
+
+ void atomicNeg64(BaseIndex address)
+ {
+ m_assembler.lock();
+ neg64(address);
+ }
+
+ void atomicNot64(Address address)
+ {
+ m_assembler.lock();
+ not64(address);
+ }
+
+ void atomicNot64(BaseIndex address)
+ {
+ m_assembler.lock();
+ not64(address);
+ }
+
+ void atomicXchgAdd64(RegisterID reg, Address address)
+ {
+ m_assembler.lock();
+ m_assembler.xaddq_rm(reg, address.offset, address.base);
+ }
+
+ void atomicXchgAdd64(RegisterID reg, BaseIndex address)
+ {
+ m_assembler.lock();
+ m_assembler.xaddq_rm(reg, address.offset, address.base, address.index, address.scale);
+ }
+
+ void atomicXchg64(RegisterID reg, Address address)
+ {
+ m_assembler.lock();
+ m_assembler.xchgq_rm(reg, address.offset, address.base);
+ }
+
+ void atomicXchg64(RegisterID reg, BaseIndex address)
+ {
+ m_assembler.lock();
+ m_assembler.xchgq_rm(reg, address.offset, address.base, address.index, address.scale);
+ }
+
+ void loadAcq64(Address src, RegisterID dest)
+ {
+ load64(src, dest);
+ }
+
+ void loadAcq64(BaseIndex src, RegisterID dest)
+ {
+ load64(src, dest);
+ }
+
+ void storeRel64(RegisterID src, Address dest)
+ {
+ store64(src, dest);
+ }
+
+ void storeRel64(RegisterID src, BaseIndex dest)
+ {
+ store64(src, dest);
+ }
+
+ void storeRel64(TrustedImm32 imm, Address dest)
+ {
+ store64(imm, dest);
+ }
+
+ void storeRel64(TrustedImm32 imm, BaseIndex dest)
+ {
+ store64(imm, dest);
+ }
+
</ins><span class="cx"> void truncateDoubleToUint32(FPRegisterID src, RegisterID dest)
</span><span class="cx"> {
</span><span class="cx"> m_assembler.cvttsd2siq_rr(src, dest);
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreassemblerX86Assemblerh"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/assembler/X86Assembler.h (213713 => 213714)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/assembler/X86Assembler.h 2017-03-10 17:32:07 UTC (rev 213713)
+++ trunk/Source/JavaScriptCore/assembler/X86Assembler.h 2017-03-10 17:49:42 UTC (rev 213714)
</span><span class="lines">@@ -190,16 +190,20 @@
</span><span class="cx"> OP_ADD_EvGv = 0x01,
</span><span class="cx"> OP_ADD_GvEv = 0x03,
</span><span class="cx"> OP_ADD_EAXIv = 0x05,
</span><ins>+ OP_OR_EvGb = 0x08,
</ins><span class="cx"> OP_OR_EvGv = 0x09,
</span><span class="cx"> OP_OR_GvEv = 0x0B,
</span><span class="cx"> OP_OR_EAXIv = 0x0D,
</span><span class="cx"> OP_2BYTE_ESCAPE = 0x0F,
</span><ins>+ OP_AND_EvGb = 0x20,
</ins><span class="cx"> OP_AND_EvGv = 0x21,
</span><span class="cx"> OP_AND_GvEv = 0x23,
</span><ins>+ OP_SUB_EvGb = 0x28,
</ins><span class="cx"> OP_SUB_EvGv = 0x29,
</span><span class="cx"> OP_SUB_GvEv = 0x2B,
</span><span class="cx"> OP_SUB_EAXIv = 0x2D,
</span><span class="cx"> PRE_PREDICT_BRANCH_NOT_TAKEN = 0x2E,
</span><ins>+ OP_XOR_EvGb = 0x30,
</ins><span class="cx"> OP_XOR_EvGv = 0x31,
</span><span class="cx"> OP_XOR_GvEv = 0x33,
</span><span class="cx"> OP_XOR_EAXIv = 0x35,
</span><span class="lines">@@ -223,6 +227,7 @@
</span><span class="cx"> OP_GROUP1_EvIb = 0x83,
</span><span class="cx"> OP_TEST_EbGb = 0x84,
</span><span class="cx"> OP_TEST_EvGv = 0x85,
</span><ins>+ OP_XCHG_EvGb = 0x86,
</ins><span class="cx"> OP_XCHG_EvGv = 0x87,
</span><span class="cx"> OP_MOV_EbGb = 0x88,
</span><span class="cx"> OP_MOV_EvGv = 0x89,
</span><span class="lines">@@ -252,6 +257,7 @@
</span><span class="cx"> PRE_SSE_F2 = 0xF2,
</span><span class="cx"> PRE_SSE_F3 = 0xF3,
</span><span class="cx"> OP_HLT = 0xF4,
</span><ins>+ OP_GROUP3_Eb = 0xF6,
</ins><span class="cx"> OP_GROUP3_EbIb = 0xF6,
</span><span class="cx"> OP_GROUP3_Ev = 0xF7,
</span><span class="cx"> OP_GROUP3_EvIz = 0xF7, // OP_GROUP3_Ev has an immediate, when instruction is a test.
</span><span class="lines">@@ -290,6 +296,8 @@
</span><span class="cx"> OP_SETCC = 0x90,
</span><span class="cx"> OP2_3BYTE_ESCAPE_AE = 0xAE,
</span><span class="cx"> OP2_IMUL_GvEv = 0xAF,
</span><ins>+ OP2_CMPXCHGb = 0xB0,
+ OP2_CMPXCHG = 0xB1,
</ins><span class="cx"> OP2_MOVZX_GvEb = 0xB6,
</span><span class="cx"> OP2_BSF = 0xBC,
</span><span class="cx"> OP2_TZCNT = 0xBC,
</span><span class="lines">@@ -298,6 +306,8 @@
</span><span class="cx"> OP2_MOVSX_GvEb = 0xBE,
</span><span class="cx"> OP2_MOVZX_GvEw = 0xB7,
</span><span class="cx"> OP2_MOVSX_GvEw = 0xBF,
</span><ins>+ OP2_XADDb = 0xC0,
+ OP2_XADD = 0xC1,
</ins><span class="cx"> OP2_PEXTRW_GdUdIb = 0xC5,
</span><span class="cx"> OP2_PSLLQ_UdqIb = 0x73,
</span><span class="cx"> OP2_PSRLQ_UdqIb = 0x73,
</span><span class="lines">@@ -440,6 +450,11 @@
</span><span class="cx"> m_formatter.oneByteOp(OP_ADD_GvEv, dst, base, offset);
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+ void addl_mr(int offset, RegisterID base, RegisterID index, int scale, RegisterID dst)
+ {
+ m_formatter.oneByteOp(OP_ADD_GvEv, dst, base, index, scale, offset);
+ }
+
</ins><span class="cx"> #if !CPU(X86_64)
</span><span class="cx"> void addl_mr(const void* addr, RegisterID dst)
</span><span class="cx"> {
</span><span class="lines">@@ -562,11 +577,21 @@
</span><span class="cx"> m_formatter.oneByteOp64(OP_ADD_GvEv, dst, base, offset);
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+ void addq_mr(int offset, RegisterID base, RegisterID index, int scale, RegisterID dst)
+ {
+ m_formatter.oneByteOp64(OP_ADD_GvEv, dst, base, index, scale, offset);
+ }
+
</ins><span class="cx"> void addq_rm(RegisterID src, int offset, RegisterID base)
</span><span class="cx"> {
</span><span class="cx"> m_formatter.oneByteOp64(OP_ADD_EvGv, src, base, offset);
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+ void addq_rm(RegisterID src, int offset, RegisterID base, RegisterID index, int scale)
+ {
+ m_formatter.oneByteOp64(OP_ADD_EvGv, src, base, index, scale, offset);
+ }
+
</ins><span class="cx"> void addq_ir(int imm, RegisterID dst)
</span><span class="cx"> {
</span><span class="cx"> if (CAN_SIGN_EXTEND_8_32(imm)) {
</span><span class="lines">@@ -591,6 +616,17 @@
</span><span class="cx"> m_formatter.immediate32(imm);
</span><span class="cx"> }
</span><span class="cx"> }
</span><ins>+
+ void addq_im(int imm, int offset, RegisterID base, RegisterID index, int scale)
+ {
+ if (CAN_SIGN_EXTEND_8_32(imm)) {
+ m_formatter.oneByteOp64(OP_GROUP1_EvIb, GROUP1_OP_ADD, base, index, scale, offset);
+ m_formatter.immediate8(imm);
+ } else {
+ m_formatter.oneByteOp64(OP_GROUP1_EvIz, GROUP1_OP_ADD, base, index, scale, offset);
+ m_formatter.immediate32(imm);
+ }
+ }
</ins><span class="cx"> #else
</span><span class="cx"> void addl_im(int imm, const void* addr)
</span><span class="cx"> {
</span><span class="lines">@@ -614,11 +650,43 @@
</span><span class="cx"> m_formatter.oneByteOp(OP_AND_GvEv, dst, base, offset);
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+ void andl_mr(int offset, RegisterID base, RegisterID index, int scale, RegisterID dst)
+ {
+ m_formatter.oneByteOp(OP_AND_GvEv, dst, base, index, scale, offset);
+ }
+
</ins><span class="cx"> void andl_rm(RegisterID src, int offset, RegisterID base)
</span><span class="cx"> {
</span><span class="cx"> m_formatter.oneByteOp(OP_AND_EvGv, src, base, offset);
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+ void andl_rm(RegisterID src, int offset, RegisterID base, RegisterID index, int scale)
+ {
+ m_formatter.oneByteOp(OP_AND_EvGv, src, base, index, scale, offset);
+ }
+
+ void andw_rm(RegisterID src, int offset, RegisterID base)
+ {
+ m_formatter.prefix(PRE_OPERAND_SIZE);
+ andl_rm(src, offset, base);
+ }
+
+ void andw_rm(RegisterID src, int offset, RegisterID base, RegisterID index, int scale)
+ {
+ m_formatter.prefix(PRE_OPERAND_SIZE);
+ andl_rm(src, offset, base, index, scale);
+ }
+
+ void andb_rm(RegisterID src, int offset, RegisterID base)
+ {
+ m_formatter.oneByteOp(OP_AND_EvGb, src, base, offset);
+ }
+
+ void andb_rm(RegisterID src, int offset, RegisterID base, RegisterID index, int scale)
+ {
+ m_formatter.oneByteOp(OP_AND_EvGb, src, base, index, scale, offset);
+ }
+
</ins><span class="cx"> void andl_ir(int imm, RegisterID dst)
</span><span class="cx"> {
</span><span class="cx"> if (CAN_SIGN_EXTEND_8_32(imm)) {
</span><span class="lines">@@ -641,6 +709,53 @@
</span><span class="cx"> }
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+ void andl_im(int imm, int offset, RegisterID base, RegisterID index, int scale)
+ {
+ if (CAN_SIGN_EXTEND_8_32(imm)) {
+ m_formatter.oneByteOp(OP_GROUP1_EvIb, GROUP1_OP_AND, base, index, scale, offset);
+ m_formatter.immediate8(imm);
+ } else {
+ m_formatter.oneByteOp(OP_GROUP1_EvIz, GROUP1_OP_AND, base, index, scale, offset);
+ m_formatter.immediate32(imm);
+ }
+ }
+
+ void andw_im(int imm, int offset, RegisterID base)
+ {
+ m_formatter.prefix(PRE_OPERAND_SIZE);
+ if (CAN_SIGN_EXTEND_8_32(imm)) {
+ m_formatter.oneByteOp(OP_GROUP1_EvIb, GROUP1_OP_AND, base, offset);
+ m_formatter.immediate8(imm);
+ } else {
+ m_formatter.oneByteOp(OP_GROUP1_EvIz, GROUP1_OP_AND, base, offset);
+ m_formatter.immediate16(imm);
+ }
+ }
+
+ void andw_im(int imm, int offset, RegisterID base, RegisterID index, int scale)
+ {
+ m_formatter.prefix(PRE_OPERAND_SIZE);
+ if (CAN_SIGN_EXTEND_8_32(imm)) {
+ m_formatter.oneByteOp(OP_GROUP1_EvIb, GROUP1_OP_AND, base, index, scale, offset);
+ m_formatter.immediate8(imm);
+ } else {
+ m_formatter.oneByteOp(OP_GROUP1_EvIz, GROUP1_OP_AND, base, index, scale, offset);
+ m_formatter.immediate16(imm);
+ }
+ }
+
+ void andb_im(int imm, int offset, RegisterID base)
+ {
+ m_formatter.oneByteOp(OP_GROUP1_EbIb, GROUP1_OP_AND, base, offset);
+ m_formatter.immediate8(imm);
+ }
+
+ void andb_im(int imm, int offset, RegisterID base, RegisterID index, int scale)
+ {
+ m_formatter.oneByteOp(OP_GROUP1_EbIb, GROUP1_OP_AND, base, index, scale, offset);
+ m_formatter.immediate8(imm);
+ }
+
</ins><span class="cx"> #if CPU(X86_64)
</span><span class="cx"> void andq_rr(RegisterID src, RegisterID dst)
</span><span class="cx"> {
</span><span class="lines">@@ -657,6 +772,48 @@
</span><span class="cx"> m_formatter.immediate32(imm);
</span><span class="cx"> }
</span><span class="cx"> }
</span><ins>+
+ void andq_mr(int offset, RegisterID base, RegisterID dst)
+ {
+ m_formatter.oneByteOp64(OP_AND_GvEv, dst, base, offset);
+ }
+
+ void andq_mr(int offset, RegisterID base, RegisterID index, int scale, RegisterID dst)
+ {
+ m_formatter.oneByteOp64(OP_AND_GvEv, dst, base, index, scale, offset);
+ }
+
+ void andq_rm(RegisterID src, int offset, RegisterID base)
+ {
+ m_formatter.oneByteOp64(OP_AND_EvGv, src, base, offset);
+ }
+
+ void andq_rm(RegisterID src, int offset, RegisterID base, RegisterID index, int scale)
+ {
+ m_formatter.oneByteOp64(OP_AND_EvGv, src, base, index, scale, offset);
+ }
+
+ void andq_im(int imm, int offset, RegisterID base)
+ {
+ if (CAN_SIGN_EXTEND_8_32(imm)) {
+ m_formatter.oneByteOp64(OP_GROUP1_EvIb, GROUP1_OP_AND, base, offset);
+ m_formatter.immediate8(imm);
+ } else {
+ m_formatter.oneByteOp64(OP_GROUP1_EvIz, GROUP1_OP_AND, base, offset);
+ m_formatter.immediate32(imm);
+ }
+ }
+
+ void andq_im(int imm, int offset, RegisterID base, RegisterID index, int scale)
+ {
+ if (CAN_SIGN_EXTEND_8_32(imm)) {
+ m_formatter.oneByteOp64(OP_GROUP1_EvIb, GROUP1_OP_AND, base, index, scale, offset);
+ m_formatter.immediate8(imm);
+ } else {
+ m_formatter.oneByteOp64(OP_GROUP1_EvIz, GROUP1_OP_AND, base, index, scale, offset);
+ m_formatter.immediate32(imm);
+ }
+ }
</ins><span class="cx"> #else
</span><span class="cx"> void andl_im(int imm, const void* addr)
</span><span class="cx"> {
</span><span class="lines">@@ -703,6 +860,11 @@
</span><span class="cx"> {
</span><span class="cx"> m_formatter.oneByteOp64(OP_GROUP5_Ev, GROUP1_OP_ADD, base, offset);
</span><span class="cx"> }
</span><ins>+
+ void incq_m(int offset, RegisterID base, RegisterID index, int scale)
+ {
+ m_formatter.oneByteOp64(OP_GROUP5_Ev, GROUP1_OP_ADD, base, index, scale, offset);
+ }
</ins><span class="cx"> #endif // CPU(X86_64)
</span><span class="cx">
</span><span class="cx"> void negl_r(RegisterID dst)
</span><span class="lines">@@ -715,6 +877,16 @@
</span><span class="cx"> {
</span><span class="cx"> m_formatter.oneByteOp64(OP_GROUP3_Ev, GROUP3_OP_NEG, dst);
</span><span class="cx"> }
</span><ins>+
+ void negq_m(int offset, RegisterID base)
+ {
+ m_formatter.oneByteOp64(OP_GROUP3_Ev, GROUP3_OP_NEG, base, offset);
+ }
+
+ void negq_m(int offset, RegisterID base, RegisterID index, int scale)
+ {
+ m_formatter.oneByteOp64(OP_GROUP3_Ev, GROUP3_OP_NEG, base, index, scale, offset);
+ }
</ins><span class="cx"> #endif
</span><span class="cx">
</span><span class="cx"> void negl_m(int offset, RegisterID base)
</span><span class="lines">@@ -722,6 +894,33 @@
</span><span class="cx"> m_formatter.oneByteOp(OP_GROUP3_Ev, GROUP3_OP_NEG, base, offset);
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+ void negl_m(int offset, RegisterID base, RegisterID index, int scale)
+ {
+ m_formatter.oneByteOp(OP_GROUP3_Ev, GROUP3_OP_NEG, base, index, scale, offset);
+ }
+
+ void negw_m(int offset, RegisterID base)
+ {
+ m_formatter.prefix(PRE_OPERAND_SIZE);
+ negl_m(offset, base);
+ }
+
+ void negw_m(int offset, RegisterID base, RegisterID index, int scale)
+ {
+ m_formatter.prefix(PRE_OPERAND_SIZE);
+ negl_m(offset, base, index, scale);
+ }
+
+ void negb_m(int offset, RegisterID base)
+ {
+ m_formatter.oneByteOp(OP_GROUP3_Eb, GROUP3_OP_NEG, base, offset);
+ }
+
+ void negb_m(int offset, RegisterID base, RegisterID index, int scale)
+ {
+ m_formatter.oneByteOp(OP_GROUP3_Eb, GROUP3_OP_NEG, base, index, scale, offset);
+ }
+
</ins><span class="cx"> void notl_r(RegisterID dst)
</span><span class="cx"> {
</span><span class="cx"> m_formatter.oneByteOp(OP_GROUP3_Ev, GROUP3_OP_NOT, dst);
</span><span class="lines">@@ -732,6 +931,33 @@
</span><span class="cx"> m_formatter.oneByteOp(OP_GROUP3_Ev, GROUP3_OP_NOT, base, offset);
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+ void notl_m(int offset, RegisterID base, RegisterID index, int scale)
+ {
+ m_formatter.oneByteOp(OP_GROUP3_Ev, GROUP3_OP_NOT, base, index, scale, offset);
+ }
+
+ void notw_m(int offset, RegisterID base)
+ {
+ m_formatter.prefix(PRE_OPERAND_SIZE);
+ notl_m(offset, base);
+ }
+
+ void notw_m(int offset, RegisterID base, RegisterID index, int scale)
+ {
+ m_formatter.prefix(PRE_OPERAND_SIZE);
+ notl_m(offset, base, index, scale);
+ }
+
+ void notb_m(int offset, RegisterID base)
+ {
+ m_formatter.oneByteOp(OP_GROUP3_Eb, GROUP3_OP_NOT, base, offset);
+ }
+
+ void notb_m(int offset, RegisterID base, RegisterID index, int scale)
+ {
+ m_formatter.oneByteOp(OP_GROUP3_Eb, GROUP3_OP_NOT, base, index, scale, offset);
+ }
+
</ins><span class="cx"> #if CPU(X86_64)
</span><span class="cx"> void notq_r(RegisterID dst)
</span><span class="cx"> {
</span><span class="lines">@@ -742,6 +968,11 @@
</span><span class="cx"> {
</span><span class="cx"> m_formatter.oneByteOp64(OP_GROUP3_Ev, GROUP3_OP_NOT, base, offset);
</span><span class="cx"> }
</span><ins>+
+ void notq_m(int offset, RegisterID base, RegisterID index, int scale)
+ {
+ m_formatter.oneByteOp64(OP_GROUP3_Ev, GROUP3_OP_NOT, base, index, scale, offset);
+ }
</ins><span class="cx"> #endif
</span><span class="cx">
</span><span class="cx"> void orl_rr(RegisterID src, RegisterID dst)
</span><span class="lines">@@ -754,11 +985,43 @@
</span><span class="cx"> m_formatter.oneByteOp(OP_OR_GvEv, dst, base, offset);
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+ void orl_mr(int offset, RegisterID base, RegisterID index, int scale, RegisterID dst)
+ {
+ m_formatter.oneByteOp(OP_OR_GvEv, dst, base, index, scale, offset);
+ }
+
</ins><span class="cx"> void orl_rm(RegisterID src, int offset, RegisterID base)
</span><span class="cx"> {
</span><span class="cx"> m_formatter.oneByteOp(OP_OR_EvGv, src, base, offset);
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+ void orl_rm(RegisterID src, int offset, RegisterID base, RegisterID index, int scale)
+ {
+ m_formatter.oneByteOp(OP_OR_EvGv, src, base, index, scale, offset);
+ }
+
+ void orw_rm(RegisterID src, int offset, RegisterID base)
+ {
+ m_formatter.prefix(PRE_OPERAND_SIZE);
+ orl_rm(src, offset, base);
+ }
+
+ void orw_rm(RegisterID src, int offset, RegisterID base, RegisterID index, int scale)
+ {
+ m_formatter.prefix(PRE_OPERAND_SIZE);
+ orl_rm(src, offset, base, index, scale);
+ }
+
+ void orb_rm(RegisterID src, int offset, RegisterID base)
+ {
+ m_formatter.oneByteOp(OP_OR_EvGb, src, base, offset);
+ }
+
+ void orb_rm(RegisterID src, int offset, RegisterID base, RegisterID index, int scale)
+ {
+ m_formatter.oneByteOp(OP_OR_EvGb, src, base, index, scale, offset);
+ }
+
</ins><span class="cx"> void orl_ir(int imm, RegisterID dst)
</span><span class="cx"> {
</span><span class="cx"> if (CAN_SIGN_EXTEND_8_32(imm)) {
</span><span class="lines">@@ -784,6 +1047,53 @@
</span><span class="cx"> }
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+ void orl_im(int imm, int offset, RegisterID base, RegisterID index, int scale)
+ {
+ if (CAN_SIGN_EXTEND_8_32(imm)) {
+ m_formatter.oneByteOp(OP_GROUP1_EvIb, GROUP1_OP_OR, base, index, scale, offset);
+ m_formatter.immediate8(imm);
+ } else {
+ m_formatter.oneByteOp(OP_GROUP1_EvIz, GROUP1_OP_OR, base, index, scale, offset);
+ m_formatter.immediate32(imm);
+ }
+ }
+
+ void orw_im(int imm, int offset, RegisterID base)
+ {
+ m_formatter.prefix(PRE_OPERAND_SIZE);
+ if (CAN_SIGN_EXTEND_8_32(imm)) {
+ m_formatter.oneByteOp(OP_GROUP1_EvIb, GROUP1_OP_OR, base, offset);
+ m_formatter.immediate8(imm);
+ } else {
+ m_formatter.oneByteOp(OP_GROUP1_EvIz, GROUP1_OP_OR, base, offset);
+ m_formatter.immediate16(imm);
+ }
+ }
+
+ void orw_im(int imm, int offset, RegisterID base, RegisterID index, int scale)
+ {
+ m_formatter.prefix(PRE_OPERAND_SIZE);
+ if (CAN_SIGN_EXTEND_8_32(imm)) {
+ m_formatter.oneByteOp(OP_GROUP1_EvIb, GROUP1_OP_OR, base, index, scale, offset);
+ m_formatter.immediate8(imm);
+ } else {
+ m_formatter.oneByteOp(OP_GROUP1_EvIz, GROUP1_OP_OR, base, index, scale, offset);
+ m_formatter.immediate16(imm);
+ }
+ }
+
+ void orb_im(int imm, int offset, RegisterID base)
+ {
+ m_formatter.oneByteOp(OP_GROUP1_EbIb, GROUP1_OP_OR, base, offset);
+ m_formatter.immediate8(imm);
+ }
+
+ void orb_im(int imm, int offset, RegisterID base, RegisterID index, int scale)
+ {
+ m_formatter.oneByteOp(OP_GROUP1_EbIb, GROUP1_OP_OR, base, index, scale, offset);
+ m_formatter.immediate8(imm);
+ }
+
</ins><span class="cx"> #if CPU(X86_64)
</span><span class="cx"> void orq_rr(RegisterID src, RegisterID dst)
</span><span class="cx"> {
</span><span class="lines">@@ -790,6 +1100,48 @@
</span><span class="cx"> m_formatter.oneByteOp64(OP_OR_EvGv, src, dst);
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+ void orq_mr(int offset, RegisterID base, RegisterID dst)
+ {
+ m_formatter.oneByteOp64(OP_OR_GvEv, dst, base, offset);
+ }
+
+ void orq_mr(int offset, RegisterID base, RegisterID index, int scale, RegisterID dst)
+ {
+ m_formatter.oneByteOp64(OP_OR_GvEv, dst, base, index, scale, offset);
+ }
+
+ void orq_rm(RegisterID src, int offset, RegisterID base)
+ {
+ m_formatter.oneByteOp64(OP_OR_EvGv, src, base, offset);
+ }
+
+ void orq_rm(RegisterID src, int offset, RegisterID base, RegisterID index, int scale)
+ {
+ m_formatter.oneByteOp64(OP_OR_EvGv, src, base, index, scale, offset);
+ }
+
+ void orq_im(int imm, int offset, RegisterID base)
+ {
+ if (CAN_SIGN_EXTEND_8_32(imm)) {
+ m_formatter.oneByteOp64(OP_GROUP1_EvIb, GROUP1_OP_OR, base, offset);
+ m_formatter.immediate8(imm);
+ } else {
+ m_formatter.oneByteOp64(OP_GROUP1_EvIz, GROUP1_OP_OR, base, offset);
+ m_formatter.immediate32(imm);
+ }
+ }
+
+ void orq_im(int imm, int offset, RegisterID base, RegisterID index, int scale)
+ {
+ if (CAN_SIGN_EXTEND_8_32(imm)) {
+ m_formatter.oneByteOp64(OP_GROUP1_EvIb, GROUP1_OP_OR, base, index, scale, offset);
+ m_formatter.immediate8(imm);
+ } else {
+ m_formatter.oneByteOp64(OP_GROUP1_EvIz, GROUP1_OP_OR, base, index, scale, offset);
+ m_formatter.immediate32(imm);
+ }
+ }
+
</ins><span class="cx"> void orq_ir(int imm, RegisterID dst)
</span><span class="cx"> {
</span><span class="cx"> if (CAN_SIGN_EXTEND_8_32(imm)) {
</span><span class="lines">@@ -831,11 +1183,43 @@
</span><span class="cx"> m_formatter.oneByteOp(OP_SUB_GvEv, dst, base, offset);
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+ void subl_mr(int offset, RegisterID base, RegisterID index, int scale, RegisterID dst)
+ {
+ m_formatter.oneByteOp(OP_SUB_GvEv, dst, base, index, scale, offset);
+ }
+
</ins><span class="cx"> void subl_rm(RegisterID src, int offset, RegisterID base)
</span><span class="cx"> {
</span><span class="cx"> m_formatter.oneByteOp(OP_SUB_EvGv, src, base, offset);
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+ void subl_rm(RegisterID src, int offset, RegisterID base, RegisterID index, int scale)
+ {
+ m_formatter.oneByteOp(OP_SUB_EvGv, src, base, index, scale, offset);
+ }
+
+ void subw_rm(RegisterID src, int offset, RegisterID base)
+ {
+ m_formatter.prefix(PRE_OPERAND_SIZE);
+ m_formatter.oneByteOp(OP_SUB_EvGv, src, base, offset);
+ }
+
+ void subw_rm(RegisterID src, int offset, RegisterID base, RegisterID index, int scale)
+ {
+ m_formatter.prefix(PRE_OPERAND_SIZE);
+ m_formatter.oneByteOp(OP_SUB_EvGv, src, base, index, scale, offset);
+ }
+
+ void subb_rm(RegisterID src, int offset, RegisterID base)
+ {
+ m_formatter.oneByteOp(OP_SUB_EvGb, src, base, offset);
+ }
+
+ void subb_rm(RegisterID src, int offset, RegisterID base, RegisterID index, int scale)
+ {
+ m_formatter.oneByteOp(OP_SUB_EvGb, src, base, index, scale, offset);
+ }
+
</ins><span class="cx"> void subl_ir(int imm, RegisterID dst)
</span><span class="cx"> {
</span><span class="cx"> if (CAN_SIGN_EXTEND_8_32(imm)) {
</span><span class="lines">@@ -861,6 +1245,53 @@
</span><span class="cx"> }
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+ void subl_im(int imm, int offset, RegisterID base, RegisterID index, int scale)
+ {
+ if (CAN_SIGN_EXTEND_8_32(imm)) {
+ m_formatter.oneByteOp(OP_GROUP1_EvIb, GROUP1_OP_SUB, base, index, scale, offset);
+ m_formatter.immediate8(imm);
+ } else {
+ m_formatter.oneByteOp(OP_GROUP1_EvIz, GROUP1_OP_SUB, base, index, scale, offset);
+ m_formatter.immediate32(imm);
+ }
+ }
+
+ void subw_im(int imm, int offset, RegisterID base)
+ {
+ m_formatter.prefix(PRE_OPERAND_SIZE);
+ if (CAN_SIGN_EXTEND_8_32(imm)) {
+ m_formatter.oneByteOp(OP_GROUP1_EvIb, GROUP1_OP_SUB, base, offset);
+ m_formatter.immediate8(imm);
+ } else {
+ m_formatter.oneByteOp(OP_GROUP1_EvIz, GROUP1_OP_SUB, base, offset);
+ m_formatter.immediate16(imm);
+ }
+ }
+
+ void subw_im(int imm, int offset, RegisterID base, RegisterID index, int scale)
+ {
+ m_formatter.prefix(PRE_OPERAND_SIZE);
+ if (CAN_SIGN_EXTEND_8_32(imm)) {
+ m_formatter.oneByteOp(OP_GROUP1_EvIb, GROUP1_OP_SUB, base, index, scale, offset);
+ m_formatter.immediate8(imm);
+ } else {
+ m_formatter.oneByteOp(OP_GROUP1_EvIz, GROUP1_OP_SUB, base, index, scale, offset);
+ m_formatter.immediate16(imm);
+ }
+ }
+
+ void subb_im(int imm, int offset, RegisterID base)
+ {
+ m_formatter.oneByteOp(OP_GROUP1_EbIb, GROUP1_OP_SUB, base, offset);
+ m_formatter.immediate8(imm);
+ }
+
+ void subb_im(int imm, int offset, RegisterID base, RegisterID index, int scale)
+ {
+ m_formatter.oneByteOp(OP_GROUP1_EbIb, GROUP1_OP_SUB, base, index, scale, offset);
+ m_formatter.immediate8(imm);
+ }
+
</ins><span class="cx"> #if CPU(X86_64)
</span><span class="cx"> void subq_rr(RegisterID src, RegisterID dst)
</span><span class="cx"> {
</span><span class="lines">@@ -872,11 +1303,21 @@
</span><span class="cx"> m_formatter.oneByteOp64(OP_SUB_GvEv, dst, base, offset);
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+ void subq_mr(int offset, RegisterID base, RegisterID index, int scale, RegisterID dst)
+ {
+ m_formatter.oneByteOp64(OP_SUB_GvEv, dst, base, index, scale, offset);
+ }
+
</ins><span class="cx"> void subq_rm(RegisterID src, int offset, RegisterID base)
</span><span class="cx"> {
</span><span class="cx"> m_formatter.oneByteOp64(OP_SUB_EvGv, src, base, offset);
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+ void subq_rm(RegisterID src, int offset, RegisterID base, RegisterID index, int scale)
+ {
+ m_formatter.oneByteOp64(OP_SUB_EvGv, src, base, index, scale, offset);
+ }
+
</ins><span class="cx"> void subq_ir(int imm, RegisterID dst)
</span><span class="cx"> {
</span><span class="cx"> if (CAN_SIGN_EXTEND_8_32(imm)) {
</span><span class="lines">@@ -901,6 +1342,17 @@
</span><span class="cx"> m_formatter.immediate32(imm);
</span><span class="cx"> }
</span><span class="cx"> }
</span><ins>+
+ void subq_im(int imm, int offset, RegisterID base, RegisterID index, int scale)
+ {
+ if (CAN_SIGN_EXTEND_8_32(imm)) {
+ m_formatter.oneByteOp64(OP_GROUP1_EvIb, GROUP1_OP_SUB, base, index, scale, offset);
+ m_formatter.immediate8(imm);
+ } else {
+ m_formatter.oneByteOp64(OP_GROUP1_EvIz, GROUP1_OP_SUB, base, index, scale, offset);
+ m_formatter.immediate32(imm);
+ }
+ }
</ins><span class="cx"> #else
</span><span class="cx"> void subl_im(int imm, const void* addr)
</span><span class="cx"> {
</span><span class="lines">@@ -924,11 +1376,21 @@
</span><span class="cx"> m_formatter.oneByteOp(OP_XOR_GvEv, dst, base, offset);
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+ void xorl_mr(int offset, RegisterID base, RegisterID index, int scale, RegisterID dst)
+ {
+ m_formatter.oneByteOp(OP_XOR_GvEv, dst, base, index, scale, offset);
+ }
+
</ins><span class="cx"> void xorl_rm(RegisterID src, int offset, RegisterID base)
</span><span class="cx"> {
</span><span class="cx"> m_formatter.oneByteOp(OP_XOR_EvGv, src, base, offset);
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+ void xorl_rm(RegisterID src, int offset, RegisterID base, RegisterID index, int scale)
+ {
+ m_formatter.oneByteOp(OP_XOR_EvGv, src, base, index, scale, offset);
+ }
+
</ins><span class="cx"> void xorl_im(int imm, int offset, RegisterID base)
</span><span class="cx"> {
</span><span class="cx"> if (CAN_SIGN_EXTEND_8_32(imm)) {
</span><span class="lines">@@ -940,6 +1402,75 @@
</span><span class="cx"> }
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+ void xorl_im(int imm, int offset, RegisterID base, RegisterID index, int scale)
+ {
+ if (CAN_SIGN_EXTEND_8_32(imm)) {
+ m_formatter.oneByteOp(OP_GROUP1_EvIb, GROUP1_OP_XOR, base, index, scale, offset);
+ m_formatter.immediate8(imm);
+ } else {
+ m_formatter.oneByteOp(OP_GROUP1_EvIz, GROUP1_OP_XOR, base, index, scale, offset);
+ m_formatter.immediate32(imm);
+ }
+ }
+
+ void xorw_rm(RegisterID src, int offset, RegisterID base)
+ {
+ m_formatter.prefix(PRE_OPERAND_SIZE);
+ xorl_rm(src, offset, base);
+ }
+
+ void xorw_rm(RegisterID src, int offset, RegisterID base, RegisterID index, int scale)
+ {
+ m_formatter.prefix(PRE_OPERAND_SIZE);
+ xorl_rm(src, offset, base, index, scale);
+ }
+
+ void xorw_im(int imm, int offset, RegisterID base)
+ {
+ m_formatter.prefix(PRE_OPERAND_SIZE);
+ if (CAN_SIGN_EXTEND_8_32(imm)) {
+ m_formatter.oneByteOp(OP_GROUP1_EvIb, GROUP1_OP_XOR, base, offset);
+ m_formatter.immediate8(imm);
+ } else {
+ m_formatter.oneByteOp(OP_GROUP1_EvIz, GROUP1_OP_XOR, base, offset);
+ m_formatter.immediate16(imm);
+ }
+ }
+
+ void xorw_im(int imm, int offset, RegisterID base, RegisterID index, int scale)
+ {
+ m_formatter.prefix(PRE_OPERAND_SIZE);
+ if (CAN_SIGN_EXTEND_8_32(imm)) {
+ m_formatter.oneByteOp(OP_GROUP1_EvIb, GROUP1_OP_XOR, base, index, scale, offset);
+ m_formatter.immediate8(imm);
+ } else {
+ m_formatter.oneByteOp(OP_GROUP1_EvIz, GROUP1_OP_XOR, base, index, scale, offset);
+ m_formatter.immediate16(imm);
+ }
+ }
+
+ void xorb_rm(RegisterID src, int offset, RegisterID base)
+ {
+ m_formatter.oneByteOp(OP_XOR_EvGb, src, base, offset);
+ }
+
+ void xorb_rm(RegisterID src, int offset, RegisterID base, RegisterID index, int scale)
+ {
+ m_formatter.oneByteOp(OP_XOR_EvGb, src, base, index, scale, offset);
+ }
+
+ void xorb_im(int imm, int offset, RegisterID base)
+ {
+ m_formatter.oneByteOp(OP_GROUP1_EbIb, GROUP1_OP_XOR, base, offset);
+ m_formatter.immediate8(imm);
+ }
+
+ void xorb_im(int imm, int offset, RegisterID base, RegisterID index, int scale)
+ {
+ m_formatter.oneByteOp(OP_GROUP1_EbIb, GROUP1_OP_XOR, base, index, scale, offset);
+ m_formatter.immediate8(imm);
+ }
+
</ins><span class="cx"> void xorl_ir(int imm, RegisterID dst)
</span><span class="cx"> {
</span><span class="cx"> if (CAN_SIGN_EXTEND_8_32(imm)) {
</span><span class="lines">@@ -974,11 +1505,47 @@
</span><span class="cx"> }
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+ void xorq_im(int imm, int offset, RegisterID base)
+ {
+ if (CAN_SIGN_EXTEND_8_32(imm)) {
+ m_formatter.oneByteOp64(OP_GROUP1_EvIb, GROUP1_OP_XOR, base, offset);
+ m_formatter.immediate8(imm);
+ } else {
+ m_formatter.oneByteOp64(OP_GROUP1_EvIz, GROUP1_OP_XOR, base, offset);
+ m_formatter.immediate32(imm);
+ }
+ }
+
+ void xorq_im(int imm, int offset, RegisterID base, RegisterID index, int scale)
+ {
+ if (CAN_SIGN_EXTEND_8_32(imm)) {
+ m_formatter.oneByteOp64(OP_GROUP1_EvIb, GROUP1_OP_XOR, base, index, scale, offset);
+ m_formatter.immediate8(imm);
+ } else {
+ m_formatter.oneByteOp64(OP_GROUP1_EvIz, GROUP1_OP_XOR, base, index, scale, offset);
+ m_formatter.immediate32(imm);
+ }
+ }
+
</ins><span class="cx"> void xorq_rm(RegisterID src, int offset, RegisterID base)
</span><span class="cx"> {
</span><span class="cx"> m_formatter.oneByteOp64(OP_XOR_EvGv, src, base, offset);
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+ void xorq_rm(RegisterID src, int offset, RegisterID base, RegisterID index, int scale)
+ {
+ m_formatter.oneByteOp64(OP_XOR_EvGv, src, base, index, scale, offset);
+ }
+
+ void xorq_mr(int offset, RegisterID base, RegisterID dest)
+ {
+ m_formatter.oneByteOp64(OP_XOR_GvEv, dest, base, offset);
+ }
+
+ void xorq_mr(int offset, RegisterID base, RegisterID index, int scale, RegisterID dest)
+ {
+ m_formatter.oneByteOp64(OP_XOR_GvEv, dest, base, index, scale, offset);
+ }
</ins><span class="cx"> #endif
</span><span class="cx">
</span><span class="cx"> void lzcnt_rr(RegisterID src, RegisterID dst)
</span><span class="lines">@@ -1586,11 +2153,38 @@
</span><span class="cx"> m_formatter.oneByteOp(OP_XCHG_EvGv, src, dst);
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+ void xchgb_rm(RegisterID src, int offset, RegisterID base)
+ {
+ m_formatter.oneByteOp(OP_XCHG_EvGb, src, base, offset);
+ }
+
+ void xchgb_rm(RegisterID src, int offset, RegisterID base, RegisterID index, int scale)
+ {
+ m_formatter.oneByteOp(OP_XCHG_EvGb, src, base, index, scale, offset);
+ }
+
+ void xchgw_rm(RegisterID src, int offset, RegisterID base)
+ {
+ m_formatter.prefix(PRE_OPERAND_SIZE);
+ m_formatter.oneByteOp(OP_XCHG_EvGv, src, base, offset);
+ }
+
+ void xchgw_rm(RegisterID src, int offset, RegisterID base, RegisterID index, int scale)
+ {
+ m_formatter.prefix(PRE_OPERAND_SIZE);
+ m_formatter.oneByteOp(OP_XCHG_EvGv, src, base, index, scale, offset);
+ }
+
</ins><span class="cx"> void xchgl_rm(RegisterID src, int offset, RegisterID base)
</span><span class="cx"> {
</span><span class="cx"> m_formatter.oneByteOp(OP_XCHG_EvGv, src, base, offset);
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+ void xchgl_rm(RegisterID src, int offset, RegisterID base, RegisterID index, int scale)
+ {
+ m_formatter.oneByteOp(OP_XCHG_EvGv, src, base, index, scale, offset);
+ }
+
</ins><span class="cx"> #if CPU(X86_64)
</span><span class="cx"> void xchgq_rr(RegisterID src, RegisterID dst)
</span><span class="cx"> {
</span><span class="lines">@@ -1606,6 +2200,11 @@
</span><span class="cx"> {
</span><span class="cx"> m_formatter.oneByteOp64(OP_XCHG_EvGv, src, base, offset);
</span><span class="cx"> }
</span><ins>+
+ void xchgq_rm(RegisterID src, int offset, RegisterID base, RegisterID index, int scale)
+ {
+ m_formatter.oneByteOp64(OP_XCHG_EvGv, src, base, index, scale, offset);
+ }
</ins><span class="cx"> #endif
</span><span class="cx">
</span><span class="cx"> void movl_rr(RegisterID src, RegisterID dst)
</span><span class="lines">@@ -1731,6 +2330,20 @@
</span><span class="cx"> m_formatter.oneByteOp8(OP_MOV_EvGv, src, base, index, scale, offset);
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+ void movw_im(int imm, int offset, RegisterID base)
+ {
+ m_formatter.prefix(PRE_OPERAND_SIZE);
+ m_formatter.oneByteOp(OP_GROUP11_EvIz, GROUP11_MOV, base, offset);
+ m_formatter.immediate16(imm);
+ }
+
+ void movw_im(int imm, int offset, RegisterID base, RegisterID index, int scale)
+ {
+ m_formatter.prefix(PRE_OPERAND_SIZE);
+ m_formatter.oneByteOp(OP_GROUP11_EvIz, GROUP11_MOV, base, index, scale, offset);
+ m_formatter.immediate16(imm);
+ }
+
</ins><span class="cx"> void movl_EAXm(const void* addr)
</span><span class="cx"> {
</span><span class="cx"> m_formatter.oneByteOp(OP_MOV_OvEAX);
</span><span class="lines">@@ -1800,6 +2413,12 @@
</span><span class="cx"> m_formatter.immediate32(imm);
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+ void movq_i32m(int imm, int offset, RegisterID base, RegisterID index, int scale)
+ {
+ m_formatter.oneByteOp64(OP_GROUP11_EvIz, GROUP11_MOV, base, index, scale, offset);
+ m_formatter.immediate32(imm);
+ }
+
</ins><span class="cx"> void movq_i64r(int64_t imm, RegisterID dst)
</span><span class="cx"> {
</span><span class="cx"> m_formatter.oneByteOp64(OP_MOV_EAXIv, dst);
</span><span class="lines">@@ -2781,6 +3400,94 @@
</span><span class="cx"> m_formatter.prefix(PRE_LOCK);
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+ void cmpxchgb_rm(RegisterID src, int offset, RegisterID base)
+ {
+ m_formatter.twoByteOp8(OP2_CMPXCHGb, src, base, offset);
+ }
+
+ void cmpxchgb_rm(RegisterID src, int offset, RegisterID base, RegisterID index, int scale)
+ {
+ m_formatter.twoByteOp8(OP2_CMPXCHGb, src, base, index, scale, offset);
+ }
+
+ void cmpxchgw_rm(RegisterID src, int offset, RegisterID base)
+ {
+ m_formatter.prefix(PRE_OPERAND_SIZE);
+ m_formatter.twoByteOp(OP2_CMPXCHG, src, base, offset);
+ }
+
+ void cmpxchgw_rm(RegisterID src, int offset, RegisterID base, RegisterID index, int scale)
+ {
+ m_formatter.prefix(PRE_OPERAND_SIZE);
+ m_formatter.twoByteOp(OP2_CMPXCHG, src, base, index, scale, offset);
+ }
+
+ void cmpxchgl_rm(RegisterID src, int offset, RegisterID base)
+ {
+ m_formatter.twoByteOp(OP2_CMPXCHG, src, base, offset);
+ }
+
+ void cmpxchgl_rm(RegisterID src, int offset, RegisterID base, RegisterID index, int scale)
+ {
+ m_formatter.twoByteOp(OP2_CMPXCHG, src, base, index, scale, offset);
+ }
+
+#if CPU(X86_64)
+ void cmpxchgq_rm(RegisterID src, int offset, RegisterID base)
+ {
+ m_formatter.twoByteOp64(OP2_CMPXCHG, src, base, offset);
+ }
+
+ void cmpxchgq_rm(RegisterID src, int offset, RegisterID base, RegisterID index, int scale)
+ {
+ m_formatter.twoByteOp64(OP2_CMPXCHG, src, base, index, scale, offset);
+ }
+#endif // CPU(X86_64)
+
+ void xaddb_rm(RegisterID src, int offset, RegisterID base)
+ {
+ m_formatter.twoByteOp8(OP2_XADDb, src, base, offset);
+ }
+
+ void xaddb_rm(RegisterID src, int offset, RegisterID base, RegisterID index, int scale)
+ {
+ m_formatter.twoByteOp8(OP2_XADDb, src, base, index, scale, offset);
+ }
+
+ void xaddw_rm(RegisterID src, int offset, RegisterID base)
+ {
+ m_formatter.prefix(PRE_OPERAND_SIZE);
+ m_formatter.twoByteOp(OP2_XADD, src, base, offset);
+ }
+
+ void xaddw_rm(RegisterID src, int offset, RegisterID base, RegisterID index, int scale)
+ {
+ m_formatter.prefix(PRE_OPERAND_SIZE);
+ m_formatter.twoByteOp(OP2_XADD, src, base, index, scale, offset);
+ }
+
+ void xaddl_rm(RegisterID src, int offset, RegisterID base)
+ {
+ m_formatter.twoByteOp(OP2_XADD, src, base, offset);
+ }
+
+ void xaddl_rm(RegisterID src, int offset, RegisterID base, RegisterID index, int scale)
+ {
+ m_formatter.twoByteOp(OP2_XADD, src, base, index, scale, offset);
+ }
+
+#if CPU(X86_64)
+ void xaddq_rm(RegisterID src, int offset, RegisterID base)
+ {
+ m_formatter.twoByteOp64(OP2_XADD, src, base, offset);
+ }
+
+ void xaddq_rm(RegisterID src, int offset, RegisterID base, RegisterID index, int scale)
+ {
+ m_formatter.twoByteOp64(OP2_XADD, src, base, index, scale, offset);
+ }
+#endif // CPU(X86_64)
+
</ins><span class="cx"> void mfence()
</span><span class="cx"> {
</span><span class="cx"> m_formatter.threeByteOp(OP2_3BYTE_ESCAPE_AE, OP3_MFENCE);
</span><span class="lines">@@ -3763,6 +4470,24 @@
</span><span class="cx"> writer.registerModRM(groupOp, rm);
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+ void twoByteOp8(TwoByteOpcodeID opcode, int reg, RegisterID base, int offset)
+ {
+ SingleInstructionBufferWriter writer(m_buffer);
+ writer.emitRexIf(byteRegRequiresRex(reg, base), reg, 0, base);
+ writer.putByteUnchecked(OP_2BYTE_ESCAPE);
+ writer.putByteUnchecked(opcode);
+ writer.memoryModRM(reg, base, offset);
+ }
+
+ void twoByteOp8(TwoByteOpcodeID opcode, int reg, RegisterID base, RegisterID index, int scale, int offset)
+ {
+ SingleInstructionBufferWriter writer(m_buffer);
+ writer.emitRexIf(byteRegRequiresRex(reg) || regRequiresRex(index, base), reg, index, base);
+ writer.putByteUnchecked(OP_2BYTE_ESCAPE);
+ writer.putByteUnchecked(opcode);
+ writer.memoryModRM(reg, base, index, scale, offset);
+ }
+
</ins><span class="cx"> // Immediates:
</span><span class="cx"> //
</span><span class="cx"> // An immedaite should be appended where appropriate after an op has been emitted.
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreb3B3AtomicValuecpp"></a>
<div class="addfile"><h4>Added: trunk/Source/JavaScriptCore/b3/B3AtomicValue.cpp (0 => 213714)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/b3/B3AtomicValue.cpp (rev 0)
+++ trunk/Source/JavaScriptCore/b3/B3AtomicValue.cpp 2017-03-10 17:49:42 UTC (rev 213714)
</span><span class="lines">@@ -0,0 +1,86 @@
</span><ins>+/*
+ * Copyright (C) 2017 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+ * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+ * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+ * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include "config.h"
+#include "B3AtomicValue.h"
+
+#if ENABLE(B3_JIT)
+
+namespace JSC { namespace B3 {
+
+AtomicValue::~AtomicValue()
+{
+}
+
+void AtomicValue::dumpMeta(CommaPrinter& comma, PrintStream& out) const
+{
+ out.print(comma, "width = ", m_width);
+
+ MemoryValue::dumpMeta(comma, out);
+}
+
+Value* AtomicValue::cloneImpl() const
+{
+ return new AtomicValue(*this);
+}
+
+AtomicValue::AtomicValue(Kind kind, Origin origin, Width width, Value* operand, Value* pointer, int32_t offset, HeapRange range, HeapRange fenceRange)
+ : MemoryValue(CheckedOpcode, kind, operand->type(), origin, offset, range, fenceRange, operand, pointer)
+ , m_width(width)
+{
+ ASSERT(bestType(GP, accessWidth()) == accessType());
+
+ switch (kind.opcode()) {
+ case AtomicXchgAdd:
+ case AtomicXchgAnd:
+ case AtomicXchgOr:
+ case AtomicXchgSub:
+ case AtomicXchgXor:
+ case AtomicXchg:
+ break;
+ default:
+ ASSERT_NOT_REACHED();
+ }
+}
+
+AtomicValue::AtomicValue(Kind kind, Origin origin, Width width, Value* expectedValue, Value* newValue, Value* pointer, int32_t offset, HeapRange range, HeapRange fenceRange)
+ : MemoryValue(CheckedOpcode, kind, kind.opcode() == AtomicWeakCAS ? Int32 : expectedValue->type(), origin, offset, range, fenceRange, expectedValue, newValue, pointer)
+ , m_width(width)
+{
+ ASSERT(bestType(GP, accessWidth()) == accessType());
+
+ switch (kind.opcode()) {
+ case AtomicWeakCAS:
+ case AtomicStrongCAS:
+ break;
+ default:
+ ASSERT_NOT_REACHED();
+ }
+}
+
+} } // namespace JSC::B3
+
+#endif // ENABLE(B3_JIT)
+
</ins></span></pre></div>
<a id="trunkSourceJavaScriptCoreb3B3AtomicValueh"></a>
<div class="addfile"><h4>Added: trunk/Source/JavaScriptCore/b3/B3AtomicValue.h (0 => 213714)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/b3/B3AtomicValue.h (rev 0)
+++ trunk/Source/JavaScriptCore/b3/B3AtomicValue.h 2017-03-10 17:49:42 UTC (rev 213714)
</span><span class="lines">@@ -0,0 +1,66 @@
</span><ins>+/*
+ * Copyright (C) 2017 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+ * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+ * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+ * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#pragma once
+
+#if ENABLE(B3_JIT)
+
+#include "B3MemoryValue.h"
+#include "B3Width.h"
+
+namespace JSC { namespace B3 {
+
+class JS_EXPORT_PRIVATE AtomicValue : public MemoryValue {
+public:
+ static bool accepts(Kind kind)
+ {
+ return isAtomic(kind.opcode());
+ }
+
+ ~AtomicValue();
+
+ Type accessType() const { return child(0)->type(); }
+
+ Width accessWidth() const { return m_width; }
+
+protected:
+ void dumpMeta(CommaPrinter&, PrintStream&) const override;
+
+ Value* cloneImpl() const override;
+
+private:
+ friend class Procedure;
+
+ AtomicValue(Kind, Origin, Width, Value* operand, Value* pointer, int32_t offset = 0, HeapRange range = HeapRange::top(), HeapRange fenceRange = HeapRange::top());
+
+ AtomicValue(Kind, Origin, Width, Value* expectedValue, Value* newValue, Value* pointer, int32_t offset = 0, HeapRange range = HeapRange::top(), HeapRange fenceRange = HeapRange::top());
+
+ Width m_width;
+};
+
+} } // namespace JSC::B3
+
+#endif // ENABLE(B3_JIT)
+
</ins></span></pre></div>
<a id="trunkSourceJavaScriptCoreb3B3BasicBlockh"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/b3/B3BasicBlock.h (213713 => 213714)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/b3/B3BasicBlock.h 2017-03-10 17:32:07 UTC (rev 213713)
+++ trunk/Source/JavaScriptCore/b3/B3BasicBlock.h 2017-03-10 17:49:42 UTC (rev 213714)
</span><span class="lines">@@ -1,5 +1,5 @@
</span><span class="cx"> /*
</span><del>- * Copyright (C) 2015-2016 Apple Inc. All rights reserved.
</del><ins>+ * Copyright (C) 2015-2017 Apple Inc. All rights reserved.
</ins><span class="cx"> *
</span><span class="cx"> * Redistribution and use in source and binary forms, with or without
</span><span class="cx"> * modification, are permitted provided that the following conditions
</span><span class="lines">@@ -40,6 +40,7 @@
</span><span class="cx"> class InsertionSet;
</span><span class="cx"> class Procedure;
</span><span class="cx"> class Value;
</span><ins>+template<typename> class GenericBlockInsertionSet;
</ins><span class="cx">
</span><span class="cx"> class BasicBlock {
</span><span class="cx"> WTF_MAKE_NONCOPYABLE(BasicBlock);
</span><span class="lines">@@ -158,6 +159,7 @@
</span><span class="cx"> friend class BlockInsertionSet;
</span><span class="cx"> friend class InsertionSet;
</span><span class="cx"> friend class Procedure;
</span><ins>+ template<typename> friend class GenericBlockInsertionSet;
</ins><span class="cx">
</span><span class="cx"> // Instantiate via Procedure.
</span><span class="cx"> BasicBlock(unsigned index, double frequency);
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreb3B3BlockInsertionSetcpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/b3/B3BlockInsertionSet.cpp (213713 => 213714)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/b3/B3BlockInsertionSet.cpp 2017-03-10 17:32:07 UTC (rev 213713)
+++ trunk/Source/JavaScriptCore/b3/B3BlockInsertionSet.cpp 2017-03-10 17:49:42 UTC (rev 213714)
</span><span class="lines">@@ -1,5 +1,5 @@
</span><span class="cx"> /*
</span><del>- * Copyright (C) 2015 Apple Inc. All rights reserved.
</del><ins>+ * Copyright (C) 2015-2017 Apple Inc. All rights reserved.
</ins><span class="cx"> *
</span><span class="cx"> * Redistribution and use in source and binary forms, with or without
</span><span class="cx"> * modification, are permitted provided that the following conditions
</span><span class="lines">@@ -36,35 +36,13 @@
</span><span class="cx"> namespace JSC { namespace B3 {
</span><span class="cx">
</span><span class="cx"> BlockInsertionSet::BlockInsertionSet(Procedure &proc)
</span><del>- : m_proc(proc)
</del><ins>+ : GenericBlockInsertionSet(proc.m_blocks)
+ , m_proc(proc)
</ins><span class="cx"> {
</span><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> BlockInsertionSet::~BlockInsertionSet() { }
</span><span class="cx">
</span><del>-void BlockInsertionSet::insert(BlockInsertion&& insertion)
-{
- m_insertions.append(WTFMove(insertion));
-}
-
-BasicBlock* BlockInsertionSet::insert(unsigned index, double frequency)
-{
- std::unique_ptr<BasicBlock> block(new BasicBlock(UINT_MAX, frequency));
- BasicBlock* result = block.get();
- insert(BlockInsertion(index, WTFMove(block)));
- return result;
-}
-
-BasicBlock* BlockInsertionSet::insertBefore(BasicBlock* before, double frequency)
-{
- return insert(before->index(), frequency == frequency ? frequency : before->frequency());
-}
-
-BasicBlock* BlockInsertionSet::insertAfter(BasicBlock* after, double frequency)
-{
- return insert(after->index() + 1, frequency == frequency ? frequency : after->frequency());
-}
-
</del><span class="cx"> BasicBlock* BlockInsertionSet::splitForward(
</span><span class="cx"> BasicBlock* block, unsigned& valueIndex, InsertionSet* insertionSet, double frequency)
</span><span class="cx"> {
</span><span class="lines">@@ -102,32 +80,6 @@
</span><span class="cx"> return result;
</span><span class="cx"> }
</span><span class="cx">
</span><del>-bool BlockInsertionSet::execute()
-{
- if (m_insertions.isEmpty())
- return false;
-
- // We allow insertions to be given to us in any order. So, we need to sort them before
- // running WTF::executeInsertions. We strongly prefer a stable sort and we want it to be
- // fast, so we use bubble sort.
- bubbleSort(m_insertions.begin(), m_insertions.end());
-
- executeInsertions(m_proc.m_blocks, m_insertions);
-
- // Prune out empty entries. This isn't strictly necessary but it's
- // healthy to keep the block list from growing.
- m_proc.m_blocks.removeAllMatching(
- [&] (std::unique_ptr<BasicBlock>& blockPtr) -> bool {
- return !blockPtr;
- });
-
- // Make sure that the blocks know their new indices.
- for (unsigned i = 0; i < m_proc.m_blocks.size(); ++i)
- m_proc.m_blocks[i]->m_index = i;
-
- return true;
-}
-
</del><span class="cx"> } } // namespace JSC::B3
</span><span class="cx">
</span><span class="cx"> #endif // ENABLE(B3_JIT)
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreb3B3BlockInsertionSeth"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/b3/B3BlockInsertionSet.h (213713 => 213714)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/b3/B3BlockInsertionSet.h 2017-03-10 17:32:07 UTC (rev 213713)
+++ trunk/Source/JavaScriptCore/b3/B3BlockInsertionSet.h 2017-03-10 17:49:42 UTC (rev 213714)
</span><span class="lines">@@ -1,5 +1,5 @@
</span><span class="cx"> /*
</span><del>- * Copyright (C) 2015-2016 Apple Inc. All rights reserved.
</del><ins>+ * Copyright (C) 2015-2017 Apple Inc. All rights reserved.
</ins><span class="cx"> *
</span><span class="cx"> * Redistribution and use in source and binary forms, with or without
</span><span class="cx"> * modification, are permitted provided that the following conditions
</span><span class="lines">@@ -27,6 +27,7 @@
</span><span class="cx">
</span><span class="cx"> #if ENABLE(B3_JIT)
</span><span class="cx">
</span><ins>+#include "B3GenericBlockInsertionSet.h"
</ins><span class="cx"> #include "B3Procedure.h"
</span><span class="cx"> #include <wtf/Insertion.h>
</span><span class="cx"> #include <wtf/Vector.h>
</span><span class="lines">@@ -35,26 +36,13 @@
</span><span class="cx">
</span><span class="cx"> class InsertionSet;
</span><span class="cx">
</span><del>-typedef WTF::Insertion<std::unique_ptr<BasicBlock>> BlockInsertion;
</del><ins>+typedef GenericBlockInsertionSet<BasicBlock>::BlockInsertion BlockInsertion;
</ins><span class="cx">
</span><del>-class BlockInsertionSet {
</del><ins>+class BlockInsertionSet : public GenericBlockInsertionSet<BasicBlock> {
</ins><span class="cx"> public:
</span><span class="cx"> BlockInsertionSet(Procedure&);
</span><span class="cx"> ~BlockInsertionSet();
</span><span class="cx">
</span><del>- void insert(BlockInsertion&&);
-
- // Insert a new block at a given index.
- BasicBlock* insert(unsigned index, double frequency = PNaN);
-
- // Inserts a new block before the given block. Usually you will not pass the frequency
- // argument. Passing PNaN causes us to just use the frequency of the 'before' block. That's
- // usually what you want.
- BasicBlock* insertBefore(BasicBlock* before, double frequency = PNaN);
-
- // Inserts a new block after the given block.
- BasicBlock* insertAfter(BasicBlock* after, double frequency = PNaN);
-
</del><span class="cx"> // A helper to split a block when forward iterating over it. It creates a new block to hold
</span><span class="cx"> // everything before the instruction at valueIndex. The current block is left with
</span><span class="cx"> // everything at and after valueIndex. If the optional InsertionSet is provided, it will get
</span><span class="lines">@@ -80,12 +68,9 @@
</span><span class="cx"> BasicBlock* splitForward(
</span><span class="cx"> BasicBlock*, unsigned& valueIndex, InsertionSet* = nullptr,
</span><span class="cx"> double frequency = PNaN);
</span><del>-
- bool execute();
</del><span class="cx">
</span><span class="cx"> private:
</span><span class="cx"> Procedure& m_proc;
</span><del>- Vector<BlockInsertion, 8> m_insertions;
</del><span class="cx"> };
</span><span class="cx">
</span><span class="cx"> } } // namespace JSC::B3
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreb3B3Effectscpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/b3/B3Effects.cpp (213713 => 213714)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/b3/B3Effects.cpp 2017-03-10 17:32:07 UTC (rev 213713)
+++ trunk/Source/JavaScriptCore/b3/B3Effects.cpp 2017-03-10 17:49:42 UTC (rev 213714)
</span><span class="lines">@@ -1,5 +1,5 @@
</span><span class="cx"> /*
</span><del>- * Copyright (C) 2015-2016 Apple Inc. All rights reserved.
</del><ins>+ * Copyright (C) 2015-2017 Apple Inc. All rights reserved.
</ins><span class="cx"> *
</span><span class="cx"> * Redistribution and use in source and binary forms, with or without
</span><span class="cx"> * modification, are permitted provided that the following conditions
</span><span class="lines">@@ -82,7 +82,8 @@
</span><span class="cx"> || interferesWithWritesPinned(other, *this)
</span><span class="cx"> || writes.overlaps(other.writes)
</span><span class="cx"> || writes.overlaps(other.reads)
</span><del>- || reads.overlaps(other.writes);
</del><ins>+ || reads.overlaps(other.writes)
+ || (fence && other.fence);
</ins><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> bool Effects::operator==(const Effects& other) const
</span><span class="lines">@@ -95,7 +96,8 @@
</span><span class="cx"> && writesPinned == other.writesPinned
</span><span class="cx"> && readsPinned == other.readsPinned
</span><span class="cx"> && writes == other.writes
</span><del>- && reads == other.reads;
</del><ins>+ && reads == other.reads
+ && fence == other.fence;
</ins><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> bool Effects::operator!=(const Effects& other) const
</span><span class="lines">@@ -120,6 +122,8 @@
</span><span class="cx"> out.print(comma, "WritesPinned");
</span><span class="cx"> if (readsPinned)
</span><span class="cx"> out.print(comma, "ReadsPinned");
</span><ins>+ if (fence)
+ out.print(comma, "Fence");
</ins><span class="cx"> if (writes)
</span><span class="cx"> out.print(comma, "Writes:", writes);
</span><span class="cx"> if (reads)
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreb3B3Effectsh"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/b3/B3Effects.h (213713 => 213714)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/b3/B3Effects.h 2017-03-10 17:32:07 UTC (rev 213713)
+++ trunk/Source/JavaScriptCore/b3/B3Effects.h 2017-03-10 17:49:42 UTC (rev 213714)
</span><span class="lines">@@ -1,5 +1,5 @@
</span><span class="cx"> /*
</span><del>- * Copyright (C) 2015-2016 Apple Inc. All rights reserved.
</del><ins>+ * Copyright (C) 2015-2017 Apple Inc. All rights reserved.
</ins><span class="cx"> *
</span><span class="cx"> * Redistribution and use in source and binary forms, with or without
</span><span class="cx"> * modification, are permitted provided that the following conditions
</span><span class="lines">@@ -71,6 +71,10 @@
</span><span class="cx"> // https://bugs.webkit.org/show_bug.cgi?id=163173
</span><span class="cx"> bool readsPinned { false };
</span><span class="cx"> bool writesPinned { false };
</span><ins>+
+ // Memory fences cannot be reordered around each other regardless of their effects. This is flagged
+ // if the operation is a memory fence.
+ bool fence { false };
</ins><span class="cx">
</span><span class="cx"> HeapRange writes;
</span><span class="cx"> HeapRange reads;
</span><span class="lines">@@ -89,6 +93,7 @@
</span><span class="cx"> result.reads = HeapRange::top();
</span><span class="cx"> result.readsPinned = true;
</span><span class="cx"> result.writesPinned = true;
</span><ins>+ result.fence = true;
</ins><span class="cx"> return result;
</span><span class="cx"> }
</span><span class="cx">
</span><span class="lines">@@ -103,7 +108,7 @@
</span><span class="cx">
</span><span class="cx"> bool mustExecute() const
</span><span class="cx"> {
</span><del>- return terminal || exitsSideways || writesLocalState || writes || writesPinned;
</del><ins>+ return terminal || exitsSideways || writesLocalState || writes || writesPinned || fence;
</ins><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> // Returns true if reordering instructions with these respective effects would change program
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreb3B3EliminateCommonSubexpressionscpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/b3/B3EliminateCommonSubexpressions.cpp (213713 => 213714)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/b3/B3EliminateCommonSubexpressions.cpp 2017-03-10 17:32:07 UTC (rev 213713)
+++ trunk/Source/JavaScriptCore/b3/B3EliminateCommonSubexpressions.cpp 2017-03-10 17:49:42 UTC (rev 213714)
</span><span class="lines">@@ -1,5 +1,5 @@
</span><span class="cx"> /*
</span><del>- * Copyright (C) 2016 Apple Inc. All rights reserved.
</del><ins>+ * Copyright (C) 2016-2017 Apple Inc. All rights reserved.
</ins><span class="cx"> *
</span><span class="cx"> * Redistribution and use in source and binary forms, with or without
</span><span class="cx"> * modification, are permitted provided that the following conditions
</span><span class="lines">@@ -142,6 +142,7 @@
</span><span class="cx">
</span><span class="cx"> RangeSet<HeapRange> reads; // This only gets used for forward store elimination.
</span><span class="cx"> RangeSet<HeapRange> writes; // This gets used for both load and store elimination.
</span><ins>+ bool fence;
</ins><span class="cx">
</span><span class="cx"> MemoryValueMap storesAtHead;
</span><span class="cx"> MemoryValueMap memoryValuesAtTail;
</span><span class="lines">@@ -174,12 +175,14 @@
</span><span class="cx">
</span><span class="cx"> if (memory && memory->isStore()
</span><span class="cx"> && !data.reads.overlaps(memory->range())
</span><del>- && !data.writes.overlaps(memory->range()))
</del><ins>+ && !data.writes.overlaps(memory->range())
+ && (!data.fence || !memory->hasFence()))
</ins><span class="cx"> data.storesAtHead.add(memory);
</span><span class="cx"> data.reads.add(effects.reads);
</span><span class="cx">
</span><span class="cx"> if (HeapRange writes = effects.writes)
</span><span class="cx"> clobber(data, writes);
</span><ins>+ data.fence |= effects.fence;
</ins><span class="cx">
</span><span class="cx"> if (memory)
</span><span class="cx"> data.memoryValuesAtTail.add(memory);
</span><span class="lines">@@ -445,8 +448,6 @@
</span><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> default:
</span><del>- dataLog("Bad memory value: ", deepDump(m_proc, m_value), "\n");
- RELEASE_ASSERT_NOT_REACHED();
</del><span class="cx"> break;
</span><span class="cx"> }
</span><span class="cx"> }
</span><span class="lines">@@ -478,6 +479,9 @@
</span><span class="cx"> template<typename Filter>
</span><span class="cx"> bool findStoreAfterClobber(Value* ptr, HeapRange range, const Filter& filter)
</span><span class="cx"> {
</span><ins>+ if (m_value->as<MemoryValue>()->hasFence())
+ return false;
+
</ins><span class="cx"> // We can eliminate a store if every forward path hits a store to the same location before
</span><span class="cx"> // hitting any operation that observes the store. This search seems like it should be
</span><span class="cx"> // expensive, but in the overwhelming majority of cases it will almost immediately hit an
</span><span class="lines">@@ -614,6 +618,12 @@
</span><span class="cx"> if (verbose)
</span><span class="cx"> dataLog(*m_value, ": looking backward for ", *ptr, "...\n");
</span><span class="cx">
</span><ins>+ if (m_value->as<MemoryValue>()->hasFence()) {
+ if (verbose)
+ dataLog(" Giving up because fences.\n");
+ return { };
+ }
+
</ins><span class="cx"> if (MemoryValue* match = m_data.memoryValuesAtTail.find(ptr, filter)) {
</span><span class="cx"> if (verbose)
</span><span class="cx"> dataLog(" Found ", *match, " locally.\n");
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreb3B3Generatecpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/b3/B3Generate.cpp (213713 => 213714)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/b3/B3Generate.cpp 2017-03-10 17:32:07 UTC (rev 213713)
+++ trunk/Source/JavaScriptCore/b3/B3Generate.cpp 2017-03-10 17:49:42 UTC (rev 213714)
</span><span class="lines">@@ -1,5 +1,5 @@
</span><span class="cx"> /*
</span><del>- * Copyright (C) 2015-2016 Apple Inc. All rights reserved.
</del><ins>+ * Copyright (C) 2015-2017 Apple Inc. All rights reserved.
</ins><span class="cx"> *
</span><span class="cx"> * Redistribution and use in source and binary forms, with or without
</span><span class="cx"> * modification, are permitted provided that the following conditions
</span><span class="lines">@@ -43,6 +43,7 @@
</span><span class="cx"> #include "B3LowerToAir.h"
</span><span class="cx"> #include "B3MoveConstants.h"
</span><span class="cx"> #include "B3Procedure.h"
</span><ins>+#include "B3PureCSE.h"
</ins><span class="cx"> #include "B3ReduceDoubleToFloat.h"
</span><span class="cx"> #include "B3ReduceStrength.h"
</span><span class="cx"> #include "B3TimingScope.h"
</span><span class="lines">@@ -92,6 +93,7 @@
</span><span class="cx"> // https://bugs.webkit.org/show_bug.cgi?id=150507
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+ // This puts the IR in quirks mode.
</ins><span class="cx"> lowerMacros(procedure);
</span><span class="cx">
</span><span class="cx"> if (optLevel >= 1) {
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreb3B3GenericBlockInsertionSeth"></a>
<div class="addfile"><h4>Added: trunk/Source/JavaScriptCore/b3/B3GenericBlockInsertionSet.h (0 => 213714)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/b3/B3GenericBlockInsertionSet.h (rev 0)
+++ trunk/Source/JavaScriptCore/b3/B3GenericBlockInsertionSet.h 2017-03-10 17:49:42 UTC (rev 213714)
</span><span class="lines">@@ -0,0 +1,111 @@
</span><ins>+/*
+ * Copyright (C) 2017 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+ * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+ * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+ * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#pragma once
+
+#if ENABLE(B3_JIT)
+
+#include "PureNaN.h"
+#include <climits>
+#include <wtf/BubbleSort.h>
+#include <wtf/Insertion.h>
+#include <wtf/Vector.h>
+
+namespace JSC { namespace B3 {
+
+class InsertionSet;
+
+template<typename BasicBlock>
+class GenericBlockInsertionSet {
+public:
+ typedef WTF::Insertion<std::unique_ptr<BasicBlock>> BlockInsertion;
+
+ GenericBlockInsertionSet(Vector<std::unique_ptr<BasicBlock>>& blocks)
+ : m_blocks(blocks)
+ {
+ }
+
+ void insert(BlockInsertion&& insertion)
+ {
+ m_insertions.append(WTFMove(insertion));
+ }
+
+ // Insert a new block at a given index.
+ BasicBlock* insert(unsigned index, double frequency = PNaN)
+ {
+ std::unique_ptr<BasicBlock> block(new BasicBlock(UINT_MAX, frequency));
+ BasicBlock* result = block.get();
+ insert(BlockInsertion(index, WTFMove(block)));
+ return result;
+ }
+
+ // Inserts a new block before the given block. Usually you will not pass the frequency
+ // argument. Passing PNaN causes us to just use the frequency of the 'before' block. That's
+ // usually what you want.
+ BasicBlock* insertBefore(BasicBlock* before, double frequency = PNaN)
+ {
+ return insert(before->index(), frequency == frequency ? frequency : before->frequency());
+ }
+
+ // Inserts a new block after the given block.
+ BasicBlock* insertAfter(BasicBlock* after, double frequency = PNaN)
+ {
+ return insert(after->index() + 1, frequency == frequency ? frequency : after->frequency());
+ }
+
+ bool execute()
+ {
+ if (m_insertions.isEmpty())
+ return false;
+
+ // We allow insertions to be given to us in any order. So, we need to sort them before
+ // running WTF::executeInsertions. We strongly prefer a stable sort and we want it to be
+ // fast, so we use bubble sort.
+ bubbleSort(m_insertions.begin(), m_insertions.end());
+
+ executeInsertions(m_blocks, m_insertions);
+
+ // Prune out empty entries. This isn't strictly necessary but it's
+ // healthy to keep the block list from growing.
+ m_blocks.removeAllMatching(
+ [&] (std::unique_ptr<BasicBlock>& blockPtr) -> bool {
+ return !blockPtr;
+ });
+
+ // Make sure that the blocks know their new indices.
+ for (unsigned i = 0; i < m_blocks.size(); ++i)
+ m_blocks[i]->m_index = i;
+
+ return true;
+ }
+
+private:
+ Vector<std::unique_ptr<BasicBlock>>& m_blocks;
+ Vector<BlockInsertion, 8> m_insertions;
+};
+
+} } // namespace JSC::B3
+
+#endif // ENABLE(B3_JIT)
</ins></span></pre></div>
<a id="trunkSourceJavaScriptCoreb3B3HeapRangeh"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/b3/B3HeapRange.h (213713 => 213714)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/b3/B3HeapRange.h 2017-03-10 17:32:07 UTC (rev 213713)
+++ trunk/Source/JavaScriptCore/b3/B3HeapRange.h 2017-03-10 17:49:42 UTC (rev 213714)
</span><span class="lines">@@ -1,5 +1,5 @@
</span><span class="cx"> /*
</span><del>- * Copyright (C) 2015 Apple Inc. All rights reserved.
</del><ins>+ * Copyright (C) 2015-2017 Apple Inc. All rights reserved.
</ins><span class="cx"> *
</span><span class="cx"> * Redistribution and use in source and binary forms, with or without
</span><span class="cx"> * modification, are permitted provided that the following conditions
</span><span class="lines">@@ -88,6 +88,13 @@
</span><span class="cx"> return !(*this == other);
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+ HeapRange operator|(const HeapRange& other) const
+ {
+ return HeapRange(
+ std::min(m_begin, other.m_begin),
+ std::max(m_end, other.m_end));
+ }
+
</ins><span class="cx"> explicit operator bool() const { return m_begin != m_end; }
</span><span class="cx">
</span><span class="cx"> unsigned begin() const { return m_begin; }
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreb3B3InsertionSetcpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/b3/B3InsertionSet.cpp (213713 => 213714)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/b3/B3InsertionSet.cpp 2017-03-10 17:32:07 UTC (rev 213713)
+++ trunk/Source/JavaScriptCore/b3/B3InsertionSet.cpp 2017-03-10 17:49:42 UTC (rev 213714)
</span><span class="lines">@@ -1,5 +1,5 @@
</span><span class="cx"> /*
</span><del>- * Copyright (C) 2015-2016 Apple Inc. All rights reserved.
</del><ins>+ * Copyright (C) 2015-2017 Apple Inc. All rights reserved.
</ins><span class="cx"> *
</span><span class="cx"> * Redistribution and use in source and binary forms, with or without
</span><span class="cx"> * modification, are permitted provided that the following conditions
</span><span class="lines">@@ -58,6 +58,11 @@
</span><span class="cx"> return insertBottom(index, likeValue->origin(), likeValue->type());
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+Value* InsertionSet::insertClone(size_t index, Value* value)
+{
+ return insertValue(index, m_procedure.clone(value));
+}
+
</ins><span class="cx"> void InsertionSet::execute(BasicBlock* block)
</span><span class="cx"> {
</span><span class="cx"> bubbleSort(m_insertions.begin(), m_insertions.end());
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreb3B3InsertionSeth"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/b3/B3InsertionSet.h (213713 => 213714)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/b3/B3InsertionSet.h 2017-03-10 17:32:07 UTC (rev 213713)
+++ trunk/Source/JavaScriptCore/b3/B3InsertionSet.h 2017-03-10 17:49:42 UTC (rev 213714)
</span><span class="lines">@@ -1,5 +1,5 @@
</span><span class="cx"> /*
</span><del>- * Copyright (C) 2015-2016 Apple Inc. All rights reserved.
</del><ins>+ * Copyright (C) 2015-2017 Apple Inc. All rights reserved.
</ins><span class="cx"> *
</span><span class="cx"> * Redistribution and use in source and binary forms, with or without
</span><span class="cx"> * modification, are permitted provided that the following conditions
</span><span class="lines">@@ -71,6 +71,8 @@
</span><span class="cx">
</span><span class="cx"> Value* insertBottom(size_t index, Origin, Type);
</span><span class="cx"> Value* insertBottom(size_t index, Value*);
</span><ins>+
+ Value* insertClone(size_t index, Value*);
</ins><span class="cx">
</span><span class="cx"> void execute(BasicBlock*);
</span><span class="cx">
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreb3B3LegalizeMemoryOffsetscpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/b3/B3LegalizeMemoryOffsets.cpp (213713 => 213714)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/b3/B3LegalizeMemoryOffsets.cpp 2017-03-10 17:32:07 UTC (rev 213713)
+++ trunk/Source/JavaScriptCore/b3/B3LegalizeMemoryOffsets.cpp 2017-03-10 17:49:42 UTC (rev 213714)
</span><span class="lines">@@ -28,9 +28,8 @@
</span><span class="cx">
</span><span class="cx"> #if ENABLE(B3_JIT)
</span><span class="cx">
</span><del>-#include "AirArg.h"
</del><span class="cx"> #include "B3InsertionSetInlines.h"
</span><del>-#include "B3MemoryValue.h"
</del><ins>+#include "B3MemoryValueInlines.h"
</ins><span class="cx"> #include "B3PhaseScope.h"
</span><span class="cx"> #include "B3ProcedureInlines.h"
</span><span class="cx"> #include "B3ValueInlines.h"
</span><span class="lines">@@ -49,20 +48,21 @@
</span><span class="cx">
</span><span class="cx"> void run()
</span><span class="cx"> {
</span><del>- if (!isARM64())
- return;
-
</del><ins>+ // FIXME: Perhaps this should be moved to lowerMacros, and quirks mode can impose the requirement
+ // that the offset is legal. But for now this is sort of OK because we run pureCSE after. Also,
+ // we should probably have something better than just pureCSE to clean up the code that this
+ // introduces.
+ // https://bugs.webkit.org/show_bug.cgi?id=169246
+
</ins><span class="cx"> for (BasicBlock* block : m_proc) {
</span><span class="cx"> for (unsigned index = 0; index < block->size(); ++index) {
</span><span class="cx"> MemoryValue* memoryValue = block->at(index)->as<MemoryValue>();
</span><span class="cx"> if (!memoryValue)
</span><span class="cx"> continue;
</span><del>-
- int32_t offset = memoryValue->offset();
- Width width = memoryValue->accessWidth();
- if (!Air::Arg::isValidAddrForm(offset, width)) {
</del><ins>+
+ if (!memoryValue->isLegalOffset(memoryValue->offset())) {
</ins><span class="cx"> Value* base = memoryValue->lastChild();
</span><del>- Value* offsetValue = m_insertionSet.insertIntConstant(index, memoryValue->origin(), pointerType(), offset);
</del><ins>+ Value* offsetValue = m_insertionSet.insertIntConstant(index, memoryValue->origin(), pointerType(), memoryValue->offset());
</ins><span class="cx"> Value* resolvedAddress = m_proc.add<Value>(Add, memoryValue->origin(), base, offsetValue);
</span><span class="cx"> m_insertionSet.insertValue(index, resolvedAddress);
</span><span class="cx">
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreb3B3LowerMacroscpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/b3/B3LowerMacros.cpp (213713 => 213714)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/b3/B3LowerMacros.cpp 2017-03-10 17:32:07 UTC (rev 213713)
+++ trunk/Source/JavaScriptCore/b3/B3LowerMacros.cpp 2017-03-10 17:49:42 UTC (rev 213714)
</span><span class="lines">@@ -1,5 +1,5 @@
</span><span class="cx"> /*
</span><del>- * Copyright (C) 2015-2016 Apple Inc. All rights reserved.
</del><ins>+ * Copyright (C) 2015-2017 Apple Inc. All rights reserved.
</ins><span class="cx"> *
</span><span class="cx"> * Redistribution and use in source and binary forms, with or without
</span><span class="cx"> * modification, are permitted provided that the following conditions
</span><span class="lines">@@ -29,13 +29,15 @@
</span><span class="cx"> #if ENABLE(B3_JIT)
</span><span class="cx">
</span><span class="cx"> #include "AllowMacroScratchRegisterUsage.h"
</span><ins>+#include "B3AtomicValue.h"
</ins><span class="cx"> #include "B3BasicBlockInlines.h"
</span><span class="cx"> #include "B3BlockInsertionSet.h"
</span><span class="cx"> #include "B3CCallValue.h"
</span><span class="cx"> #include "B3CaseCollectionInlines.h"
</span><span class="cx"> #include "B3ConstPtrValue.h"
</span><ins>+#include "B3FenceValue.h"
</ins><span class="cx"> #include "B3InsertionSetInlines.h"
</span><del>-#include "B3MemoryValue.h"
</del><ins>+#include "B3MemoryValueInlines.h"
</ins><span class="cx"> #include "B3PatchpointValue.h"
</span><span class="cx"> #include "B3PhaseScope.h"
</span><span class="cx"> #include "B3ProcedureInlines.h"
</span><span class="lines">@@ -42,6 +44,7 @@
</span><span class="cx"> #include "B3StackmapGenerationParams.h"
</span><span class="cx"> #include "B3SwitchValue.h"
</span><span class="cx"> #include "B3UpsilonValue.h"
</span><ins>+#include "B3UseCounts.h"
</ins><span class="cx"> #include "B3ValueInlines.h"
</span><span class="cx"> #include "CCallHelpers.h"
</span><span class="cx"> #include "LinkBuffer.h"
</span><span class="lines">@@ -58,11 +61,14 @@
</span><span class="cx"> : m_proc(proc)
</span><span class="cx"> , m_blockInsertionSet(proc)
</span><span class="cx"> , m_insertionSet(proc)
</span><ins>+ , m_useCounts(proc)
</ins><span class="cx"> {
</span><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> bool run()
</span><span class="cx"> {
</span><ins>+ RELEASE_ASSERT(!m_proc.hasQuirks());
+
</ins><span class="cx"> for (BasicBlock* block : m_proc) {
</span><span class="cx"> m_block = block;
</span><span class="cx"> processCurrentBlock();
</span><span class="lines">@@ -72,6 +78,10 @@
</span><span class="cx"> m_proc.resetReachability();
</span><span class="cx"> m_proc.invalidateCFG();
</span><span class="cx"> }
</span><ins>+
+ // This indicates that we've
+ m_proc.setHasQuirks(true);
+
</ins><span class="cx"> return m_changed;
</span><span class="cx"> }
</span><span class="cx">
</span><span class="lines">@@ -185,7 +195,127 @@
</span><span class="cx"> m_changed = true;
</span><span class="cx"> break;
</span><span class="cx"> }
</span><ins>+
+ case Depend: {
+ if (isX86()) {
+ // Create a load-load fence. This codegens to nothing on X86. We use it to tell the
+ // compiler not to block load motion.
+ FenceValue* fence = m_insertionSet.insert<FenceValue>(m_index, m_origin);
+ fence->read = HeapRange();
+ fence->write = HeapRange::top();
+
+ // Kill the Depend, which should unlock a bunch of code simplification.
+ m_value->replaceWithBottom(m_insertionSet, m_index);
+
+ m_changed = true;
+ }
+ break;
+ }
</ins><span class="cx">
</span><ins>+ case AtomicWeakCAS:
+ case AtomicStrongCAS: {
+ AtomicValue* atomic = m_value->as<AtomicValue>();
+ Width width = atomic->accessWidth();
+
+ if (isCanonicalWidth(width))
+ break;
+
+ Value* expectedValue = atomic->child(0);
+
+ if (!isX86()) {
+ // On ARM, the load part of the CAS does a load with zero extension. Therefore, we need
+ // to zero-extend the input.
+ Value* maskedExpectedValue = m_insertionSet.insert<Value>(
+ m_index, BitAnd, m_origin, expectedValue,
+ m_insertionSet.insertIntConstant(m_index, expectedValue, mask(width)));
+
+ atomic->child(0) = maskedExpectedValue;
+ }
+
+ if (atomic->opcode() == AtomicStrongCAS) {
+ Value* newValue = m_insertionSet.insert<Value>(
+ m_index, signExtendOpcode(width), m_origin,
+ m_insertionSet.insertClone(m_index, atomic));
+
+ atomic->replaceWithIdentity(newValue);
+ }
+
+ m_changed = true;
+ break;
+ }
+
+ case AtomicXchgAdd:
+ case AtomicXchgAnd:
+ case AtomicXchgOr:
+ case AtomicXchgSub:
+ case AtomicXchgXor:
+ case AtomicXchg: {
+ // On X86, these may actually return garbage in the high bits. On ARM64, these sorta
+ // zero-extend their high bits, except that the high bits might get polluted by high
+ // bits in the operand. So, either way, we need to throw a sign-extend on these
+ // things.
+
+ if (isX86()) {
+ if (m_value->opcode() == AtomicXchgSub && m_useCounts.numUses(m_value)) {
+ // On x86, xchgadd is better than xchgsub if it has any users.
+ m_value->setOpcodeUnsafely(AtomicXchgAdd);
+ m_value->child(0) = m_insertionSet.insert<Value>(
+ m_index, Neg, m_origin, m_value->child(0));
+ }
+
+ bool exempt = false;
+ switch (m_value->opcode()) {
+ case AtomicXchgAnd:
+ case AtomicXchgOr:
+ case AtomicXchgSub:
+ case AtomicXchgXor:
+ exempt = true;
+ break;
+ default:
+ break;
+ }
+ if (exempt)
+ break;
+ }
+
+ AtomicValue* atomic = m_value->as<AtomicValue>();
+ Width width = atomic->accessWidth();
+
+ if (isCanonicalWidth(width))
+ break;
+
+ Value* newValue = m_insertionSet.insert<Value>(
+ m_index, signExtendOpcode(width), m_origin,
+ m_insertionSet.insertClone(m_index, atomic));
+
+ atomic->replaceWithIdentity(newValue);
+ m_changed = true;
+ break;
+ }
+
+ case Load8Z:
+ case Load16Z: {
+ if (isX86())
+ break;
+
+ MemoryValue* memory = m_value->as<MemoryValue>();
+ if (!memory->hasFence())
+ break;
+
+ // Sub-width load-acq on ARM64 always sign extends.
+ Value* newLoad = m_insertionSet.insertClone(m_index, memory);
+ newLoad->setOpcodeUnsafely(memory->opcode() == Load8Z ? Load8S : Load16S);
+
+ Value* newValue = m_insertionSet.insert<Value>(
+ m_index, BitAnd, m_origin, newLoad,
+ m_insertionSet.insertIntConstant(
+ m_index, m_origin, Int32, mask(memory->accessWidth())));
+
+ m_value->replaceWithIdentity(newValue);
+ m_changed = true;
+ break;
+ }
+
</ins><span class="cx"> default:
</span><span class="cx"> break;
</span><span class="cx"> }
</span><span class="lines">@@ -470,6 +600,7 @@
</span><span class="cx"> Procedure& m_proc;
</span><span class="cx"> BlockInsertionSet m_blockInsertionSet;
</span><span class="cx"> InsertionSet m_insertionSet;
</span><ins>+ UseCounts m_useCounts;
</ins><span class="cx"> BasicBlock* m_block;
</span><span class="cx"> unsigned m_index;
</span><span class="cx"> Value* m_value;
</span><span class="lines">@@ -477,21 +608,12 @@
</span><span class="cx"> bool m_changed { false };
</span><span class="cx"> };
</span><span class="cx">
</span><del>-bool lowerMacrosImpl(Procedure& proc)
-{
- LowerMacros lowerMacros(proc);
- return lowerMacros.run();
-}
-
</del><span class="cx"> } // anonymous namespace
</span><span class="cx">
</span><span class="cx"> bool lowerMacros(Procedure& proc)
</span><span class="cx"> {
</span><del>- PhaseScope phaseScope(proc, "lowerMacros");
- bool result = lowerMacrosImpl(proc);
- if (shouldValidateIR())
- RELEASE_ASSERT(!lowerMacrosImpl(proc));
- return result;
</del><ins>+ LowerMacros lowerMacros(proc);
+ return lowerMacros.run();
</ins><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> } } // namespace JSC::B3
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreb3B3LowerMacrosAfterOptimizationscpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/b3/B3LowerMacrosAfterOptimizations.cpp (213713 => 213714)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/b3/B3LowerMacrosAfterOptimizations.cpp 2017-03-10 17:32:07 UTC (rev 213713)
+++ trunk/Source/JavaScriptCore/b3/B3LowerMacrosAfterOptimizations.cpp 2017-03-10 17:49:42 UTC (rev 213714)
</span><span class="lines">@@ -1,5 +1,5 @@
</span><span class="cx"> /*
</span><del>- * Copyright (C) 2015-2016 Apple Inc. All rights reserved.
</del><ins>+ * Copyright (C) 2015-2017 Apple Inc. All rights reserved.
</ins><span class="cx"> *
</span><span class="cx"> * Redistribution and use in source and binary forms, with or without
</span><span class="cx"> * modification, are permitted provided that the following conditions
</span><span class="lines">@@ -156,15 +156,14 @@
</span><span class="cx"> case RotL: {
</span><span class="cx"> // ARM64 doesn't have a rotate left.
</span><span class="cx"> if (isARM64()) {
</span><del>- if (isARM64()) {
- Value* newShift = m_insertionSet.insert<Value>(m_index, Neg, m_value->origin(), m_value->child(1));
- Value* rotate = m_insertionSet.insert<Value>(m_index, RotR, m_value->origin(), m_value->child(0), newShift);
- m_value->replaceWithIdentity(rotate);
- break;
- }
</del><ins>+ Value* newShift = m_insertionSet.insert<Value>(m_index, Neg, m_value->origin(), m_value->child(1));
+ Value* rotate = m_insertionSet.insert<Value>(m_index, RotR, m_value->origin(), m_value->child(0), newShift);
+ m_value->replaceWithIdentity(rotate);
+ break;
</ins><span class="cx"> }
</span><span class="cx"> break;
</span><span class="cx"> }
</span><ins>+
</ins><span class="cx"> default:
</span><span class="cx"> break;
</span><span class="cx"> }
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreb3B3LowerToAircpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/b3/B3LowerToAir.cpp (213713 => 213714)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/b3/B3LowerToAir.cpp 2017-03-10 17:32:07 UTC (rev 213713)
+++ trunk/Source/JavaScriptCore/b3/B3LowerToAir.cpp 2017-03-10 17:49:42 UTC (rev 213714)
</span><span class="lines">@@ -28,6 +28,7 @@
</span><span class="cx">
</span><span class="cx"> #if ENABLE(B3_JIT)
</span><span class="cx">
</span><ins>+#include "AirBlockInsertionSet.h"
</ins><span class="cx"> #include "AirCCallSpecial.h"
</span><span class="cx"> #include "AirCode.h"
</span><span class="cx"> #include "AirInsertionSet.h"
</span><span class="lines">@@ -34,6 +35,7 @@
</span><span class="cx"> #include "AirInstInlines.h"
</span><span class="cx"> #include "AirStackSlot.h"
</span><span class="cx"> #include "B3ArgumentRegValue.h"
</span><ins>+#include "B3AtomicValue.h"
</ins><span class="cx"> #include "B3BasicBlockInlines.h"
</span><span class="cx"> #include "B3BlockWorklist.h"
</span><span class="cx"> #include "B3CCallValue.h"
</span><span class="lines">@@ -41,7 +43,7 @@
</span><span class="cx"> #include "B3Commutativity.h"
</span><span class="cx"> #include "B3Dominators.h"
</span><span class="cx"> #include "B3FenceValue.h"
</span><del>-#include "B3MemoryValue.h"
</del><ins>+#include "B3MemoryValueInlines.h"
</ins><span class="cx"> #include "B3PatchpointSpecial.h"
</span><span class="cx"> #include "B3PatchpointValue.h"
</span><span class="cx"> #include "B3PhaseScope.h"
</span><span class="lines">@@ -72,6 +74,16 @@
</span><span class="cx">
</span><span class="cx"> const bool verbose = false;
</span><span class="cx">
</span><ins>+// FIXME: We wouldn't need this if Air supported Width modifiers in Air::Kind.
+// https://bugs.webkit.org/show_bug.cgi?id=169247
+#define OPCODE_FOR_WIDTH(opcode, width) ( \
+ (width) == Width8 ? opcode ## 8 : \
+ (width) == Width16 ? opcode ## 16 : \
+ (width) == Width32 ? opcode ## 32 : \
+ opcode ## 64)
+#define OPCODE_FOR_CANONICAL_WIDTH(opcode, width) ( \
+ (width) == Width64 ? opcode ## 64 : opcode ## 32)
+
</ins><span class="cx"> class LowerToAir {
</span><span class="cx"> public:
</span><span class="cx"> LowerToAir(Procedure& procedure)
</span><span class="lines">@@ -83,6 +95,12 @@
</span><span class="cx"> , m_dominators(procedure.dominators())
</span><span class="cx"> , m_procedure(procedure)
</span><span class="cx"> , m_code(procedure.code())
</span><ins>+ , m_blockInsertionSet(m_code)
+#if CPU(X86) || CPU(X86_64)
+ , m_eax(X86Registers::eax)
+ , m_ecx(X86Registers::ecx)
+ , m_edx(X86Registers::edx)
+#endif
</ins><span class="cx"> {
</span><span class="cx"> }
</span><span class="cx">
</span><span class="lines">@@ -124,8 +142,6 @@
</span><span class="cx"> // hoisted address expression before we duplicate it back into the loop.
</span><span class="cx"> for (B3::BasicBlock* block : m_procedure.blocksInPreOrder()) {
</span><span class="cx"> m_block = block;
</span><del>- // Reset some state.
- m_insts.resize(0);
</del><span class="cx">
</span><span class="cx"> m_isRare = !m_fastWorklist.saw(block);
</span><span class="cx">
</span><span class="lines">@@ -132,6 +148,12 @@
</span><span class="cx"> if (verbose)
</span><span class="cx"> dataLog("Lowering Block ", *block, ":\n");
</span><span class="cx">
</span><ins>+ // Make sure that the successors are set up correctly.
+ for (B3::FrequentedBlock successor : block->successors()) {
+ m_blockToBlock[block]->successors().append(
+ Air::FrequentedBlock(m_blockToBlock[successor.block()], successor.frequency()));
+ }
+
</ins><span class="cx"> // Process blocks in reverse order so we see uses before defs. That's what allows us
</span><span class="cx"> // to match patterns effectively.
</span><span class="cx"> for (unsigned i = block->size(); i--;) {
</span><span class="lines">@@ -149,19 +171,10 @@
</span><span class="cx"> }
</span><span class="cx"> }
</span><span class="cx">
</span><del>- // Now append the instructions. m_insts contains them in reverse order, so we process
- // it in reverse.
- for (unsigned i = m_insts.size(); i--;) {
- for (Inst& inst : m_insts[i])
- m_blockToBlock[block]->appendInst(WTFMove(inst));
- }
-
- // Make sure that the successors are set up correctly.
- for (B3::FrequentedBlock successor : block->successors()) {
- m_blockToBlock[block]->successors().append(
- Air::FrequentedBlock(m_blockToBlock[successor.block()], successor.frequency()));
- }
</del><ins>+ finishAppendingInstructions(m_blockToBlock[block]);
</ins><span class="cx"> }
</span><ins>+
+ m_blockInsertionSet.execute();
</ins><span class="cx">
</span><span class="cx"> Air::InsertionSet insertionSet(m_code);
</span><span class="cx"> for (Inst& inst : m_prologue)
</span><span class="lines">@@ -448,12 +461,12 @@
</span><span class="cx"> return std::nullopt;
</span><span class="cx"> return scale;
</span><span class="cx"> }
</span><del>-
</del><ins>+
</ins><span class="cx"> // This turns the given operand into an address.
</span><span class="cx"> Arg effectiveAddr(Value* address, int32_t offset, Width width)
</span><span class="cx"> {
</span><span class="cx"> ASSERT(Arg::isValidAddrForm(offset, width));
</span><del>-
</del><ins>+
</ins><span class="cx"> auto fallback = [&] () -> Arg {
</span><span class="cx"> return Arg::addr(tmp(address), offset);
</span><span class="cx"> };
</span><span class="lines">@@ -535,12 +548,15 @@
</span><span class="cx"> MemoryValue* value = memoryValue->as<MemoryValue>();
</span><span class="cx"> if (!value)
</span><span class="cx"> return Arg();
</span><ins>+
+ if (value->requiresSimpleAddr())
+ return Arg::simpleAddr(tmp(value->lastChild()));
</ins><span class="cx">
</span><span class="cx"> int32_t offset = value->offset();
</span><span class="cx"> Width width = value->accessWidth();
</span><span class="cx">
</span><span class="cx"> Arg result = effectiveAddr(value->lastChild(), offset, width);
</span><del>- ASSERT(result.isValidForm(width));
</del><ins>+ RELEASE_ASSERT(result.isValidForm(width));
</ins><span class="cx">
</span><span class="cx"> return result;
</span><span class="cx"> }
</span><span class="lines">@@ -561,11 +577,19 @@
</span><span class="cx">
</span><span class="cx"> ArgPromise loadPromiseAnyOpcode(Value* loadValue)
</span><span class="cx"> {
</span><ins>+ RELEASE_ASSERT(loadValue->as<MemoryValue>());
</ins><span class="cx"> if (!canBeInternal(loadValue))
</span><span class="cx"> return Arg();
</span><span class="cx"> if (crossesInterference(loadValue))
</span><span class="cx"> return Arg();
</span><del>- ArgPromise result(addr(loadValue), loadValue);
</del><ins>+ // On x86, all loads have fences. Doing this kind of instruction selection will move the load,
+ // but that's fine because our interference analysis stops the motion of fences around other
+ // fences. So, any load motion we introduce here would not be observable.
+ if (!isX86() && loadValue->as<MemoryValue>()->hasFence())
+ return Arg();
+ Arg loadAddr = addr(loadValue);
+ RELEASE_ASSERT(loadAddr);
+ ArgPromise result(loadAddr, loadValue);
</ins><span class="cx"> if (loadValue->traps())
</span><span class="cx"> result.setTraps(true);
</span><span class="cx"> return result;
</span><span class="lines">@@ -738,7 +762,7 @@
</span><span class="cx">
</span><span class="cx"> return true;
</span><span class="cx"> }
</span><del>-
</del><ins>+
</ins><span class="cx"> template<Air::Opcode opcode32, Air::Opcode opcode64, Air::Opcode opcodeDouble, Air::Opcode opcodeFloat, Commutativity commutativity = NotCommutative>
</span><span class="cx"> void appendBinOp(Value* left, Value* right)
</span><span class="cx"> {
</span><span class="lines">@@ -897,11 +921,9 @@
</span><span class="cx"> return;
</span><span class="cx"> }
</span><span class="cx">
</span><del>-#if CPU(X86) || CPU(X86_64)
</del><span class="cx"> append(Move, tmp(value), tmp(m_value));
</span><del>- append(Move, tmp(amount), Tmp(X86Registers::ecx));
- append(opcode, Tmp(X86Registers::ecx), tmp(m_value));
-#endif
</del><ins>+ append(Move, tmp(amount), m_ecx);
+ append(opcode, m_ecx, tmp(m_value));
</ins><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> template<Air::Opcode opcode32, Air::Opcode opcode64>
</span><span class="lines">@@ -930,10 +952,16 @@
</span><span class="cx"> Air::Opcode opcode32, Air::Opcode opcode64, Commutativity commutativity = NotCommutative>
</span><span class="cx"> bool tryAppendStoreBinOp(Value* left, Value* right)
</span><span class="cx"> {
</span><ins>+ RELEASE_ASSERT(m_value->as<MemoryValue>());
+
</ins><span class="cx"> Air::Opcode opcode = tryOpcodeForType(opcode32, opcode64, left->type());
</span><span class="cx"> if (opcode == Air::Oops)
</span><span class="cx"> return false;
</span><span class="cx">
</span><ins>+ // On x86, all stores have fences, and this isn't reordering the store itself.
+ if (!isX86() && m_value->as<MemoryValue>()->hasFence())
+ return false;
+
</ins><span class="cx"> Arg storeAddr = addr(m_value);
</span><span class="cx"> ASSERT(storeAddr);
</span><span class="cx">
</span><span class="lines">@@ -993,11 +1021,50 @@
</span><span class="cx">
</span><span class="cx"> return Inst(move, m_value, tmp(value), dest);
</span><span class="cx"> }
</span><ins>+
+ Air::Opcode storeOpcode(Width width, Bank bank, bool release)
+ {
+ switch (width) {
+ case Width8:
+ RELEASE_ASSERT(bank == GP);
+ return release ? StoreRel8 : Air::Store8;
+ case Width16:
+ RELEASE_ASSERT(bank == GP);
+ return release ? StoreRel16 : Air::Store16;
+ case Width32:
+ switch (bank) {
+ case GP:
+ return release ? StoreRel32 : Move32;
+ case FP:
+ RELEASE_ASSERT(!release);
+ return MoveFloat;
+ }
+ break;
+ case Width64:
+ RELEASE_ASSERT(is64Bit());
+ switch (bank) {
+ case GP:
+ return release ? StoreRel64 : Move;
+ case FP:
+ RELEASE_ASSERT(!release);
+ return MoveDouble;
+ }
+ break;
+ }
+ RELEASE_ASSERT_NOT_REACHED();
+ }
+
+ Air::Opcode storeOpcode(Value* value)
+ {
+ MemoryValue* memory = value->as<MemoryValue>();
+ RELEASE_ASSERT(memory->isStore());
+ return storeOpcode(memory->accessWidth(), memory->accessBank(), memory->hasFence());
+ }
</ins><span class="cx">
</span><span class="cx"> Inst createStore(Value* value, const Arg& dest)
</span><span class="cx"> {
</span><del>- Air::Opcode moveOpcode = moveForType(value->type());
- return createStore(moveOpcode, value, dest);
</del><ins>+ Air::Opcode moveOpcode = storeOpcode(value);
+ return createStore(moveOpcode, value->child(0), dest);
</ins><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> template<typename... Args>
</span><span class="lines">@@ -1005,7 +1072,7 @@
</span><span class="cx"> {
</span><span class="cx"> append(trappingInst(m_value, createStore(std::forward<Args>(args)...)));
</span><span class="cx"> }
</span><del>-
</del><ins>+
</ins><span class="cx"> Air::Opcode moveForType(Type type)
</span><span class="cx"> {
</span><span class="cx"> switch (type) {
</span><span class="lines">@@ -1072,7 +1139,41 @@
</span><span class="cx"> {
</span><span class="cx"> m_insts.last().append(inst);
</span><span class="cx"> }
</span><ins>+
+ void finishAppendingInstructions(Air::BasicBlock* target)
+ {
+ // Now append the instructions. m_insts contains them in reverse order, so we process
+ // it in reverse.
+ for (unsigned i = m_insts.size(); i--;) {
+ for (Inst& inst : m_insts[i])
+ target->appendInst(WTFMove(inst));
+ }
+ m_insts.resize(0);
+ }
+
+ Air::BasicBlock* newBlock()
+ {
+ return m_blockInsertionSet.insertAfter(m_blockToBlock[m_block]);
+ }
</ins><span class="cx">
</span><ins>+ // NOTE: This will create a continuation block (`nextBlock`) *after* any blocks you've created using
+ // newBlock(). So, it's preferable to create all of your blocks upfront using newBlock(). Also note
+ // that any code you emit before this will be prepended to the continuation, and any code you emit
+ // after this will be appended to the previous block.
+ void splitBlock(Air::BasicBlock*& previousBlock, Air::BasicBlock*& nextBlock)
+ {
+ Air::BasicBlock* block = m_blockToBlock[m_block];
+
+ previousBlock = block;
+ nextBlock = m_blockInsertionSet.insertAfter(block);
+
+ finishAppendingInstructions(nextBlock);
+ nextBlock->successors() = block->successors();
+ block->successors().clear();
+
+ m_insts.append(Vector<Inst>());
+ }
+
</ins><span class="cx"> template<typename T, typename... Arguments>
</span><span class="cx"> T* ensureSpecial(T*& field, Arguments&&... arguments)
</span><span class="cx"> {
</span><span class="lines">@@ -1122,7 +1223,7 @@
</span><span class="cx"> break;
</span><span class="cx"> case ValueRep::StackArgument:
</span><span class="cx"> arg = Arg::callArg(value.rep().offsetFromSP());
</span><del>- appendStore(value.value(), arg);
</del><ins>+ appendStore(moveForType(value.value()->type()), value.value(), arg);
</ins><span class="cx"> break;
</span><span class="cx"> default:
</span><span class="cx"> RELEASE_ASSERT_NOT_REACHED();
</span><span class="lines">@@ -1253,7 +1354,9 @@
</span><span class="cx">
</span><span class="cx"> // It's safe to share value, but since we're sharing, it means that we aren't locking it.
</span><span class="cx"> // If we don't lock it, then fusing loads is off limits and all of value's children will
</span><del>- // have to go through the sharing path as well.
</del><ins>+ // have to go through the sharing path as well. Fusing loads is off limits because the load
+ // could already have been emitted elsehwere - so fusing it here would duplicate the load.
+ // We don't consider that to be a legal optimization.
</ins><span class="cx"> canCommitInternal = false;
</span><span class="cx">
</span><span class="cx"> return Fuse;
</span><span class="lines">@@ -1471,6 +1574,7 @@
</span><span class="cx"> // FIXME: If this is unsigned then we can chop things off of the immediate.
</span><span class="cx"> // This might make the immediate more legal. Perhaps that's a job for
</span><span class="cx"> // strength reduction?
</span><ins>+ // https://bugs.webkit.org/show_bug.cgi?id=169248
</ins><span class="cx">
</span><span class="cx"> if (rightImm) {
</span><span class="cx"> if (Inst result = tryTest(width, loadPromise(left, loadOpcode), rightImm)) {
</span><span class="lines">@@ -1960,6 +2064,314 @@
</span><span class="cx"> return true;
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+ void appendX86Div(B3::Opcode op)
+ {
+ Air::Opcode convertToDoubleWord;
+ Air::Opcode div;
+ switch (m_value->type()) {
+ case Int32:
+ convertToDoubleWord = X86ConvertToDoubleWord32;
+ div = X86Div32;
+ break;
+ case Int64:
+ convertToDoubleWord = X86ConvertToQuadWord64;
+ div = X86Div64;
+ break;
+ default:
+ RELEASE_ASSERT_NOT_REACHED();
+ return;
+ }
+
+ ASSERT(op == Div || op == Mod);
+ Tmp result = op == Div ? m_eax : m_edx;
+
+ append(Move, tmp(m_value->child(0)), m_eax);
+ append(convertToDoubleWord, m_eax, m_edx);
+ append(div, m_eax, m_edx, tmp(m_value->child(1)));
+ append(Move, result, tmp(m_value));
+ }
+
+ void appendX86UDiv(B3::Opcode op)
+ {
+ Air::Opcode div = m_value->type() == Int32 ? X86UDiv32 : X86UDiv64;
+
+ ASSERT(op == UDiv || op == UMod);
+ Tmp result = op == UDiv ? m_eax : m_edx;
+
+ append(Move, tmp(m_value->child(0)), m_eax);
+ append(Xor64, m_edx, m_edx);
+ append(div, m_eax, m_edx, tmp(m_value->child(1)));
+ append(Move, result, tmp(m_value));
+ }
+
+ Air::Opcode loadLinkOpcode(Width width, bool fence)
+ {
+ return fence ? OPCODE_FOR_WIDTH(LoadLinkAcq, width) : OPCODE_FOR_WIDTH(LoadLink, width);
+ }
+
+ Air::Opcode storeCondOpcode(Width width, bool fence)
+ {
+ return fence ? OPCODE_FOR_WIDTH(StoreCondRel, width) : OPCODE_FOR_WIDTH(StoreCond, width);
+ }
+
+ // This can emit code for the following patterns:
+ // AtomicWeakCAS
+ // BitXor(AtomicWeakCAS, 1)
+ // AtomicStrongCAS
+ // Equal(AtomicStrongCAS, expected)
+ // NotEqual(AtomicStrongCAS, expected)
+ // Branch(AtomicWeakCAS)
+ // Branch(Equal(AtomicStrongCAS, expected))
+ // Branch(NotEqual(AtomicStrongCAS, expected))
+ //
+ // It assumes that atomicValue points to the CAS, and m_value points to the instruction being
+ // generated. It assumes that you've consumed everything that needs to be consumed.
+ void appendCAS(Value* atomicValue, bool invert)
+ {
+ AtomicValue* atomic = atomicValue->as<AtomicValue>();
+ RELEASE_ASSERT(atomic);
+
+ bool isBranch = m_value->opcode() == Branch;
+ bool isStrong = atomic->opcode() == AtomicStrongCAS;
+ bool returnsOldValue = m_value->opcode() == AtomicStrongCAS;
+ bool hasFence = atomic->hasFence();
+
+ Width width = atomic->accessWidth();
+ Arg address = addr(atomic);
+
+ Tmp valueResultTmp;
+ Tmp boolResultTmp;
+ if (returnsOldValue) {
+ RELEASE_ASSERT(!invert);
+ valueResultTmp = tmp(m_value);
+ boolResultTmp = m_code.newTmp(GP);
+ } else if (isBranch) {
+ valueResultTmp = m_code.newTmp(GP);
+ boolResultTmp = m_code.newTmp(GP);
+ } else {
+ valueResultTmp = m_code.newTmp(GP);
+ boolResultTmp = tmp(m_value);
+ }
+
+ Tmp successBoolResultTmp;
+ if (isStrong && !isBranch)
+ successBoolResultTmp = m_code.newTmp(GP);
+ else
+ successBoolResultTmp = boolResultTmp;
+
+ Tmp expectedValueTmp = tmp(atomic->child(0));
+ Tmp newValueTmp = tmp(atomic->child(1));
+
+ Air::FrequentedBlock success;
+ Air::FrequentedBlock failure;
+ if (isBranch) {
+ success = m_blockToBlock[m_block]->successor(invert);
+ failure = m_blockToBlock[m_block]->successor(!invert);
+ }
+
+ if (isX86()) {
+ append(relaxedMoveForType(atomic->accessType()), immOrTmp(atomic->child(0)), m_eax);
+ if (returnsOldValue) {
+ append(OPCODE_FOR_WIDTH(AtomicStrongCAS, width), m_eax, newValueTmp, address);
+ append(relaxedMoveForType(atomic->accessType()), m_eax, valueResultTmp);
+ } else if (isBranch) {
+ append(OPCODE_FOR_WIDTH(BranchAtomicStrongCAS, width), Arg::statusCond(MacroAssembler::Success), m_eax, newValueTmp, address);
+ m_blockToBlock[m_block]->setSuccessors(success, failure);
+ } else
+ append(OPCODE_FOR_WIDTH(AtomicStrongCAS, width), Arg::statusCond(invert ? MacroAssembler::Failure : MacroAssembler::Success), m_eax, tmp(atomic->child(1)), address, boolResultTmp);
+ return;
+ }
+
+ RELEASE_ASSERT(isARM64());
+ // We wish to emit:
+ //
+ // Block #reloop:
+ // LoadLink
+ // Branch NotEqual
+ // Successors: Then:#fail, Else: #store
+ // Block #store:
+ // StoreCond
+ // Xor $1, %result <--- only if !invert
+ // Jump
+ // Successors: #done
+ // Block #fail:
+ // Move $invert, %result
+ // Jump
+ // Successors: #done
+ // Block #done:
+
+ Air::BasicBlock* reloopBlock = newBlock();
+ Air::BasicBlock* storeBlock = newBlock();
+ Air::BasicBlock* successBlock = nullptr;
+ if (!isBranch && isStrong)
+ successBlock = newBlock();
+ Air::BasicBlock* failBlock = nullptr;
+ if (!isBranch) {
+ failBlock = newBlock();
+ failure = failBlock;
+ }
+ Air::BasicBlock* strongFailBlock;
+ if (isStrong && hasFence)
+ strongFailBlock = newBlock();
+ Air::FrequentedBlock comparisonFail = failure;
+ Air::FrequentedBlock weakFail;
+ if (isStrong) {
+ if (hasFence)
+ comparisonFail = strongFailBlock;
+ weakFail = reloopBlock;
+ } else
+ weakFail = failure;
+ Air::BasicBlock* beginBlock;
+ Air::BasicBlock* doneBlock;
+ splitBlock(beginBlock, doneBlock);
+
+ append(Air::Jump);
+ beginBlock->setSuccessors(reloopBlock);
+
+ reloopBlock->append(loadLinkOpcode(width, atomic->hasFence()), m_value, address, valueResultTmp);
+ reloopBlock->append(OPCODE_FOR_CANONICAL_WIDTH(Branch, width), m_value, Arg::relCond(MacroAssembler::NotEqual), valueResultTmp, expectedValueTmp);
+ reloopBlock->setSuccessors(comparisonFail, storeBlock);
+
+ storeBlock->append(storeCondOpcode(width, atomic->hasFence()), m_value, newValueTmp, address, successBoolResultTmp);
+ if (isBranch) {
+ storeBlock->append(BranchTest32, m_value, Arg::resCond(MacroAssembler::Zero), boolResultTmp, boolResultTmp);
+ storeBlock->setSuccessors(success, weakFail);
+ doneBlock->successors().clear();
+ RELEASE_ASSERT(!doneBlock->size());
+ doneBlock->append(Air::Oops, m_value);
+ } else {
+ if (isStrong) {
+ storeBlock->append(BranchTest32, m_value, Arg::resCond(MacroAssembler::Zero), successBoolResultTmp, successBoolResultTmp);
+ storeBlock->setSuccessors(successBlock, reloopBlock);
+
+ successBlock->append(Move, m_value, Arg::imm(!invert), boolResultTmp);
+ successBlock->append(Air::Jump, m_value);
+ successBlock->setSuccessors(doneBlock);
+ } else {
+ if (!invert)
+ storeBlock->append(Xor32, m_value, Arg::bitImm(1), boolResultTmp, boolResultTmp);
+
+ storeBlock->append(Air::Jump, m_value);
+ storeBlock->setSuccessors(doneBlock);
+ }
+
+ failBlock->append(Move, m_value, Arg::imm(invert), boolResultTmp);
+ failBlock->append(Air::Jump, m_value);
+ failBlock->setSuccessors(doneBlock);
+ }
+
+ if (isStrong && hasFence) {
+ Tmp tmp = m_code.newTmp(GP);
+ strongFailBlock->append(storeCondOpcode(width, atomic->hasFence()), m_value, valueResultTmp, address, tmp);
+ strongFailBlock->append(BranchTest32, m_value, Arg::resCond(MacroAssembler::Zero), tmp, tmp);
+ strongFailBlock->setSuccessors(failure, reloopBlock);
+ }
+ }
+
+ bool appendVoidAtomic(Air::Opcode atomicOpcode)
+ {
+ if (m_useCounts.numUses(m_value))
+ return false;
+
+ Arg address = addr(m_value);
+
+ if (isValidForm(atomicOpcode, Arg::Imm, address.kind()) && imm(m_value->child(0))) {
+ append(atomicOpcode, imm(m_value->child(0)), address);
+ return true;
+ }
+
+ if (isValidForm(atomicOpcode, Arg::Tmp, address.kind())) {
+ append(atomicOpcode, tmp(m_value->child(0)), address);
+ return true;
+ }
+
+ return false;
+ }
+
+ void appendGeneralAtomic(Air::Opcode opcode, Commutativity commutativity = NotCommutative)
+ {
+ AtomicValue* atomic = m_value->as<AtomicValue>();
+
+ Arg address = addr(m_value);
+ Tmp oldValue = m_code.newTmp(GP);
+ Tmp newValue = opcode == Air::Nop ? tmp(atomic->child(0)) : m_code.newTmp(GP);
+
+ // We need a CAS loop or a LL/SC loop. Using prepare/attempt jargon, we want:
+ //
+ // Block #reloop:
+ // Prepare
+ // opcode
+ // Attempt
+ // Successors: Then:#done, Else:#reloop
+ // Block #done:
+ // Move oldValue, result
+
+ append(relaxedMoveForType(atomic->type()), oldValue, tmp(atomic));
+
+ Air::BasicBlock* reloopBlock = newBlock();
+ Air::BasicBlock* beginBlock;
+ Air::BasicBlock* doneBlock;
+ splitBlock(beginBlock, doneBlock);
+
+ append(Air::Jump);
+ beginBlock->setSuccessors(reloopBlock);
+
+ Air::Opcode prepareOpcode;
+ if (isX86()) {
+ switch (atomic->accessWidth()) {
+ case Width8:
+ prepareOpcode = Load8SignedExtendTo32;
+ break;
+ case Width16:
+ prepareOpcode = Load16SignedExtendTo32;
+ break;
+ case Width32:
+ prepareOpcode = Move32;
+ break;
+ case Width64:
+ prepareOpcode = Move;
+ break;
+ }
+ } else {
+ RELEASE_ASSERT(isARM64());
+ prepareOpcode = loadLinkOpcode(atomic->accessWidth(), atomic->hasFence());
+ }
+ reloopBlock->append(prepareOpcode, m_value, address, oldValue);
+
+ if (opcode != Air::Nop) {
+ // FIXME: If we ever have to write this again, we need to find a way to share the code with
+ // appendBinOp.
+ // https://bugs.webkit.org/show_bug.cgi?id=169249
+ if (commutativity == Commutative && imm(atomic->child(0)) && isValidForm(opcode, Arg::Imm, Arg::Tmp, Arg::Tmp))
+ reloopBlock->append(opcode, m_value, imm(atomic->child(0)), oldValue, newValue);
+ else if (imm(atomic->child(0)) && isValidForm(opcode, Arg::Tmp, Arg::Imm, Arg::Tmp))
+ reloopBlock->append(opcode, m_value, oldValue, imm(atomic->child(0)), newValue);
+ else if (commutativity == Commutative && bitImm(atomic->child(0)) && isValidForm(opcode, Arg::BitImm, Arg::Tmp, Arg::Tmp))
+ reloopBlock->append(opcode, m_value, bitImm(atomic->child(0)), oldValue, newValue);
+ else if (isValidForm(opcode, Arg::Tmp, Arg::Tmp, Arg::Tmp))
+ reloopBlock->append(opcode, m_value, oldValue, tmp(atomic->child(0)), newValue);
+ else {
+ reloopBlock->append(relaxedMoveForType(atomic->type()), m_value, oldValue, newValue);
+ if (imm(atomic->child(0)) && isValidForm(opcode, Arg::Imm, Arg::Tmp))
+ reloopBlock->append(opcode, m_value, imm(atomic->child(0)), newValue);
+ else
+ reloopBlock->append(opcode, m_value, tmp(atomic->child(0)), newValue);
+ }
+ }
+
+ if (isX86()) {
+ Air::Opcode casOpcode = OPCODE_FOR_WIDTH(BranchAtomicStrongCAS, atomic->accessWidth());
+ reloopBlock->append(relaxedMoveForType(atomic->type()), m_value, oldValue, m_eax);
+ reloopBlock->append(casOpcode, m_value, Arg::statusCond(MacroAssembler::Success), m_eax, newValue, address);
+ } else {
+ RELEASE_ASSERT(isARM64());
+ Tmp boolResult = m_code.newTmp(GP);
+ reloopBlock->append(storeCondOpcode(atomic->accessWidth(), atomic->hasFence()), m_value, newValue, address, boolResult);
+ reloopBlock->append(BranchTest32, m_value, Arg::resCond(MacroAssembler::Zero), boolResult, boolResult);
+ }
+ reloopBlock->setSuccessors(doneBlock, reloopBlock);
+ }
+
</ins><span class="cx"> void lower()
</span><span class="cx"> {
</span><span class="cx"> switch (m_value->opcode()) {
</span><span class="lines">@@ -1970,30 +2382,50 @@
</span><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> case Load: {
</span><del>- append(trappingInst(m_value, moveForType(m_value->type()), m_value, addr(m_value), tmp(m_value)));
</del><ins>+ MemoryValue* memory = m_value->as<MemoryValue>();
+ Air::Opcode opcode = Air::Oops;
+ if (memory->hasFence()) {
+ switch (memory->type()) {
+ case Int32:
+ opcode = LoadAcq32;
+ break;
+ case Int64:
+ opcode = LoadAcq64;
+ break;
+ default:
+ RELEASE_ASSERT_NOT_REACHED();
+ break;
+ }
+ } else
+ opcode = moveForType(memory->type());
+ append(trappingInst(m_value, opcode, m_value, addr(m_value), tmp(m_value)));
</ins><span class="cx"> return;
</span><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> case Load8S: {
</span><del>- append(trappingInst(m_value, Load8SignedExtendTo32, m_value, addr(m_value), tmp(m_value)));
</del><ins>+ Air::Opcode opcode = m_value->as<MemoryValue>()->hasFence() ? LoadAcq8SignedExtendTo32 : Load8SignedExtendTo32;
+ append(trappingInst(m_value, opcode, m_value, addr(m_value), tmp(m_value)));
</ins><span class="cx"> return;
</span><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> case Load8Z: {
</span><del>- append(trappingInst(m_value, Load8, m_value, addr(m_value), tmp(m_value)));
</del><ins>+ Air::Opcode opcode = m_value->as<MemoryValue>()->hasFence() ? LoadAcq8 : Load8;
+ append(trappingInst(m_value, opcode, m_value, addr(m_value), tmp(m_value)));
</ins><span class="cx"> return;
</span><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> case Load16S: {
</span><del>- append(trappingInst(m_value, Load16SignedExtendTo32, m_value, addr(m_value), tmp(m_value)));
</del><ins>+ Air::Opcode opcode = m_value->as<MemoryValue>()->hasFence() ? LoadAcq16SignedExtendTo32 : Load16SignedExtendTo32;
+ append(trappingInst(m_value, opcode, m_value, addr(m_value), tmp(m_value)));
</ins><span class="cx"> return;
</span><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> case Load16Z: {
</span><del>- append(trappingInst(m_value, Load16, m_value, addr(m_value), tmp(m_value)));
</del><ins>+ Air::Opcode opcode = m_value->as<MemoryValue>()->hasFence() ? LoadAcq16 : Load16;
+ append(trappingInst(m_value, opcode, m_value, addr(m_value), tmp(m_value)));
</ins><span class="cx"> return;
</span><span class="cx"> }
</span><del>-
</del><ins>+
</ins><span class="cx"> case Add: {
</span><span class="cx"> if (tryAppendLea())
</span><span class="cx"> return;
</span><span class="lines">@@ -2091,7 +2523,7 @@
</span><span class="cx"> if (m_value->isChill())
</span><span class="cx"> RELEASE_ASSERT(isARM64());
</span><span class="cx"> if (isInt(m_value->type()) && isX86()) {
</span><del>- lowerX86Div(Div);
</del><ins>+ appendX86Div(Div);
</ins><span class="cx"> return;
</span><span class="cx"> }
</span><span class="cx"> ASSERT(!isX86() || isFloat(m_value->type()));
</span><span class="lines">@@ -2102,7 +2534,7 @@
</span><span class="cx">
</span><span class="cx"> case UDiv: {
</span><span class="cx"> if (isInt(m_value->type()) && isX86()) {
</span><del>- lowerX86UDiv(UDiv);
</del><ins>+ appendX86UDiv(UDiv);
</ins><span class="cx"> return;
</span><span class="cx"> }
</span><span class="cx">
</span><span class="lines">@@ -2116,13 +2548,13 @@
</span><span class="cx"> case Mod: {
</span><span class="cx"> RELEASE_ASSERT(isX86());
</span><span class="cx"> RELEASE_ASSERT(!m_value->isChill());
</span><del>- lowerX86Div(Mod);
</del><ins>+ appendX86Div(Mod);
</ins><span class="cx"> return;
</span><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> case UMod: {
</span><span class="cx"> RELEASE_ASSERT(isX86());
</span><del>- lowerX86UDiv(UMod);
</del><ins>+ appendX86UDiv(UMod);
</ins><span class="cx"> return;
</span><span class="cx"> }
</span><span class="cx">
</span><span class="lines">@@ -2161,10 +2593,28 @@
</span><span class="cx"> appendUnOp<Not32, Not64>(m_value->child(0));
</span><span class="cx"> return;
</span><span class="cx"> }
</span><ins>+
+ // This pattern is super useful on both x86 and ARM64, since the inversion of the CAS result
+ // can be done with zero cost on x86 (just flip the set from E to NE) and it's a progression
+ // on ARM64 (since STX returns 0 on success, so ordinarily we have to flip it).
+ if (m_value->child(1)->isInt(1)
+ && isAtomicCAS(m_value->child(0)->opcode())
+ && canBeInternal(m_value->child(0))) {
+ commitInternal(m_value->child(0));
+ appendCAS(m_value->child(0), true);
+ return;
+ }
+
</ins><span class="cx"> appendBinOp<Xor32, Xor64, XorDouble, XorFloat, Commutative>(
</span><span class="cx"> m_value->child(0), m_value->child(1));
</span><span class="cx"> return;
</span><span class="cx"> }
</span><ins>+
+ case Depend: {
+ RELEASE_ASSERT(isARM64());
+ appendUnOp<Depend32, Depend64>(m_value->child(0));
+ return;
+ }
</ins><span class="cx">
</span><span class="cx"> case Shl: {
</span><span class="cx"> if (m_value->child(1)->isInt32(1)) {
</span><span class="lines">@@ -2265,7 +2715,7 @@
</span><span class="cx"> }
</span><span class="cx"> }
</span><span class="cx">
</span><del>- appendStore(valueToStore, addr(m_value));
</del><ins>+ appendStore(m_value, addr(m_value));
</ins><span class="cx"> return;
</span><span class="cx"> }
</span><span class="cx">
</span><span class="lines">@@ -2286,7 +2736,7 @@
</span><span class="cx"> return;
</span><span class="cx"> }
</span><span class="cx"> }
</span><del>- appendStore(Air::Store8, valueToStore, addr(m_value));
</del><ins>+ appendStore(m_value, addr(m_value));
</ins><span class="cx"> return;
</span><span class="cx"> }
</span><span class="cx">
</span><span class="lines">@@ -2307,7 +2757,7 @@
</span><span class="cx"> return;
</span><span class="cx"> }
</span><span class="cx"> }
</span><del>- appendStore(Air::Store16, valueToStore, addr(m_value));
</del><ins>+ appendStore(m_value, addr(m_value));
</ins><span class="cx"> return;
</span><span class="cx"> }
</span><span class="cx">
</span><span class="lines">@@ -2415,7 +2865,27 @@
</span><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> case Equal:
</span><del>- case NotEqual:
</del><ins>+ case NotEqual: {
+ // FIXME: Teach this to match patterns that arise from subwidth CAS. The CAS's result has to
+ // be either zero- or sign-extended, and the value it's compared to should also be zero- or
+ // sign-extended in a matching way. It's not super clear that this is very profitable.
+ // https://bugs.webkit.org/show_bug.cgi?id=169250
+ if (m_value->child(0)->opcode() == AtomicStrongCAS
+ && m_value->child(0)->as<AtomicValue>()->isCanonicalWidth()
+ && m_value->child(0)->child(0) == m_value->child(1)
+ && canBeInternal(m_value->child(0))) {
+ ASSERT(!m_locked.contains(m_value->child(0)->child(1)));
+ ASSERT(!m_locked.contains(m_value->child(1)));
+
+ commitInternal(m_value->child(0));
+ appendCAS(m_value->child(0), m_value->opcode() == NotEqual);
+ return;
+ }
+
+ m_insts.last().append(createCompare(m_value));
+ return;
+ }
+
</ins><span class="cx"> case LessThan:
</span><span class="cx"> case GreaterThan:
</span><span class="cx"> case LessEqual:
</span><span class="lines">@@ -2440,6 +2910,7 @@
</span><span class="cx"> config.moveConditionallyFloat = MoveConditionallyFloat;
</span><span class="cx"> } else {
</span><span class="cx"> // FIXME: it's not obvious that these are particularly efficient.
</span><ins>+ // https://bugs.webkit.org/show_bug.cgi?id=169251
</ins><span class="cx"> config.moveConditionally32 = MoveDoubleConditionally32;
</span><span class="cx"> config.moveConditionally64 = MoveDoubleConditionally64;
</span><span class="cx"> config.moveConditionallyTest32 = MoveDoubleConditionallyTest32;
</span><span class="lines">@@ -2726,6 +3197,46 @@
</span><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> case Branch: {
</span><ins>+ if (canBeInternal(m_value->child(0))) {
+ Value* branchChild = m_value->child(0);
+ switch (branchChild->opcode()) {
+ case AtomicWeakCAS:
+ commitInternal(branchChild);
+ appendCAS(branchChild, false);
+ return;
+
+ case AtomicStrongCAS:
+ // A branch is a comparison to zero.
+ // FIXME: Teach this to match patterns that arise from subwidth CAS.
+ // https://bugs.webkit.org/show_bug.cgi?id=169250
+ if (branchChild->child(0)->isInt(0)
+ && branchChild->as<AtomicValue>()->isCanonicalWidth()) {
+ commitInternal(branchChild);
+ appendCAS(branchChild, true);
+ return;
+ }
+ break;
+
+ case Equal:
+ case NotEqual:
+ // FIXME: Teach this to match patterns that arise from subwidth CAS.
+ // https://bugs.webkit.org/show_bug.cgi?id=169250
+ if (branchChild->child(0)->opcode() == AtomicStrongCAS
+ && branchChild->child(0)->as<AtomicValue>()->isCanonicalWidth()
+ && canBeInternal(branchChild->child(0))
+ && branchChild->child(0)->child(0) == branchChild->child(1)) {
+ commitInternal(branchChild);
+ commitInternal(branchChild->child(0));
+ appendCAS(branchChild->child(0), branchChild->opcode() == NotEqual);
+ return;
+ }
+ break;
+
+ default:
+ break;
+ }
+ }
+
</ins><span class="cx"> m_insts.last().append(createBranch(m_value->child(0)));
</span><span class="cx"> return;
</span><span class="cx"> }
</span><span class="lines">@@ -2783,7 +3294,81 @@
</span><span class="cx"> append(Air::EntrySwitch);
</span><span class="cx"> return;
</span><span class="cx"> }
</span><ins>+
+ case AtomicWeakCAS:
+ case AtomicStrongCAS: {
+ appendCAS(m_value, false);
+ return;
+ }
+
+ case AtomicXchgAdd: {
+ AtomicValue* atomic = m_value->as<AtomicValue>();
+ if (appendVoidAtomic(OPCODE_FOR_WIDTH(AtomicAdd, atomic->accessWidth())))
+ return;
+
+ Arg address = addr(atomic);
+ Air::Opcode opcode = OPCODE_FOR_WIDTH(AtomicXchgAdd, atomic->accessWidth());
+ if (isValidForm(opcode, Arg::Tmp, address.kind())) {
+ append(relaxedMoveForType(atomic->type()), tmp(atomic->child(0)), tmp(atomic));
+ append(opcode, tmp(atomic), address);
+ return;
+ }
</ins><span class="cx">
</span><ins>+ appendGeneralAtomic(OPCODE_FOR_CANONICAL_WIDTH(Add, atomic->accessWidth()), Commutative);
+ return;
+ }
+
+ case AtomicXchgSub: {
+ AtomicValue* atomic = m_value->as<AtomicValue>();
+ if (appendVoidAtomic(OPCODE_FOR_WIDTH(AtomicSub, atomic->accessWidth())))
+ return;
+
+ appendGeneralAtomic(OPCODE_FOR_CANONICAL_WIDTH(Sub, atomic->accessWidth()));
+ return;
+ }
+
+ case AtomicXchgAnd: {
+ AtomicValue* atomic = m_value->as<AtomicValue>();
+ if (appendVoidAtomic(OPCODE_FOR_WIDTH(AtomicAnd, atomic->accessWidth())))
+ return;
+
+ appendGeneralAtomic(OPCODE_FOR_CANONICAL_WIDTH(And, atomic->accessWidth()), Commutative);
+ return;
+ }
+
+ case AtomicXchgOr: {
+ AtomicValue* atomic = m_value->as<AtomicValue>();
+ if (appendVoidAtomic(OPCODE_FOR_WIDTH(AtomicOr, atomic->accessWidth())))
+ return;
+
+ appendGeneralAtomic(OPCODE_FOR_CANONICAL_WIDTH(Or, atomic->accessWidth()), Commutative);
+ return;
+ }
+
+ case AtomicXchgXor: {
+ AtomicValue* atomic = m_value->as<AtomicValue>();
+ if (appendVoidAtomic(OPCODE_FOR_WIDTH(AtomicXor, atomic->accessWidth())))
+ return;
+
+ appendGeneralAtomic(OPCODE_FOR_CANONICAL_WIDTH(Xor, atomic->accessWidth()), Commutative);
+ return;
+ }
+
+ case AtomicXchg: {
+ AtomicValue* atomic = m_value->as<AtomicValue>();
+
+ Arg address = addr(atomic);
+ Air::Opcode opcode = OPCODE_FOR_WIDTH(AtomicXchg, atomic->accessWidth());
+ if (isValidForm(opcode, Arg::Tmp, address.kind())) {
+ append(relaxedMoveForType(atomic->type()), tmp(atomic->child(0)), tmp(atomic));
+ append(opcode, tmp(atomic), address);
+ return;
+ }
+
+ appendGeneralAtomic(Air::Nop);
+ return;
+ }
+
</ins><span class="cx"> default:
</span><span class="cx"> break;
</span><span class="cx"> }
</span><span class="lines">@@ -2791,64 +3376,7 @@
</span><span class="cx"> dataLog("FATAL: could not lower ", deepDump(m_procedure, m_value), "\n");
</span><span class="cx"> RELEASE_ASSERT_NOT_REACHED();
</span><span class="cx"> }
</span><del>-
- void lowerX86Div(B3::Opcode op)
- {
-#if CPU(X86) || CPU(X86_64)
- Tmp eax = Tmp(X86Registers::eax);
- Tmp edx = Tmp(X86Registers::edx);
-
- Air::Opcode convertToDoubleWord;
- Air::Opcode div;
- switch (m_value->type()) {
- case Int32:
- convertToDoubleWord = X86ConvertToDoubleWord32;
- div = X86Div32;
- break;
- case Int64:
- convertToDoubleWord = X86ConvertToQuadWord64;
- div = X86Div64;
- break;
- default:
- RELEASE_ASSERT_NOT_REACHED();
- return;
- }
-
- ASSERT(op == Div || op == Mod);
- X86Registers::RegisterID result = op == Div ? X86Registers::eax : X86Registers::edx;
-
- append(Move, tmp(m_value->child(0)), eax);
- append(convertToDoubleWord, eax, edx);
- append(div, eax, edx, tmp(m_value->child(1)));
- append(Move, Tmp(result), tmp(m_value));
-
-#else
- UNUSED_PARAM(op);
- UNREACHABLE_FOR_PLATFORM();
-#endif
- }
-
- void lowerX86UDiv(B3::Opcode op)
- {
-#if CPU(X86) || CPU(X86_64)
- Tmp eax = Tmp(X86Registers::eax);
- Tmp edx = Tmp(X86Registers::edx);
-
- Air::Opcode div = m_value->type() == Int32 ? X86UDiv32 : X86UDiv64;
-
- ASSERT(op == UDiv || op == UMod);
- X86Registers::RegisterID result = op == UDiv ? X86Registers::eax : X86Registers::edx;
-
- append(Move, tmp(m_value->child(0)), eax);
- append(Xor64, edx, edx);
- append(div, eax, edx, tmp(m_value->child(1)));
- append(Move, Tmp(result), tmp(m_value));
-#else
- UNUSED_PARAM(op);
- UNREACHABLE_FOR_PLATFORM();
-#endif
- }
-
</del><ins>+
</ins><span class="cx"> IndexSet<Value> m_locked; // These are values that will have no Tmp in Air.
</span><span class="cx"> IndexMap<Value, Tmp> m_valueToTmp; // These are values that must have a Tmp in Air. We say that a Value* with a non-null Tmp is "pinned".
</span><span class="cx"> IndexMap<Value, Tmp> m_phiToTmp; // Each Phi gets its own Tmp.
</span><span class="lines">@@ -2863,7 +3391,7 @@
</span><span class="cx">
</span><span class="cx"> Vector<Vector<Inst, 4>> m_insts;
</span><span class="cx"> Vector<Inst> m_prologue;
</span><del>-
</del><ins>+
</ins><span class="cx"> B3::BasicBlock* m_block;
</span><span class="cx"> bool m_isRare;
</span><span class="cx"> unsigned m_index;
</span><span class="lines">@@ -2874,6 +3402,12 @@
</span><span class="cx">
</span><span class="cx"> Procedure& m_procedure;
</span><span class="cx"> Code& m_code;
</span><ins>+
+ Air::BlockInsertionSet m_blockInsertionSet;
+
+ Tmp m_eax;
+ Tmp m_ecx;
+ Tmp m_edx;
</ins><span class="cx"> };
</span><span class="cx">
</span><span class="cx"> } // anonymous namespace
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreb3B3LowerToAirh"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/b3/B3LowerToAir.h (213713 => 213714)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/b3/B3LowerToAir.h 2017-03-10 17:32:07 UTC (rev 213713)
+++ trunk/Source/JavaScriptCore/b3/B3LowerToAir.h 2017-03-10 17:49:42 UTC (rev 213714)
</span><span class="lines">@@ -1,5 +1,5 @@
</span><span class="cx"> /*
</span><del>- * Copyright (C) 2015-2016 Apple Inc. All rights reserved.
</del><ins>+ * Copyright (C) 2015-2017 Apple Inc. All rights reserved.
</ins><span class="cx"> *
</span><span class="cx"> * Redistribution and use in source and binary forms, with or without
</span><span class="cx"> * modification, are permitted provided that the following conditions
</span><span class="lines">@@ -32,7 +32,8 @@
</span><span class="cx"> class Procedure;
</span><span class="cx"> namespace Air { class Code; }
</span><span class="cx">
</span><del>-// This lowers the current B3 procedure to an Air code.
</del><ins>+// This lowers the current B3 procedure to an Air code. Note that this assumes that the procedure is in
+// quirks mode, but it does not assert this, to simplify how we write tests.
</ins><span class="cx">
</span><span class="cx"> JS_EXPORT_PRIVATE void lowerToAir(Procedure&);
</span><span class="cx">
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreb3B3MemoryValuecpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/b3/B3MemoryValue.cpp (213713 => 213714)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/b3/B3MemoryValue.cpp 2017-03-10 17:32:07 UTC (rev 213713)
+++ trunk/Source/JavaScriptCore/b3/B3MemoryValue.cpp 2017-03-10 17:49:42 UTC (rev 213714)
</span><span class="lines">@@ -28,6 +28,10 @@
</span><span class="cx">
</span><span class="cx"> #if ENABLE(B3_JIT)
</span><span class="cx">
</span><ins>+#include "B3AtomicValue.h"
+#include "B3MemoryValueInlines.h"
+#include "B3ValueInlines.h"
+
</ins><span class="cx"> namespace JSC { namespace B3 {
</span><span class="cx">
</span><span class="cx"> MemoryValue::~MemoryValue()
</span><span class="lines">@@ -34,39 +38,39 @@
</span><span class="cx"> {
</span><span class="cx"> }
</span><span class="cx">
</span><del>-size_t MemoryValue::accessByteSize() const
</del><ins>+bool MemoryValue::isLegalOffset(int64_t offset) const
</ins><span class="cx"> {
</span><del>- switch (opcode()) {
- case Load8Z:
- case Load8S:
- case Store8:
- return 1;
- case Load16Z:
- case Load16S:
- case Store16:
- return 2;
- case Load:
- return sizeofType(type());
- case Store:
- return sizeofType(child(0)->type());
- default:
- RELEASE_ASSERT_NOT_REACHED();
- return 0;
- }
</del><ins>+ return B3::isRepresentableAs<int32_t>(offset) && isLegalOffset(static_cast<int32_t>(offset));
</ins><span class="cx"> }
</span><span class="cx">
</span><del>-Width MemoryValue::accessWidth() const
</del><ins>+Type MemoryValue::accessType() const
</ins><span class="cx"> {
</span><del>- return widthForBytes(accessByteSize());
</del><ins>+ if (isLoad())
+ return type();
+ // This happens to work for atomics, too. That's why AtomicValue does not need to override this.
+ return child(0)->type();
</ins><span class="cx"> }
</span><span class="cx">
</span><ins>+Bank MemoryValue::accessBank() const
+{
+ return bankForType(accessType());
+}
+
+size_t MemoryValue::accessByteSize() const
+{
+ return bytes(accessWidth());
+}
+
</ins><span class="cx"> void MemoryValue::dumpMeta(CommaPrinter& comma, PrintStream& out) const
</span><span class="cx"> {
</span><span class="cx"> if (m_offset)
</span><span class="cx"> out.print(comma, "offset = ", m_offset);
</span><span class="cx"> if ((isLoad() && effects().reads != range())
</span><del>- || (isStore() && effects().writes != range()))
</del><ins>+ || (isStore() && effects().writes != range())
+ || isExotic())
</ins><span class="cx"> out.print(comma, "range = ", range());
</span><ins>+ if (isExotic())
+ out.print(comma, "fenceRange = ", fenceRange());
</ins><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> Value* MemoryValue::cloneImpl() const
</span><span class="lines">@@ -74,6 +78,62 @@
</span><span class="cx"> return new MemoryValue(*this);
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+// Use this form for Load (but not Load8Z, Load8S, or any of the Loads that have a suffix that
+// describes the returned type).
+MemoryValue::MemoryValue(Kind kind, Type type, Origin origin, Value* pointer, int32_t offset, HeapRange range, HeapRange fenceRange)
+ : Value(CheckedOpcode, kind, type, origin, pointer)
+ , m_offset(offset)
+ , m_range(range)
+ , m_fenceRange(fenceRange)
+{
+ if (!ASSERT_DISABLED) {
+ switch (kind.opcode()) {
+ case Load:
+ break;
+ case Load8Z:
+ case Load8S:
+ case Load16Z:
+ case Load16S:
+ ASSERT(type == Int32);
+ break;
+ case Store8:
+ case Store16:
+ case Store:
+ ASSERT(type == Void);
+ break;
+ default:
+ ASSERT_NOT_REACHED();
+ }
+ }
+}
+
+// Use this form for loads where the return type is implied.
+MemoryValue::MemoryValue(Kind kind, Origin origin, Value* pointer, int32_t offset, HeapRange range, HeapRange fenceRange)
+ : MemoryValue(kind, Int32, origin, pointer, offset, range, fenceRange)
+{
+ if (!ASSERT_DISABLED) {
+ switch (kind.opcode()) {
+ case Load8Z:
+ case Load8S:
+ case Load16Z:
+ case Load16S:
+ break;
+ default:
+ ASSERT_NOT_REACHED();
+ }
+ }
+}
+
+// Use this form for stores.
+MemoryValue::MemoryValue(Kind kind, Origin origin, Value* value, Value* pointer, int32_t offset, HeapRange range, HeapRange fenceRange)
+ : Value(CheckedOpcode, kind, Void, origin, value, pointer)
+ , m_offset(offset)
+ , m_range(range)
+ , m_fenceRange(fenceRange)
+{
+ ASSERT(B3::isStore(kind.opcode()));
+}
+
</ins><span class="cx"> } } // namespace JSC::B3
</span><span class="cx">
</span><span class="cx"> #endif // ENABLE(B3_JIT)
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreb3B3MemoryValueh"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/b3/B3MemoryValue.h (213713 => 213714)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/b3/B3MemoryValue.h 2017-03-10 17:32:07 UTC (rev 213713)
+++ trunk/Source/JavaScriptCore/b3/B3MemoryValue.h 2017-03-10 17:49:42 UTC (rev 213714)
</span><span class="lines">@@ -27,127 +27,88 @@
</span><span class="cx">
</span><span class="cx"> #if ENABLE(B3_JIT)
</span><span class="cx">
</span><ins>+#include "AirArg.h"
+#include "B3Bank.h"
</ins><span class="cx"> #include "B3HeapRange.h"
</span><span class="cx"> #include "B3Value.h"
</span><span class="cx">
</span><span class="cx"> namespace JSC { namespace B3 {
</span><span class="cx">
</span><del>-// FIXME: We want to allow fenced memory accesses on ARM.
-// https://bugs.webkit.org/show_bug.cgi?id=162349
-
</del><span class="cx"> class JS_EXPORT_PRIVATE MemoryValue : public Value {
</span><span class="cx"> public:
</span><span class="cx"> static bool accepts(Kind kind)
</span><span class="cx"> {
</span><del>- switch (kind.opcode()) {
- case Load8Z:
- case Load8S:
- case Load16Z:
- case Load16S:
- case Load:
- case Store8:
- case Store16:
- case Store:
- return true;
- default:
- return false;
- }
</del><ins>+ return isMemoryAccess(kind.opcode());
</ins><span class="cx"> }
</span><span class="cx">
</span><del>- static bool isStore(Kind kind)
- {
- switch (kind.opcode()) {
- case Store8:
- case Store16:
- case Store:
- return true;
- default:
- return false;
- }
- }
-
- static bool isLoad(Kind kind)
- {
- return accepts(kind) && !isStore(kind);
- }
-
</del><span class="cx"> ~MemoryValue();
</span><span class="cx">
</span><span class="cx"> int32_t offset() const { return m_offset; }
</span><span class="cx"> void setOffset(int32_t offset) { m_offset = offset; }
</span><ins>+
+ // You don't have to worry about using legal offsets unless you've entered quirks mode.
+ bool isLegalOffset(int32_t offset) const;
+ bool isLegalOffset(int64_t offset) const;
+
+ // A necessary consequence of MemoryValue having an offset is that it participates in instruction
+ // selection. This tells you if this will get lowered to something that requires an offsetless
+ // address.
+ bool requiresSimpleAddr() const;
</ins><span class="cx">
</span><span class="cx"> const HeapRange& range() const { return m_range; }
</span><span class="cx"> void setRange(const HeapRange& range) { m_range = range; }
</span><ins>+
+ // This is an alias for range.
+ const HeapRange& accessRange() const { return range(); }
+ void setAccessRange(const HeapRange& range) { setRange(range); }
+
+ const HeapRange& fenceRange() const { return m_fenceRange; }
+ void setFenceRange(const HeapRange& range) { m_fenceRange = range; }
</ins><span class="cx">
</span><del>- bool isStore() const { return type() == Void; }
- bool isLoad() const { return type() != Void; }
</del><ins>+ bool isStore() const { return B3::isStore(opcode()); }
+ bool isLoad() const { return B3::isLoad(opcode()); }
</ins><span class="cx">
</span><ins>+ bool hasFence() const { return !!fenceRange(); }
+ bool isExotic() const { return hasFence() || isAtomic(opcode()); }
+
+ Type accessType() const;
+ Bank accessBank() const;
</ins><span class="cx"> size_t accessByteSize() const;
</span><ins>+
</ins><span class="cx"> Width accessWidth() const;
</span><span class="cx">
</span><ins>+ bool isCanonicalWidth() const { return B3::isCanonicalWidth(accessWidth()); }
+
</ins><span class="cx"> protected:
</span><del>- void dumpMeta(CommaPrinter& comma, PrintStream&) const override;
</del><ins>+ void dumpMeta(CommaPrinter&, PrintStream&) const override;
</ins><span class="cx">
</span><span class="cx"> Value* cloneImpl() const override;
</span><span class="cx">
</span><ins>+ template<typename... Arguments>
+ MemoryValue(CheckedOpcodeTag, Kind kind, Type type, Origin origin, int32_t offset, HeapRange range, HeapRange fenceRange, Arguments... arguments)
+ : Value(CheckedOpcode, kind, type, origin, arguments...)
+ , m_offset(offset)
+ , m_range(range)
+ , m_fenceRange(fenceRange)
+ {
+ }
+
</ins><span class="cx"> private:
</span><span class="cx"> friend class Procedure;
</span><del>-
</del><ins>+
</ins><span class="cx"> // Use this form for Load (but not Load8Z, Load8S, or any of the Loads that have a suffix that
</span><span class="cx"> // describes the returned type).
</span><del>- MemoryValue(Kind kind, Type type, Origin origin, Value* pointer, int32_t offset = 0)
- : Value(CheckedOpcode, kind, type, origin, pointer)
- , m_offset(offset)
- , m_range(HeapRange::top())
- {
- if (!ASSERT_DISABLED) {
- switch (kind.opcode()) {
- case Load:
- break;
- case Load8Z:
- case Load8S:
- case Load16Z:
- case Load16S:
- ASSERT(type == Int32);
- break;
- case Store8:
- case Store16:
- case Store:
- ASSERT(type == Void);
- break;
- default:
- ASSERT_NOT_REACHED();
- }
- }
- }
</del><ins>+ MemoryValue(Kind, Type, Origin, Value* pointer, int32_t offset = 0, HeapRange range = HeapRange::top(), HeapRange fenceRange = HeapRange());
</ins><span class="cx">
</span><span class="cx"> // Use this form for loads where the return type is implied.
</span><del>- MemoryValue(Kind kind, Origin origin, Value* pointer, int32_t offset = 0)
- : MemoryValue(kind, Int32, origin, pointer, offset)
- {
- }
</del><ins>+ MemoryValue(Kind, Origin, Value* pointer, int32_t offset = 0, HeapRange range = HeapRange::top(), HeapRange fenceRange = HeapRange());
</ins><span class="cx">
</span><span class="cx"> // Use this form for stores.
</span><del>- MemoryValue(Kind kind, Origin origin, Value* value, Value* pointer, int32_t offset = 0)
- : Value(CheckedOpcode, kind, Void, origin, value, pointer)
- , m_offset(offset)
- , m_range(HeapRange::top())
- {
- if (!ASSERT_DISABLED) {
- switch (kind.opcode()) {
- case Store8:
- case Store16:
- case Store:
- break;
- default:
- ASSERT_NOT_REACHED();
- break;
- }
- }
- }
-
</del><ins>+ MemoryValue(Kind, Origin, Value* value, Value* pointer, int32_t offset = 0, HeapRange range = HeapRange::top(), HeapRange fenceRange = HeapRange());
+
</ins><span class="cx"> int32_t m_offset { 0 };
</span><del>- HeapRange m_range;
</del><ins>+ HeapRange m_range { HeapRange::top() };
+ HeapRange m_fenceRange { HeapRange() };
</ins><span class="cx"> };
</span><span class="cx">
</span><span class="cx"> } } // namespace JSC::B3
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreb3B3MemoryValueInlinesh"></a>
<div class="addfile"><h4>Added: trunk/Source/JavaScriptCore/b3/B3MemoryValueInlines.h (0 => 213714)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/b3/B3MemoryValueInlines.h (rev 0)
+++ trunk/Source/JavaScriptCore/b3/B3MemoryValueInlines.h 2017-03-10 17:49:42 UTC (rev 213714)
</span><span class="lines">@@ -0,0 +1,84 @@
</span><ins>+/*
+ * Copyright (C) 2017 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+ * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+ * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+ * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#pragma once
+
+#if ENABLE(B3_JIT)
+
+#include "AirArg.h"
+#include "B3AtomicValue.h"
+
+namespace JSC { namespace B3 {
+
+inline bool MemoryValue::isLegalOffset(int32_t offset) const
+{
+ // NOTE: This is inline because it constant-folds to true on x86!
+
+ // So far only X86 allows exotic loads to have an offset.
+ if (requiresSimpleAddr())
+ return !offset;
+
+ return Air::Arg::isValidAddrForm(offset, accessWidth());
+}
+
+inline bool MemoryValue::requiresSimpleAddr() const
+{
+ return !isX86() && isExotic();
+}
+
+inline Width MemoryValue::accessWidth() const
+{
+ switch (opcode()) {
+ case Load8Z:
+ case Load8S:
+ case Store8:
+ return Width8;
+ case Load16Z:
+ case Load16S:
+ case Store16:
+ return Width16;
+ case Load:
+ return widthForType(type());
+ case Store:
+ return widthForType(child(0)->type());
+ case AtomicWeakCAS:
+ case AtomicStrongCAS:
+ case AtomicXchgAdd:
+ case AtomicXchgAnd:
+ case AtomicXchgOr:
+ case AtomicXchgSub:
+ case AtomicXchgXor:
+ case AtomicXchg:
+ return as<AtomicValue>()->accessWidth();
+ default:
+ RELEASE_ASSERT_NOT_REACHED();
+ return Width8;
+ }
+}
+
+} } // namespace JSC::B3
+
+#endif // ENABLE(B3_JIT)
+
</ins></span></pre></div>
<a id="trunkSourceJavaScriptCoreb3B3MoveConstantscpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/b3/B3MoveConstants.cpp (213713 => 213714)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/b3/B3MoveConstants.cpp 2017-03-10 17:32:07 UTC (rev 213713)
+++ trunk/Source/JavaScriptCore/b3/B3MoveConstants.cpp 2017-03-10 17:49:42 UTC (rev 213714)
</span><span class="lines">@@ -32,7 +32,7 @@
</span><span class="cx"> #include "B3BasicBlockInlines.h"
</span><span class="cx"> #include "B3Dominators.h"
</span><span class="cx"> #include "B3InsertionSetInlines.h"
</span><del>-#include "B3MemoryValue.h"
</del><ins>+#include "B3MemoryValueInlines.h"
</ins><span class="cx"> #include "B3PhaseScope.h"
</span><span class="cx"> #include "B3ProcedureInlines.h"
</span><span class="cx"> #include "B3ValueInlines.h"
</span><span class="lines">@@ -203,12 +203,8 @@
</span><span class="cx"> if (!candidatePointer->hasIntPtr())
</span><span class="cx"> return false;
</span><span class="cx">
</span><del>- intptr_t offset = desiredOffset(candidatePointer);
- if (!B3::isRepresentableAs<int32_t>(static_cast<int64_t>(offset)))
- return false;
- return Air::Arg::isValidAddrForm(
- static_cast<int32_t>(offset),
- widthForBytes(memoryValue->accessByteSize()));
</del><ins>+ int64_t offset = desiredOffset(candidatePointer);
+ return memoryValue->isLegalOffset(offset);
</ins><span class="cx"> });
</span><span class="cx">
</span><span class="cx"> if (bestPointer) {
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreb3B3NativeTraitsh"></a>
<div class="addfile"><h4>Added: trunk/Source/JavaScriptCore/b3/B3NativeTraits.h (0 => 213714)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/b3/B3NativeTraits.h (rev 0)
+++ trunk/Source/JavaScriptCore/b3/B3NativeTraits.h 2017-03-10 17:49:42 UTC (rev 213714)
</span><span class="lines">@@ -0,0 +1,111 @@
</span><ins>+/*
+ * Copyright (C) 2017 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+ * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+ * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+ * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#pragma once
+
+#if ENABLE(B3_JIT)
+
+#include "B3Bank.h"
+#include "B3Type.h"
+#include "B3Width.h"
+
+namespace JSC { namespace B3 {
+
+template<typename> struct NativeTraits;
+
+template<> struct NativeTraits<int8_t> {
+ typedef int32_t CanonicalType;
+ static const Bank bank = GP;
+ static const Width width = Width8;
+ static const Type type = Int32;
+};
+
+template<> struct NativeTraits<uint8_t> {
+ typedef int32_t CanonicalType;
+ static const Bank bank = GP;
+ static const Width width = Width8;
+ static const Type type = Int32;
+};
+
+template<> struct NativeTraits<int16_t> {
+ typedef int32_t CanonicalType;
+ static const Bank bank = GP;
+ static const Width width = Width16;
+ static const Type type = Int32;
+};
+
+template<> struct NativeTraits<uint16_t> {
+ typedef int32_t CanonicalType;
+ static const Bank bank = GP;
+ static const Width width = Width16;
+ static const Type type = Int32;
+};
+
+template<> struct NativeTraits<int32_t> {
+ typedef int32_t CanonicalType;
+ static const Bank bank = GP;
+ static const Width width = Width32;
+ static const Type type = Int32;
+};
+
+template<> struct NativeTraits<uint32_t> {
+ typedef int32_t CanonicalType;
+ static const Bank bank = GP;
+ static const Width width = Width32;
+ static const Type type = Int32;
+};
+
+template<> struct NativeTraits<int64_t> {
+ typedef int64_t CanonicalType;
+ static const Bank bank = GP;
+ static const Width width = Width64;
+ static const Type type = Int64;
+};
+
+template<> struct NativeTraits<uint64_t> {
+ typedef int64_t CanonicalType;
+ static const Bank bank = GP;
+ static const Width width = Width64;
+ static const Type type = Int64;
+};
+
+template<> struct NativeTraits<float> {
+ typedef float CanonicalType;
+ static const Bank bank = FP;
+ static const Width width = Width32;
+ static const Type type = Float;
+};
+
+template<> struct NativeTraits<double> {
+ typedef double CanonicalType;
+ static const Bank bank = FP;
+ static const Width width = Width64;
+ static const Type type = Double;
+};
+
+} } // namespace JSC::B3
+
+#endif // ENABLE(B3_JIT)
+
</ins></span></pre></div>
<a id="trunkSourceJavaScriptCoreb3B3Opcodecpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/b3/B3Opcode.cpp (213713 => 213714)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/b3/B3Opcode.cpp 2017-03-10 17:32:07 UTC (rev 213713)
+++ trunk/Source/JavaScriptCore/b3/B3Opcode.cpp 2017-03-10 17:49:42 UTC (rev 213714)
</span><span class="lines">@@ -1,5 +1,5 @@
</span><span class="cx"> /*
</span><del>- * Copyright (C) 2015-2016 Apple Inc. All rights reserved.
</del><ins>+ * Copyright (C) 2015-2017 Apple Inc. All rights reserved.
</ins><span class="cx"> *
</span><span class="cx"> * Redistribution and use in source and binary forms, with or without
</span><span class="cx"> * modification, are permitted provided that the following conditions
</span><span class="lines">@@ -68,6 +68,23 @@
</span><span class="cx"> }
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+Opcode storeOpcode(Bank bank, Width width)
+{
+ switch (bank) {
+ case GP:
+ switch (width) {
+ case Width8:
+ return Store8;
+ case Width16:
+ return Store16;
+ default:
+ return Store;
+ }
+ case FP:
+ return Store;
+ }
+}
+
</ins><span class="cx"> } } // namespace JSC::B3
</span><span class="cx">
</span><span class="cx"> namespace WTF {
</span><span class="lines">@@ -263,6 +280,33 @@
</span><span class="cx"> case Store:
</span><span class="cx"> out.print("Store");
</span><span class="cx"> return;
</span><ins>+ case AtomicWeakCAS:
+ out.print("AtomicWeakCAS");
+ return;
+ case AtomicStrongCAS:
+ out.print("AtomicStrongCAS");
+ return;
+ case AtomicXchgAdd:
+ out.print("AtomicXchgAdd");
+ return;
+ case AtomicXchgAnd:
+ out.print("AtomicXchgAnd");
+ return;
+ case AtomicXchgOr:
+ out.print("AtomicXchgOr");
+ return;
+ case AtomicXchgSub:
+ out.print("AtomicXchgSub");
+ return;
+ case AtomicXchgXor:
+ out.print("AtomicXchgXor");
+ return;
+ case AtomicXchg:
+ out.print("AtomicXchg");
+ return;
+ case Depend:
+ out.print("Depend");
+ return;
</ins><span class="cx"> case WasmAddress:
</span><span class="cx"> out.print("WasmAddress");
</span><span class="cx"> return;
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreb3B3Opcodeh"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/b3/B3Opcode.h (213713 => 213714)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/b3/B3Opcode.h 2017-03-10 17:32:07 UTC (rev 213713)
+++ trunk/Source/JavaScriptCore/b3/B3Opcode.h 2017-03-10 17:49:42 UTC (rev 213714)
</span><span class="lines">@@ -1,5 +1,5 @@
</span><span class="cx"> /*
</span><del>- * Copyright (C) 2015-2016 Apple Inc. All rights reserved.
</del><ins>+ * Copyright (C) 2015-2017 Apple Inc. All rights reserved.
</ins><span class="cx"> *
</span><span class="cx"> * Redistribution and use in source and binary forms, with or without
</span><span class="cx"> * modification, are permitted provided that the following conditions
</span><span class="lines">@@ -28,6 +28,7 @@
</span><span class="cx"> #if ENABLE(B3_JIT)
</span><span class="cx">
</span><span class="cx"> #include "B3Type.h"
</span><ins>+#include "B3Width.h"
</ins><span class="cx"> #include <wtf/Optional.h>
</span><span class="cx"> #include <wtf/StdLibExtras.h>
</span><span class="cx">
</span><span class="lines">@@ -82,7 +83,6 @@
</span><span class="cx"> Mod, // All bets are off as to what will happen when you execute this for -2^31%-1 and x%0.
</span><span class="cx"> UMod,
</span><span class="cx">
</span><del>-
</del><span class="cx"> // Polymorphic negation. Note that we only need this for floating point, since integer negation
</span><span class="cx"> // is exactly like Sub(0, x). But that's not true for floating point. Sub(0, 0) is 0, while
</span><span class="cx"> // Neg(0) is -0. Also, we canonicalize Sub(0, x) into Neg(x) in case of integers.
</span><span class="lines">@@ -162,6 +162,110 @@
</span><span class="cx"> Store16,
</span><span class="cx"> // This is a polymorphic store for Int32, Int64, Float, and Double.
</span><span class="cx"> Store,
</span><ins>+
+ // Atomic compare and swap that returns a boolean. May choose to do nothing and return false. You can
+ // usually assume that this is faster and results in less code than AtomicStrongCAS, though that's
+ // not necessarily true on Intel, if instruction selection does its job. Imagine that this opcode is
+ // as if you did this atomically:
+ //
+ // template<typename T>
+ // bool AtomicWeakCAS(T expectedValue, T newValue, T* ptr)
+ // {
+ // if (!rand())
+ // return false; // Real world example of this: context switch on ARM while doing CAS.
+ // if (*ptr != expectedValue)
+ // return false;
+ // *ptr = newValue;
+ // return true;
+ // }
+ //
+ // Note that all atomics put the pointer last to be consistent with how loads and stores work. This
+ // is a goofy tradition, but it's harmless, and better than being inconsistent.
+ //
+ // Note that weak CAS has no fencing guarantees when it fails. This means that the following
+ // transformation is always valid:
+ //
+ // Before:
+ //
+ // Branch(AtomicWeakCAS(expected, new, ptr))
+ // Successors: Then:#success, Else:#fail
+ //
+ // After:
+ //
+ // Branch(Equal(Load(ptr), expected))
+ // Successors: Then:#attempt, Else:#fail
+ // BB#attempt:
+ // Branch(AtomicWeakCAS(expected, new, ptr))
+ // Successors: Then:#success, Else:#fail
+ //
+ // Both kinds of CAS for non-canonical widths (Width8 and Width16) ignore the irrelevant bits of the
+ // input.
+ AtomicWeakCAS,
+
+ // Atomic compare and swap that returns the old value. Does not have the nondeterminism of WeakCAS.
+ // This is a bit more code and a bit slower in some cases, though not by a lot. Imagine that this
+ // opcode is as if you did this atomically:
+ //
+ // template<typename T>
+ // T AtomicStrongCAS(T expectedValue, T newValue, T* ptr)
+ // {
+ // T oldValue = *ptr;
+ // if (oldValue == expectedValue)
+ // *ptr = newValue;
+ // return oldValue
+ // }
+ //
+ // AtomicStrongCAS sign-extends its result for subwidth operations.
+ //
+ // Note that AtomicWeakCAS and AtomicStrongCAS sort of have this kind of equivalence:
+ //
+ // AtomicWeakCAS(@exp, @new, @ptr) == Equal(AtomicStrongCAS(@exp, @new, @ptr), @exp)
+ //
+ // Assuming that the WeakCAS does not spuriously fail, of course.
+ AtomicStrongCAS,
+
+ // Atomically ___ a memory location and return the old value. Syntax:
+ //
+ // @oldValue = AtomicXchg___(@operand, @ptr)
+ //
+ // For non-canonical widths (Width8 and Width16), these return sign-extended results and ignore the
+ // irrelevant bits of their inputs.
+ AtomicXchgAdd,
+ AtomicXchgAnd,
+ AtomicXchgOr,
+ AtomicXchgSub,
+ AtomicXchgXor,
+
+ // FIXME: Maybe we should have AtomicXchgNeg.
+ // https://bugs.webkit.org/show_bug.cgi?id=169252
+
+ // Atomically exchange a value with a memory location. Syntax:
+ //
+ // @oldValue = AtomicXchg(@newValue, @ptr)
+ AtomicXchg,
+
+ // Introduce an invisible dependency for blocking motion of loads with respect to each other. Syntax:
+ //
+ // @result = Depend(@phantom)
+ //
+ // This is eventually codegenerated to have local semantics as if we did:
+ //
+ // @result = $0
+ //
+ // But it ensures that the users of @result cannot execute until @phantom is computed.
+ //
+ // The compiler is not allowed to reason about the fact that Depend codegenerates this way. Any kind
+ // of transformation or analysis that relies on the insight that Depend is really zero is unsound,
+ // because it unlocks reordering of users of @result and @phantom.
+ //
+ // On X86, this is lowered to a load-load fence and @result uses @phantom directly.
+ //
+ // On ARM, this is lowered as if like:
+ //
+ // @result = BitXor(@phantom, @phantom)
+ //
+ // Except that the compiler never gets an opportunity to simplify out the BitXor.
+ Depend,
</ins><span class="cx">
</span><span class="cx"> // This is used to compute the actual address of a Wasm memory operation. It takes an IntPtr
</span><span class="cx"> // and a pinned register then computes the appropriate IntPtr address. For the use-case of
</span><span class="lines">@@ -301,6 +405,112 @@
</span><span class="cx"> }
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+inline bool isLoad(Opcode opcode)
+{
+ switch (opcode) {
+ case Load8Z:
+ case Load8S:
+ case Load16Z:
+ case Load16S:
+ case Load:
+ return true;
+ default:
+ return false;
+ }
+}
+
+inline bool isStore(Opcode opcode)
+{
+ switch (opcode) {
+ case Store8:
+ case Store16:
+ case Store:
+ return true;
+ default:
+ return false;
+ }
+}
+
+inline bool isLoadStore(Opcode opcode)
+{
+ switch (opcode) {
+ case Load8Z:
+ case Load8S:
+ case Load16Z:
+ case Load16S:
+ case Load:
+ case Store8:
+ case Store16:
+ case Store:
+ return true;
+ default:
+ return false;
+ }
+}
+
+inline bool isAtomic(Opcode opcode)
+{
+ switch (opcode) {
+ case AtomicWeakCAS:
+ case AtomicStrongCAS:
+ case AtomicXchgAdd:
+ case AtomicXchgAnd:
+ case AtomicXchgOr:
+ case AtomicXchgSub:
+ case AtomicXchgXor:
+ case AtomicXchg:
+ return true;
+ default:
+ return false;
+ }
+}
+
+inline bool isAtomicCAS(Opcode opcode)
+{
+ switch (opcode) {
+ case AtomicWeakCAS:
+ case AtomicStrongCAS:
+ return true;
+ default:
+ return false;
+ }
+}
+
+inline bool isAtomicXchg(Opcode opcode)
+{
+ switch (opcode) {
+ case AtomicXchgAdd:
+ case AtomicXchgAnd:
+ case AtomicXchgOr:
+ case AtomicXchgSub:
+ case AtomicXchgXor:
+ case AtomicXchg:
+ return true;
+ default:
+ return false;
+ }
+}
+
+inline bool isMemoryAccess(Opcode opcode)
+{
+ return isAtomic(opcode) || isLoadStore(opcode);
+}
+
+inline Opcode signExtendOpcode(Width width)
+{
+ switch (width) {
+ case Width8:
+ return SExt8;
+ case Width16:
+ return SExt16;
+ default:
+ RELEASE_ASSERT_NOT_REACHED();
+ return Oops;
+ }
+}
+
+JS_EXPORT_PRIVATE Opcode storeOpcode(Bank bank, Width width);
+
</ins><span class="cx"> } } // namespace JSC::B3
</span><span class="cx">
</span><span class="cx"> namespace WTF {
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreb3B3Procedurecpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/b3/B3Procedure.cpp (213713 => 213714)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/b3/B3Procedure.cpp 2017-03-10 17:32:07 UTC (rev 213713)
+++ trunk/Source/JavaScriptCore/b3/B3Procedure.cpp 2017-03-10 17:49:42 UTC (rev 213714)
</span><span class="lines">@@ -202,6 +202,8 @@
</span><span class="cx"> }
</span><span class="cx"> dataLog(" ", deepDump(*this, value), "\n");
</span><span class="cx"> }
</span><ins>+ if (hasQuirks())
+ out.print("Has Quirks: True\n");
</ins><span class="cx"> if (variables().size()) {
</span><span class="cx"> out.print("Variables:\n");
</span><span class="cx"> for (Variable* variable : variables())
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreb3B3Procedureh"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/b3/B3Procedure.h (213713 => 213714)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/b3/B3Procedure.h 2017-03-10 17:32:07 UTC (rev 213713)
+++ trunk/Source/JavaScriptCore/b3/B3Procedure.h 2017-03-10 17:49:42 UTC (rev 213714)
</span><span class="lines">@@ -1,5 +1,5 @@
</span><span class="cx"> /*
</span><del>- * Copyright (C) 2015-2016 Apple Inc. All rights reserved.
</del><ins>+ * Copyright (C) 2015-2017 Apple Inc. All rights reserved.
</ins><span class="cx"> *
</span><span class="cx"> * Redistribution and use in source and binary forms, with or without
</span><span class="cx"> * modification, are permitted provided that the following conditions
</span><span class="lines">@@ -117,6 +117,7 @@
</span><span class="cx"> Value* addIntConstant(Origin, Type, int64_t value);
</span><span class="cx"> Value* addIntConstant(Value*, int64_t value);
</span><span class="cx">
</span><ins>+ // You're guaranteed that bottom is zero.
</ins><span class="cx"> Value* addBottom(Origin, Type);
</span><span class="cx"> Value* addBottom(Value*);
</span><span class="cx">
</span><span class="lines">@@ -195,6 +196,17 @@
</span><span class="cx"> // alive. Great for compiler-generated data sections, like switch jump tables and constant pools.
</span><span class="cx"> // This returns memory that has been zero-initialized.
</span><span class="cx"> JS_EXPORT_PRIVATE void* addDataSection(size_t);
</span><ins>+
+ // Some operations are specified in B3 IR to behave one way but on this given CPU they behave a
+ // different way. When true, those B3 IR ops switch to behaving the CPU way, and the optimizer may
+ // start taking advantage of it.
+ //
+ // One way to think of it is like this. Imagine that you find that the cleanest way of lowering
+ // something in lowerMacros is to unconditionally replace one opcode with another. This is a shortcut
+ // where you instead keep the same opcode, but rely on the opcode's meaning changes once lowerMacros
+ // sets hasQuirks.
+ bool hasQuirks() const { return m_hasQuirks; }
+ void setHasQuirks(bool value) { m_hasQuirks = value; }
</ins><span class="cx">
</span><span class="cx"> OpaqueByproducts& byproducts() { return *m_byproducts; }
</span><span class="cx">
</span><span class="lines">@@ -252,6 +264,7 @@
</span><span class="cx"> RefPtr<SharedTask<void(PrintStream&, Origin)>> m_originPrinter;
</span><span class="cx"> const void* m_frontendData;
</span><span class="cx"> PCToOriginMap m_pcToOriginMap;
</span><ins>+ bool m_hasQuirks { false };
</ins><span class="cx"> };
</span><span class="cx">
</span><span class="cx"> } } // namespace JSC::B3
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreb3B3PureCSEcpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/b3/B3PureCSE.cpp (213713 => 213714)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/b3/B3PureCSE.cpp 2017-03-10 17:32:07 UTC (rev 213713)
+++ trunk/Source/JavaScriptCore/b3/B3PureCSE.cpp 2017-03-10 17:49:42 UTC (rev 213714)
</span><span class="lines">@@ -1,5 +1,5 @@
</span><span class="cx"> /*
</span><del>- * Copyright (C) 2016 Apple Inc. All rights reserved.
</del><ins>+ * Copyright (C) 2016-2017 Apple Inc. All rights reserved.
</ins><span class="cx"> *
</span><span class="cx"> * Redistribution and use in source and binary forms, with or without
</span><span class="cx"> * modification, are permitted provided that the following conditions
</span><span class="lines">@@ -29,6 +29,7 @@
</span><span class="cx"> #if ENABLE(B3_JIT)
</span><span class="cx">
</span><span class="cx"> #include "B3Dominators.h"
</span><ins>+#include "B3PhaseScope.h"
</ins><span class="cx"> #include "B3Value.h"
</span><span class="cx">
</span><span class="cx"> namespace JSC { namespace B3 {
</span><span class="lines">@@ -89,6 +90,23 @@
</span><span class="cx"> return false;
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+bool pureCSE(Procedure& proc)
+{
+ PhaseScope phaseScope(proc, "pureCSE");
+
+ Dominators& dominators = proc.dominators();
+ PureCSE pureCSE;
+ bool result = false;
+ for (BasicBlock* block : proc.blocksInPreOrder()) {
+ for (Value* value : *block) {
+ result |= value->performSubstitution();
+ result |= pureCSE.process(value, dominators);
+ }
+ }
+
+ return result;
+}
+
</ins><span class="cx"> } } // namespace JSC::B3
</span><span class="cx">
</span><span class="cx"> #endif // ENABLE(B3_JIT)
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreb3B3PureCSEh"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/b3/B3PureCSE.h (213713 => 213714)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/b3/B3PureCSE.h 2017-03-10 17:32:07 UTC (rev 213713)
+++ trunk/Source/JavaScriptCore/b3/B3PureCSE.h 2017-03-10 17:49:42 UTC (rev 213714)
</span><span class="lines">@@ -56,6 +56,8 @@
</span><span class="cx"> HashMap<ValueKey, Matches> m_map;
</span><span class="cx"> };
</span><span class="cx">
</span><ins>+bool pureCSE(Procedure&);
+
</ins><span class="cx"> } } // namespace JSC::B3
</span><span class="cx">
</span><span class="cx"> #endif // ENABLE(B3_JIT)
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreb3B3ReduceStrengthcpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/b3/B3ReduceStrength.cpp (213713 => 213714)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/b3/B3ReduceStrength.cpp 2017-03-10 17:32:07 UTC (rev 213713)
+++ trunk/Source/JavaScriptCore/b3/B3ReduceStrength.cpp 2017-03-10 17:49:42 UTC (rev 213714)
</span><span class="lines">@@ -1,5 +1,5 @@
</span><span class="cx"> /*
</span><del>- * Copyright (C) 2015-2016 Apple Inc. All rights reserved.
</del><ins>+ * Copyright (C) 2015-2017 Apple Inc. All rights reserved.
</ins><span class="cx"> *
</span><span class="cx"> * Redistribution and use in source and binary forms, with or without
</span><span class="cx"> * modification, are permitted provided that the following conditions
</span><span class="lines">@@ -28,12 +28,13 @@
</span><span class="cx">
</span><span class="cx"> #if ENABLE(B3_JIT)
</span><span class="cx">
</span><ins>+#include "B3AtomicValue.h"
</ins><span class="cx"> #include "B3BasicBlockInlines.h"
</span><span class="cx"> #include "B3BlockInsertionSet.h"
</span><span class="cx"> #include "B3ComputeDivisionMagic.h"
</span><span class="cx"> #include "B3Dominators.h"
</span><span class="cx"> #include "B3InsertionSetInlines.h"
</span><del>-#include "B3MemoryValue.h"
</del><ins>+#include "B3MemoryValueInlines.h"
</ins><span class="cx"> #include "B3PhaseScope.h"
</span><span class="cx"> #include "B3PhiChildren.h"
</span><span class="cx"> #include "B3ProcedureInlines.h"
</span><span class="lines">@@ -602,7 +603,7 @@
</span><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> break;
</span><del>-
</del><ins>+
</ins><span class="cx"> case Neg:
</span><span class="cx"> // Turn this: Neg(constant)
</span><span class="cx"> // Into this: -constant
</span><span class="lines">@@ -1295,6 +1296,16 @@
</span><span class="cx"> break;
</span><span class="cx"> }
</span><span class="cx"> }
</span><ins>+
+ if (!m_proc.hasQuirks()) {
+ // Turn this: SExt8(AtomicXchg___)
+ // Into this: AtomicXchg___
+ if (isAtomicXchg(m_value->child(0)->opcode())
+ && m_value->child(0)->as<AtomicValue>()->accessWidth() == Width8) {
+ replaceWithIdentity(m_value->child(0));
+ break;
+ }
+ }
</ins><span class="cx"> break;
</span><span class="cx">
</span><span class="cx"> case SExt16:
</span><span class="lines">@@ -1343,6 +1354,16 @@
</span><span class="cx"> break;
</span><span class="cx"> }
</span><span class="cx"> }
</span><ins>+
+ if (!m_proc.hasQuirks()) {
+ // Turn this: SExt16(AtomicXchg___)
+ // Into this: AtomicXchg___
+ if (isAtomicXchg(m_value->child(0)->opcode())
+ && m_value->child(0)->as<AtomicValue>()->accessWidth() == Width16) {
+ replaceWithIdentity(m_value->child(0));
+ break;
+ }
+ }
</ins><span class="cx"> break;
</span><span class="cx">
</span><span class="cx"> case SExt32:
</span><span class="lines">@@ -2101,22 +2122,25 @@
</span><span class="cx"> ASSERT(m_value->numChildren() >= 2);
</span><span class="cx">
</span><span class="cx"> // Leave it alone if the right child is a constant.
</span><del>- if (m_value->child(1)->isConstant())
</del><ins>+ if (m_value->child(1)->isConstant()
+ || m_value->child(0)->opcode() == AtomicStrongCAS)
</ins><span class="cx"> return;
</span><span class="cx">
</span><del>- if (m_value->child(0)->isConstant()) {
</del><ins>+ auto swap = [&] () {
</ins><span class="cx"> std::swap(m_value->child(0), m_value->child(1));
</span><span class="cx"> m_changed = true;
</span><del>- return;
- }
-
</del><ins>+ };
+
+ if (m_value->child(0)->isConstant())
+ return swap();
+
+ if (m_value->child(1)->opcode() == AtomicStrongCAS)
+ return swap();
+
</ins><span class="cx"> // Sort the operands. This is an important canonicalization. We use the index instead of
</span><span class="cx"> // the address to make this at least slightly deterministic.
</span><del>- if (m_value->child(0)->index() > m_value->child(1)->index()) {
- std::swap(m_value->child(0), m_value->child(1));
- m_changed = true;
- return;
- }
</del><ins>+ if (m_value->child(0)->index() > m_value->child(1)->index())
+ return swap();
</ins><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> // FIXME: This should really be a forward analysis. Instead, we uses a bounded-search backwards
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreb3B3Validatecpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/b3/B3Validate.cpp (213713 => 213714)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/b3/B3Validate.cpp 2017-03-10 17:32:07 UTC (rev 213713)
+++ trunk/Source/JavaScriptCore/b3/B3Validate.cpp 2017-03-10 17:49:42 UTC (rev 213714)
</span><span class="lines">@@ -1,5 +1,5 @@
</span><span class="cx"> /*
</span><del>- * Copyright (C) 2015-2016 Apple Inc. All rights reserved.
</del><ins>+ * Copyright (C) 2015-2017 Apple Inc. All rights reserved.
</ins><span class="cx"> *
</span><span class="cx"> * Redistribution and use in source and binary forms, with or without
</span><span class="cx"> * modification, are permitted provided that the following conditions
</span><span class="lines">@@ -30,6 +30,7 @@
</span><span class="cx">
</span><span class="cx"> #include "AirCode.h"
</span><span class="cx"> #include "B3ArgumentRegValue.h"
</span><ins>+#include "B3AtomicValue.h"
</ins><span class="cx"> #include "B3BasicBlockInlines.h"
</span><span class="cx"> #include "B3Dominators.h"
</span><span class="cx"> #include "B3MemoryValue.h"
</span><span class="lines">@@ -346,6 +347,7 @@
</span><span class="cx"> VALIDATE(value->numChildren() == 1, ("At ", *value));
</span><span class="cx"> VALIDATE(value->child(0)->type() == pointerType(), ("At ", *value));
</span><span class="cx"> VALIDATE(value->type() == Int32, ("At ", *value));
</span><ins>+ validateFence(value);
</ins><span class="cx"> validateStackAccess(value);
</span><span class="cx"> break;
</span><span class="cx"> case Load:
</span><span class="lines">@@ -353,6 +355,7 @@
</span><span class="cx"> VALIDATE(value->numChildren() == 1, ("At ", *value));
</span><span class="cx"> VALIDATE(value->child(0)->type() == pointerType(), ("At ", *value));
</span><span class="cx"> VALIDATE(value->type() != Void, ("At ", *value));
</span><ins>+ validateFence(value);
</ins><span class="cx"> validateStackAccess(value);
</span><span class="cx"> break;
</span><span class="cx"> case Store8:
</span><span class="lines">@@ -362,6 +365,7 @@
</span><span class="cx"> VALIDATE(value->child(0)->type() == Int32, ("At ", *value));
</span><span class="cx"> VALIDATE(value->child(1)->type() == pointerType(), ("At ", *value));
</span><span class="cx"> VALIDATE(value->type() == Void, ("At ", *value));
</span><ins>+ validateFence(value);
</ins><span class="cx"> validateStackAccess(value);
</span><span class="cx"> break;
</span><span class="cx"> case Store:
</span><span class="lines">@@ -369,8 +373,49 @@
</span><span class="cx"> VALIDATE(value->numChildren() == 2, ("At ", *value));
</span><span class="cx"> VALIDATE(value->child(1)->type() == pointerType(), ("At ", *value));
</span><span class="cx"> VALIDATE(value->type() == Void, ("At ", *value));
</span><ins>+ validateFence(value);
</ins><span class="cx"> validateStackAccess(value);
</span><span class="cx"> break;
</span><ins>+ case AtomicWeakCAS:
+ VALIDATE(!value->kind().isChill(), ("At ", *value));
+ VALIDATE(value->numChildren() == 3, ("At ", *value));
+ VALIDATE(value->type() == Int32, ("At ", *value));
+ VALIDATE(value->child(0)->type() == value->child(1)->type(), ("At ", *value));
+ VALIDATE(isInt(value->child(0)->type()), ("At ", *value));
+ VALIDATE(value->child(2)->type() == pointerType(), ("At ", *value));
+ validateAtomic(value);
+ validateStackAccess(value);
+ break;
+ case AtomicStrongCAS:
+ VALIDATE(!value->kind().isChill(), ("At ", *value));
+ VALIDATE(value->numChildren() == 3, ("At ", *value));
+ VALIDATE(value->type() == value->child(0)->type(), ("At ", *value));
+ VALIDATE(value->type() == value->child(1)->type(), ("At ", *value));
+ VALIDATE(isInt(value->type()), ("At ", *value));
+ VALIDATE(value->child(2)->type() == pointerType(), ("At ", *value));
+ validateAtomic(value);
+ validateStackAccess(value);
+ break;
+ case AtomicXchgAdd:
+ case AtomicXchgAnd:
+ case AtomicXchgOr:
+ case AtomicXchgSub:
+ case AtomicXchgXor:
+ case AtomicXchg:
+ VALIDATE(!value->kind().isChill(), ("At ", *value));
+ VALIDATE(value->numChildren() == 2, ("At ", *value));
+ VALIDATE(value->type() == value->child(0)->type(), ("At ", *value));
+ VALIDATE(isInt(value->type()), ("At ", *value));
+ VALIDATE(value->child(1)->type() == pointerType(), ("At ", *value));
+ validateAtomic(value);
+ validateStackAccess(value);
+ break;
+ case Depend:
+ VALIDATE(!value->kind().hasExtraBits(), ("At ", *value));
+ VALIDATE(value->numChildren() == 1, ("At ", *value));
+ VALIDATE(value->type() == value->child(0)->type(), ("At ", *value));
+ VALIDATE(isInt(value->type()), ("At ", *value));
+ break;
</ins><span class="cx"> case WasmAddress:
</span><span class="cx"> VALIDATE(!value->kind().hasExtraBits(), ("At ", *value));
</span><span class="cx"> VALIDATE(value->numChildren() == 1, ("At ", *value));
</span><span class="lines">@@ -537,6 +582,20 @@
</span><span class="cx"> break;
</span><span class="cx"> }
</span><span class="cx"> }
</span><ins>+
+ void validateFence(Value* value)
+ {
+ MemoryValue* memory = value->as<MemoryValue>();
+ if (memory->hasFence())
+ VALIDATE(memory->accessBank() == GP, ("Fence at ", *memory));
+ }
+
+ void validateAtomic(Value* value)
+ {
+ AtomicValue* atomic = value->as<AtomicValue>();
+
+ VALIDATE(bestType(GP, atomic->accessWidth()) == atomic->accessType(), ("At ", *value));
+ }
</ins><span class="cx">
</span><span class="cx"> void validateStackAccess(Value* value)
</span><span class="cx"> {
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreb3B3Valuecpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/b3/B3Value.cpp (213713 => 213714)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/b3/B3Value.cpp 2017-03-10 17:32:07 UTC (rev 213713)
+++ trunk/Source/JavaScriptCore/b3/B3Value.cpp 2017-03-10 17:49:42 UTC (rev 213714)
</span><span class="lines">@@ -1,5 +1,5 @@
</span><span class="cx"> /*
</span><del>- * Copyright (C) 2015-2016 Apple Inc. All rights reserved.
</del><ins>+ * Copyright (C) 2015-2017 Apple Inc. All rights reserved.
</ins><span class="cx"> *
</span><span class="cx"> * Redistribution and use in source and binary forms, with or without
</span><span class="cx"> * modification, are permitted provided that the following conditions
</span><span class="lines">@@ -29,6 +29,7 @@
</span><span class="cx"> #if ENABLE(B3_JIT)
</span><span class="cx">
</span><span class="cx"> #include "B3ArgumentRegValue.h"
</span><ins>+#include "B3AtomicValue.h"
</ins><span class="cx"> #include "B3BasicBlockInlines.h"
</span><span class="cx"> #include "B3BottomProvider.h"
</span><span class="cx"> #include "B3CCallValue.h"
</span><span class="lines">@@ -510,6 +511,7 @@
</span><span class="cx"> case AboveEqual:
</span><span class="cx"> case BelowEqual:
</span><span class="cx"> case EqualOrUnordered:
</span><ins>+ case AtomicWeakCAS:
</ins><span class="cx"> return true;
</span><span class="cx"> case Phi:
</span><span class="cx"> // FIXME: We should have a story here.
</span><span class="lines">@@ -589,6 +591,7 @@
</span><span class="cx"> case BelowEqual:
</span><span class="cx"> case EqualOrUnordered:
</span><span class="cx"> case Select:
</span><ins>+ case Depend:
</ins><span class="cx"> break;
</span><span class="cx"> case Div:
</span><span class="cx"> case UDiv:
</span><span class="lines">@@ -600,16 +603,44 @@
</span><span class="cx"> case Load8S:
</span><span class="cx"> case Load16Z:
</span><span class="cx"> case Load16S:
</span><del>- case Load:
- result.reads = as<MemoryValue>()->range();
</del><ins>+ case Load: {
+ const MemoryValue* memory = as<MemoryValue>();
+ result.reads = memory->range();
+ if (memory->hasFence()) {
+ result.writes = memory->fenceRange();
+ result.fence = true;
+ }
</ins><span class="cx"> result.controlDependent = true;
</span><span class="cx"> break;
</span><ins>+ }
</ins><span class="cx"> case Store8:
</span><span class="cx"> case Store16:
</span><del>- case Store:
- result.writes = as<MemoryValue>()->range();
</del><ins>+ case Store: {
+ const MemoryValue* memory = as<MemoryValue>();
+ result.writes = memory->range();
+ if (memory->hasFence()) {
+ result.reads = memory->fenceRange();
+ result.fence = true;
+ }
</ins><span class="cx"> result.controlDependent = true;
</span><span class="cx"> break;
</span><ins>+ }
+ case AtomicWeakCAS:
+ case AtomicStrongCAS:
+ case AtomicXchgAdd:
+ case AtomicXchgAnd:
+ case AtomicXchgOr:
+ case AtomicXchgSub:
+ case AtomicXchgXor:
+ case AtomicXchg: {
+ const AtomicValue* atomic = as<AtomicValue>();
+ result.reads = atomic->range() | atomic->fenceRange();
+ result.writes = atomic->range() | atomic->fenceRange();
+ if (atomic->hasFence())
+ result.fence = true;
+ result.controlDependent = true;
+ break;
+ }
</ins><span class="cx"> case WasmAddress:
</span><span class="cx"> result.readsPinned = true;
</span><span class="cx"> break;
</span><span class="lines">@@ -617,18 +648,7 @@
</span><span class="cx"> const FenceValue* fence = as<FenceValue>();
</span><span class="cx"> result.reads = fence->read;
</span><span class="cx"> result.writes = fence->write;
</span><del>-
- // Prevent killing of fences that claim not to write anything. It's a bit weird that we use
- // local state as the way to do this, but it happens to work: we must assume that we cannot
- // kill writesLocalState unless we understands exactly what the instruction is doing (like
- // the way that fixSSA understands Set/Get and the way that reduceStrength and others
- // understand Upsilon). This would only become a problem if we had some analysis that was
- // looking to use the writesLocalState bit to invalidate a CSE over local state operations.
- // Then a Fence would look block, say, the elimination of a redundant Get. But it like
- // that's not at all how our optimizations for Set/Get/Upsilon/Phi work - they grok their
- // operations deeply enough that they have no need to check this bit - so this cheat is
- // fine.
- result.writesLocalState = true;
</del><ins>+ result.fence = true;
</ins><span class="cx"> break;
</span><span class="cx"> }
</span><span class="cx"> case CCall:
</span><span class="lines">@@ -694,6 +714,7 @@
</span><span class="cx"> case Check:
</span><span class="cx"> case BitwiseCast:
</span><span class="cx"> case Neg:
</span><ins>+ case Depend:
</ins><span class="cx"> return ValueKey(kind(), type(), child(0));
</span><span class="cx"> case Add:
</span><span class="cx"> case Sub:
</span><span class="lines">@@ -746,12 +767,16 @@
</span><span class="cx"> }
</span><span class="cx"> }
</span><span class="cx">
</span><del>-void Value::performSubstitution()
</del><ins>+bool Value::performSubstitution()
</ins><span class="cx"> {
</span><ins>+ bool result = false;
</ins><span class="cx"> for (Value*& child : children()) {
</span><del>- while (child->opcode() == Identity)
</del><ins>+ while (child->opcode() == Identity) {
+ result = true;
</ins><span class="cx"> child = child->child(0);
</span><ins>+ }
</ins><span class="cx"> }
</span><ins>+ return result;
</ins><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> bool Value::isFree() const
</span><span class="lines">@@ -801,6 +826,7 @@
</span><span class="cx"> case CheckAdd:
</span><span class="cx"> case CheckSub:
</span><span class="cx"> case CheckMul:
</span><ins>+ case Depend:
</ins><span class="cx"> return firstChild->type();
</span><span class="cx"> case FramePointer:
</span><span class="cx"> return pointerType();
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreb3B3Valueh"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/b3/B3Value.h (213713 => 213714)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/b3/B3Value.h 2017-03-10 17:32:07 UTC (rev 213713)
+++ trunk/Source/JavaScriptCore/b3/B3Value.h 2017-03-10 17:49:42 UTC (rev 213714)
</span><span class="lines">@@ -68,6 +68,11 @@
</span><span class="cx">
</span><span class="cx"> Opcode opcode() const { return kind().opcode(); }
</span><span class="cx">
</span><ins>+ // Note that the kind is meant to be immutable. Do this when you know that this is safe. It's not
+ // usually safe.
+ void setKindUnsafely(Kind kind) { m_kind = kind; }
+ void setOpcodeUnsafely(Opcode opcode) { m_kind.setOpcode(opcode); }
+
</ins><span class="cx"> // It's good practice to mirror Kind methods here, so you can say value->isBlah()
</span><span class="cx"> // instead of value->kind().isBlah().
</span><span class="cx"> bool isChill() const { return kind().isChill(); }
</span><span class="lines">@@ -107,6 +112,8 @@
</span><span class="cx"> // values with Identities on bottom constants. For this reason, this takes a callback that is
</span><span class="cx"> // responsible for creating bottoms. There's a utility for this, see B3BottomProvider.h. You
</span><span class="cx"> // can also access that utility using replaceWithBottom(InsertionSet&, size_t).
</span><ins>+ //
+ // You're guaranteed that bottom is zero.
</ins><span class="cx"> template<typename BottomProvider>
</span><span class="cx"> void replaceWithBottom(const BottomProvider&);
</span><span class="cx">
</span><span class="lines">@@ -256,7 +263,7 @@
</span><span class="cx"> // Makes sure that none of the children are Identity's. If a child points to Identity, this will
</span><span class="cx"> // repoint it at the Identity's child. For simplicity, this will follow arbitrarily long chains
</span><span class="cx"> // of Identity's.
</span><del>- void performSubstitution();
</del><ins>+ bool performSubstitution();
</ins><span class="cx">
</span><span class="cx"> // Free values are those whose presence is guaranteed not to hurt code. We consider constants,
</span><span class="cx"> // Identities, and Nops to be free. Constants are free because we hoist them to an optimal place.
</span><span class="lines">@@ -321,6 +328,7 @@
</span><span class="cx"> case IToF:
</span><span class="cx"> case BitwiseCast:
</span><span class="cx"> case Branch:
</span><ins>+ case Depend:
</ins><span class="cx"> if (UNLIKELY(numArgs != 1))
</span><span class="cx"> badKind(kind, numArgs);
</span><span class="cx"> break;
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreb3B3ValueReph"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/b3/B3ValueRep.h (213713 => 213714)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/b3/B3ValueRep.h 2017-03-10 17:32:07 UTC (rev 213713)
+++ trunk/Source/JavaScriptCore/b3/B3ValueRep.h 2017-03-10 17:49:42 UTC (rev 213714)
</span><span class="lines">@@ -70,7 +70,7 @@
</span><span class="cx"> // that the def happens before any of the effects of the stackmap. This is only valid for
</span><span class="cx"> // the result constraint of a Patchpoint.
</span><span class="cx"> SomeEarlyRegister,
</span><del>-
</del><ins>+
</ins><span class="cx"> // As an input representation, this forces a particular register. As an output
</span><span class="cx"> // representation, this tells us what register B3 picked.
</span><span class="cx"> Register,
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreb3B3Widthcpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/b3/B3Width.cpp (213713 => 213714)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/b3/B3Width.cpp 2017-03-10 17:32:07 UTC (rev 213713)
+++ trunk/Source/JavaScriptCore/b3/B3Width.cpp 2017-03-10 17:49:42 UTC (rev 213714)
</span><span class="lines">@@ -30,6 +30,38 @@
</span><span class="cx">
</span><span class="cx"> #include <wtf/PrintStream.h>
</span><span class="cx">
</span><ins>+namespace JSC { namespace B3 {
+
+Type bestType(Bank bank, Width width)
+{
+ switch (width) {
+ case Width8:
+ case Width16:
+ case Width32:
+ switch (bank) {
+ case GP:
+ return Int32;
+ case FP:
+ return Float;
+ }
+ RELEASE_ASSERT_NOT_REACHED();
+ return Void;
+ case Width64:
+ switch (bank) {
+ case GP:
+ return Int64;
+ case FP:
+ return Double;
+ }
+ RELEASE_ASSERT_NOT_REACHED();
+ return Void;
+ }
+ RELEASE_ASSERT_NOT_REACHED();
+ return Void;
+}
+
+} } // namespace JSC::B3
+
</ins><span class="cx"> namespace WTF {
</span><span class="cx">
</span><span class="cx"> void printInternal(PrintStream& out, JSC::B3::Width width)
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreb3B3Widthh"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/b3/B3Width.h (213713 => 213714)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/b3/B3Width.h 2017-03-10 17:32:07 UTC (rev 213713)
+++ trunk/Source/JavaScriptCore/b3/B3Width.h 2017-03-10 17:49:42 UTC (rev 213714)
</span><span class="lines">@@ -65,8 +65,21 @@
</span><span class="cx"> return Width64;
</span><span class="cx"> }
</span><span class="cx"> ASSERT_NOT_REACHED();
</span><ins>+ return Width8;
</ins><span class="cx"> }
</span><span class="cx">
</span><ins>+inline Width canonicalWidth(Width width)
+{
+ return std::max(Width32, width);
+}
+
+inline bool isCanonicalWidth(Width width)
+{
+ return width >= Width32;
+}
+
+Type bestType(Bank bank, Width width);
+
</ins><span class="cx"> inline Width conservativeWidth(Bank bank)
</span><span class="cx"> {
</span><span class="cx"> return bank == GP ? pointerWidth() : Width64;
</span><span class="lines">@@ -98,6 +111,20 @@
</span><span class="cx"> }
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+inline uint64_t mask(Width width)
+{
+ switch (width) {
+ case Width8:
+ return 0x00000000000000ffllu;
+ case Width16:
+ return 0x000000000000ffffllu;
+ case Width32:
+ return 0x00000000ffffffffllu;
+ case Width64:
+ return 0xffffffffffffffffllu;
+ }
+}
+
</ins><span class="cx"> } } // namespace JSC::B3
</span><span class="cx">
</span><span class="cx"> namespace WTF {
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreb3airAirArgcpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/b3/air/AirArg.cpp (213713 => 213714)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/b3/air/AirArg.cpp 2017-03-10 17:32:07 UTC (rev 213713)
+++ trunk/Source/JavaScriptCore/b3/air/AirArg.cpp 2017-03-10 17:49:42 UTC (rev 213714)
</span><span class="lines">@@ -102,6 +102,7 @@
</span><span class="cx"> case RelCond:
</span><span class="cx"> case ResCond:
</span><span class="cx"> case DoubleCond:
</span><ins>+ case StatusCond:
</ins><span class="cx"> case WidthArg:
</span><span class="cx"> result += static_cast<unsigned>(m_offset);
</span><span class="cx"> break;
</span><span class="lines">@@ -110,6 +111,9 @@
</span><span class="cx"> result += static_cast<unsigned>(m_offset);
</span><span class="cx"> result += static_cast<unsigned>(m_offset >> 32);
</span><span class="cx"> break;
</span><ins>+ case SimpleAddr:
+ result += m_base.internalValue();
+ break;
</ins><span class="cx"> case Addr:
</span><span class="cx"> result += m_offset;
</span><span class="cx"> result += m_base.internalValue();
</span><span class="lines">@@ -150,6 +154,9 @@
</span><span class="cx"> case BitImm64:
</span><span class="cx"> out.printf("$0x%llx", static_cast<long long unsigned>(m_offset));
</span><span class="cx"> return;
</span><ins>+ case SimpleAddr:
+ out.print("(", base(), ")");
+ return;
</ins><span class="cx"> case Addr:
</span><span class="cx"> if (offset())
</span><span class="cx"> out.print(offset());
</span><span class="lines">@@ -182,6 +189,9 @@
</span><span class="cx"> case DoubleCond:
</span><span class="cx"> out.print(asDoubleCondition());
</span><span class="cx"> return;
</span><ins>+ case StatusCond:
+ out.print(asStatusCondition());
+ return;
</ins><span class="cx"> case Special:
</span><span class="cx"> out.print(pointerDump(special()));
</span><span class="cx"> return;
</span><span class="lines">@@ -220,6 +230,9 @@
</span><span class="cx"> case Arg::BitImm64:
</span><span class="cx"> out.print("BitImm64");
</span><span class="cx"> return;
</span><ins>+ case Arg::SimpleAddr:
+ out.print("SimpleAddr");
+ return;
</ins><span class="cx"> case Arg::Addr:
</span><span class="cx"> out.print("Addr");
</span><span class="cx"> return;
</span><span class="lines">@@ -241,6 +254,9 @@
</span><span class="cx"> case Arg::DoubleCond:
</span><span class="cx"> out.print("DoubleCond");
</span><span class="cx"> return;
</span><ins>+ case Arg::StatusCond:
+ out.print("StatusCond");
+ return;
</ins><span class="cx"> case Arg::Special:
</span><span class="cx"> out.print("Special");
</span><span class="cx"> return;
</span><span class="lines">@@ -285,6 +301,9 @@
</span><span class="cx"> case Arg::EarlyDef:
</span><span class="cx"> out.print("EarlyDef");
</span><span class="cx"> return;
</span><ins>+ case Arg::EarlyZDef:
+ out.print("EarlyZDef");
+ return;
</ins><span class="cx"> case Arg::Scratch:
</span><span class="cx"> out.print("Scratch");
</span><span class="cx"> return;
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreb3airAirArgh"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/b3/air/AirArg.h (213713 => 213714)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/b3/air/AirArg.h 2017-03-10 17:32:07 UTC (rev 213713)
+++ trunk/Source/JavaScriptCore/b3/air/AirArg.h 2017-03-10 17:49:42 UTC (rev 213714)
</span><span class="lines">@@ -72,6 +72,7 @@
</span><span class="cx">
</span><span class="cx"> // These are the addresses. Instructions may load from (Use), store to (Def), or evaluate
</span><span class="cx"> // (UseAddr) addresses.
</span><ins>+ SimpleAddr,
</ins><span class="cx"> Addr,
</span><span class="cx"> Stack,
</span><span class="cx"> CallArg,
</span><span class="lines">@@ -82,6 +83,7 @@
</span><span class="cx"> RelCond,
</span><span class="cx"> ResCond,
</span><span class="cx"> DoubleCond,
</span><ins>+ StatusCond,
</ins><span class="cx"> Special,
</span><span class="cx"> WidthArg
</span><span class="cx"> };
</span><span class="lines">@@ -151,6 +153,8 @@
</span><span class="cx"> // as Use has undefined behavior: the use may happen before the def, or it may happen after
</span><span class="cx"> // it.
</span><span class="cx"> EarlyDef,
</span><ins>+
+ EarlyZDef,
</ins><span class="cx">
</span><span class="cx"> // Some instructions need a scratch register. We model this by saying that the temporary is
</span><span class="cx"> // defined early and used late. This role implies that.
</span><span class="lines">@@ -186,6 +190,7 @@
</span><span class="cx"> case ZDef:
</span><span class="cx"> case UseAddr:
</span><span class="cx"> case EarlyDef:
</span><ins>+ case EarlyZDef:
</ins><span class="cx"> return false;
</span><span class="cx"> }
</span><span class="cx"> ASSERT_NOT_REACHED();
</span><span class="lines">@@ -206,6 +211,7 @@
</span><span class="cx"> case UseAddr:
</span><span class="cx"> case Scratch:
</span><span class="cx"> case EarlyDef:
</span><ins>+ case EarlyZDef:
</ins><span class="cx"> return false;
</span><span class="cx"> }
</span><span class="cx"> ASSERT_NOT_REACHED();
</span><span class="lines">@@ -228,6 +234,7 @@
</span><span class="cx"> case UseAddr:
</span><span class="cx"> case Scratch:
</span><span class="cx"> case EarlyDef:
</span><ins>+ case EarlyZDef:
</ins><span class="cx"> return role;
</span><span class="cx"> case Use:
</span><span class="cx"> return ColdUse;
</span><span class="lines">@@ -253,6 +260,7 @@
</span><span class="cx"> case LateColdUse:
</span><span class="cx"> case Scratch:
</span><span class="cx"> case EarlyDef:
</span><ins>+ case EarlyZDef:
</ins><span class="cx"> return false;
</span><span class="cx"> }
</span><span class="cx"> ASSERT_NOT_REACHED();
</span><span class="lines">@@ -274,6 +282,7 @@
</span><span class="cx"> case ZDef:
</span><span class="cx"> case UseAddr:
</span><span class="cx"> case EarlyDef:
</span><ins>+ case EarlyZDef:
</ins><span class="cx"> return false;
</span><span class="cx"> }
</span><span class="cx"> ASSERT_NOT_REACHED();
</span><span class="lines">@@ -294,6 +303,7 @@
</span><span class="cx"> case ZDef:
</span><span class="cx"> case UseZDef:
</span><span class="cx"> case EarlyDef:
</span><ins>+ case EarlyZDef:
</ins><span class="cx"> case Scratch:
</span><span class="cx"> return true;
</span><span class="cx"> }
</span><span class="lines">@@ -315,6 +325,7 @@
</span><span class="cx"> case LateColdUse:
</span><span class="cx"> return false;
</span><span class="cx"> case EarlyDef:
</span><ins>+ case EarlyZDef:
</ins><span class="cx"> case Scratch:
</span><span class="cx"> return true;
</span><span class="cx"> }
</span><span class="lines">@@ -330,6 +341,7 @@
</span><span class="cx"> case UseAddr:
</span><span class="cx"> case LateUse:
</span><span class="cx"> case EarlyDef:
</span><ins>+ case EarlyZDef:
</ins><span class="cx"> case Scratch:
</span><span class="cx"> case LateColdUse:
</span><span class="cx"> return false;
</span><span class="lines">@@ -358,6 +370,7 @@
</span><span class="cx"> return false;
</span><span class="cx"> case ZDef:
</span><span class="cx"> case UseZDef:
</span><ins>+ case EarlyZDef:
</ins><span class="cx"> return true;
</span><span class="cx"> }
</span><span class="cx"> ASSERT_NOT_REACHED();
</span><span class="lines">@@ -416,6 +429,15 @@
</span><span class="cx"> return bigImm(bitwise_cast<intptr_t>(address));
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+ static Arg simpleAddr(Air::Tmp ptr)
+ {
+ ASSERT(ptr.isGP());
+ Arg result;
+ result.m_kind = SimpleAddr;
+ result.m_base = ptr;
+ return result;
+ }
+
</ins><span class="cx"> static Arg addr(Air::Tmp base, int32_t offset = 0)
</span><span class="cx"> {
</span><span class="cx"> ASSERT(base.isGP());
</span><span class="lines">@@ -533,6 +555,14 @@
</span><span class="cx"> return result;
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+ static Arg statusCond(MacroAssembler::StatusCondition condition)
+ {
+ Arg result;
+ result.m_kind = StatusCond;
+ result.m_offset = condition;
+ return result;
+ }
+
</ins><span class="cx"> static Arg special(Air::Special* special)
</span><span class="cx"> {
</span><span class="cx"> Arg result;
</span><span class="lines">@@ -608,6 +638,11 @@
</span><span class="cx"> }
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+ bool isSimpleAddr() const
+ {
+ return kind() == SimpleAddr;
+ }
+
</ins><span class="cx"> bool isAddr() const
</span><span class="cx"> {
</span><span class="cx"> return kind() == Addr;
</span><span class="lines">@@ -631,6 +666,7 @@
</span><span class="cx"> bool isMemory() const
</span><span class="cx"> {
</span><span class="cx"> switch (kind()) {
</span><ins>+ case SimpleAddr:
</ins><span class="cx"> case Addr:
</span><span class="cx"> case Stack:
</span><span class="cx"> case CallArg:
</span><span class="lines">@@ -641,6 +677,13 @@
</span><span class="cx"> }
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+ // Returns true if this is an idiomatic stack reference. It may return false for some kinds of
+ // stack references. The following idioms are recognized:
+ // - the Stack kind
+ // - the CallArg kind
+ // - the Addr kind with the base being either SP or FP
+ // Callers of this function are allowed to expect that if it returns true, then it must be one of
+ // these easy-to-recognize kinds. So, making this function recognize more kinds could break things.
</ins><span class="cx"> bool isStackMemory() const;
</span><span class="cx">
</span><span class="cx"> bool isRelCond() const
</span><span class="lines">@@ -658,6 +701,11 @@
</span><span class="cx"> return kind() == DoubleCond;
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+ bool isStatusCond() const
+ {
+ return kind() == StatusCond;
+ }
+
</ins><span class="cx"> bool isCondition() const
</span><span class="cx"> {
</span><span class="cx"> switch (kind()) {
</span><span class="lines">@@ -664,6 +712,7 @@
</span><span class="cx"> case RelCond:
</span><span class="cx"> case ResCond:
</span><span class="cx"> case DoubleCond:
</span><ins>+ case StatusCond:
</ins><span class="cx"> return true;
</span><span class="cx"> default:
</span><span class="cx"> return false;
</span><span class="lines">@@ -774,10 +823,16 @@
</span><span class="cx"> ASSERT(kind() == BigImm);
</span><span class="cx"> return bitwise_cast<void*>(static_cast<intptr_t>(m_offset));
</span><span class="cx"> }
</span><ins>+
+ Air::Tmp ptr() const
+ {
+ ASSERT(kind() == SimpleAddr);
+ return m_base;
+ }
</ins><span class="cx">
</span><span class="cx"> Air::Tmp base() const
</span><span class="cx"> {
</span><del>- ASSERT(kind() == Addr || kind() == Index);
</del><ins>+ ASSERT(kind() == SimpleAddr || kind() == Addr || kind() == Index);
</ins><span class="cx"> return m_base;
</span><span class="cx"> }
</span><span class="cx">
</span><span class="lines">@@ -844,6 +899,7 @@
</span><span class="cx"> case BigImm:
</span><span class="cx"> case BitImm:
</span><span class="cx"> case BitImm64:
</span><ins>+ case SimpleAddr:
</ins><span class="cx"> case Addr:
</span><span class="cx"> case Index:
</span><span class="cx"> case Stack:
</span><span class="lines">@@ -851,6 +907,7 @@
</span><span class="cx"> case RelCond:
</span><span class="cx"> case ResCond:
</span><span class="cx"> case DoubleCond:
</span><ins>+ case StatusCond:
</ins><span class="cx"> case Special:
</span><span class="cx"> case WidthArg:
</span><span class="cx"> return true;
</span><span class="lines">@@ -872,10 +929,12 @@
</span><span class="cx"> case RelCond:
</span><span class="cx"> case ResCond:
</span><span class="cx"> case DoubleCond:
</span><ins>+ case StatusCond:
</ins><span class="cx"> case Special:
</span><span class="cx"> case WidthArg:
</span><span class="cx"> case Invalid:
</span><span class="cx"> return false;
</span><ins>+ case SimpleAddr:
</ins><span class="cx"> case Addr:
</span><span class="cx"> case Index:
</span><span class="cx"> case Stack:
</span><span class="lines">@@ -1049,6 +1108,8 @@
</span><span class="cx"> return isValidBitImmForm(value());
</span><span class="cx"> case BitImm64:
</span><span class="cx"> return isValidBitImm64Form(value());
</span><ins>+ case SimpleAddr:
+ return true;
</ins><span class="cx"> case Addr:
</span><span class="cx"> case Stack:
</span><span class="cx"> case CallArg:
</span><span class="lines">@@ -1058,6 +1119,7 @@
</span><span class="cx"> case RelCond:
</span><span class="cx"> case ResCond:
</span><span class="cx"> case DoubleCond:
</span><ins>+ case StatusCond:
</ins><span class="cx"> case Special:
</span><span class="cx"> case WidthArg:
</span><span class="cx"> return true;
</span><span class="lines">@@ -1070,6 +1132,7 @@
</span><span class="cx"> {
</span><span class="cx"> switch (m_kind) {
</span><span class="cx"> case Tmp:
</span><ins>+ case SimpleAddr:
</ins><span class="cx"> case Addr:
</span><span class="cx"> functor(m_base);
</span><span class="cx"> break;
</span><span class="lines">@@ -1110,6 +1173,7 @@
</span><span class="cx"> ASSERT(isAnyUse(argRole) || isAnyDef(argRole));
</span><span class="cx"> functor(m_base, argRole, argBank, argWidth);
</span><span class="cx"> break;
</span><ins>+ case SimpleAddr:
</ins><span class="cx"> case Addr:
</span><span class="cx"> functor(m_base, Use, GP, argRole == UseAddr ? argWidth : pointerWidth());
</span><span class="cx"> break;
</span><span class="lines">@@ -1144,9 +1208,11 @@
</span><span class="cx"> ASSERT(isImm());
</span><span class="cx"> return MacroAssembler::TrustedImmPtr(pointerValue());
</span><span class="cx"> }
</span><del>-
</del><ins>+
</ins><span class="cx"> MacroAssembler::Address asAddress() const
</span><span class="cx"> {
</span><ins>+ if (isSimpleAddr())
+ return MacroAssembler::Address(m_base.gpr());
</ins><span class="cx"> ASSERT(isAddr());
</span><span class="cx"> return MacroAssembler::Address(m_base.gpr(), static_cast<int32_t>(m_offset));
</span><span class="cx"> }
</span><span class="lines">@@ -1177,6 +1243,12 @@
</span><span class="cx"> return static_cast<MacroAssembler::DoubleCondition>(m_offset);
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+ MacroAssembler::StatusCondition asStatusCondition() const
+ {
+ ASSERT(isStatusCond());
+ return static_cast<MacroAssembler::StatusCondition>(m_offset);
+ }
+
</ins><span class="cx"> // Tells you if the Arg is invertible. Only condition arguments are invertible, and even for those, there
</span><span class="cx"> // are a few exceptions - notably Overflow and Signed.
</span><span class="cx"> bool isInvertible() const
</span><span class="lines">@@ -1184,6 +1256,7 @@
</span><span class="cx"> switch (kind()) {
</span><span class="cx"> case RelCond:
</span><span class="cx"> case DoubleCond:
</span><ins>+ case StatusCond:
</ins><span class="cx"> return true;
</span><span class="cx"> case ResCond:
</span><span class="cx"> return MacroAssembler::isInvertible(asResultCondition());
</span><span class="lines">@@ -1204,6 +1277,8 @@
</span><span class="cx"> return resCond(MacroAssembler::invert(asResultCondition()));
</span><span class="cx"> case DoubleCond:
</span><span class="cx"> return doubleCond(MacroAssembler::invert(asDoubleCondition()));
</span><ins>+ case StatusCond:
+ return statusCond(MacroAssembler::invert(asStatusCondition()));
</ins><span class="cx"> default:
</span><span class="cx"> RELEASE_ASSERT_NOT_REACHED();
</span><span class="cx"> return Arg();
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreb3airAirBasicBlockcpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/b3/air/AirBasicBlock.cpp (213713 => 213714)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/b3/air/AirBasicBlock.cpp 2017-03-10 17:32:07 UTC (rev 213713)
+++ trunk/Source/JavaScriptCore/b3/air/AirBasicBlock.cpp 2017-03-10 17:49:42 UTC (rev 213714)
</span><span class="lines">@@ -1,5 +1,5 @@
</span><span class="cx"> /*
</span><del>- * Copyright (C) 2015 Apple Inc. All rights reserved.
</del><ins>+ * Copyright (C) 2015-2017 Apple Inc. All rights reserved.
</ins><span class="cx"> *
</span><span class="cx"> * Redistribution and use in source and binary forms, with or without
</span><span class="cx"> * modification, are permitted provided that the following conditions
</span><span class="lines">@@ -35,6 +35,19 @@
</span><span class="cx">
</span><span class="cx"> const char* const BasicBlock::dumpPrefix = "#";
</span><span class="cx">
</span><ins>+void BasicBlock::setSuccessors(FrequentedBlock target)
+{
+ m_successors.resize(1);
+ m_successors[0] = target;
+}
+
+void BasicBlock::setSuccessors(FrequentedBlock taken, FrequentedBlock notTaken)
+{
+ m_successors.resize(2);
+ m_successors[0] = taken;
+ m_successors[1] = notTaken;
+}
+
</ins><span class="cx"> bool BasicBlock::addPredecessor(BasicBlock* block)
</span><span class="cx"> {
</span><span class="cx"> return B3::addPredecessor(this, block);
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreb3airAirBasicBlockh"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/b3/air/AirBasicBlock.h (213713 => 213714)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/b3/air/AirBasicBlock.h 2017-03-10 17:32:07 UTC (rev 213713)
+++ trunk/Source/JavaScriptCore/b3/air/AirBasicBlock.h 2017-03-10 17:49:42 UTC (rev 213714)
</span><span class="lines">@@ -1,5 +1,5 @@
</span><span class="cx"> /*
</span><del>- * Copyright (C) 2015-2016 Apple Inc. All rights reserved.
</del><ins>+ * Copyright (C) 2015-2017 Apple Inc. All rights reserved.
</ins><span class="cx"> *
</span><span class="cx"> * Redistribution and use in source and binary forms, with or without
</span><span class="cx"> * modification, are permitted provided that the following conditions
</span><span class="lines">@@ -33,8 +33,12 @@
</span><span class="cx"> #include <wtf/FastMalloc.h>
</span><span class="cx"> #include <wtf/Noncopyable.h>
</span><span class="cx">
</span><del>-namespace JSC { namespace B3 { namespace Air {
</del><ins>+namespace JSC { namespace B3 {
</ins><span class="cx">
</span><ins>+template<typename> class GenericBlockInsertionSet;
+
+namespace Air {
+
</ins><span class="cx"> class BlockInsertionSet;
</span><span class="cx"> class Code;
</span><span class="cx"> class InsertionSet;
</span><span class="lines">@@ -99,6 +103,9 @@
</span><span class="cx"> const SuccessorList& successors() const { return m_successors; }
</span><span class="cx"> SuccessorList& successors() { return m_successors; }
</span><span class="cx">
</span><ins>+ void setSuccessors(FrequentedBlock);
+ void setSuccessors(FrequentedBlock, FrequentedBlock);
+
</ins><span class="cx"> BasicBlock* successorBlock(unsigned index) const { return successor(index).block(); }
</span><span class="cx"> BasicBlock*& successorBlock(unsigned index) { return successor(index).block(); }
</span><span class="cx"> SuccessorCollection<BasicBlock, SuccessorList> successorBlocks()
</span><span class="lines">@@ -133,6 +140,7 @@
</span><span class="cx"> friend class BlockInsertionSet;
</span><span class="cx"> friend class Code;
</span><span class="cx"> friend class InsertionSet;
</span><ins>+ template<typename> friend class B3::GenericBlockInsertionSet;
</ins><span class="cx">
</span><span class="cx"> BasicBlock(unsigned index, double frequency);
</span><span class="cx">
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreb3airAirBlockInsertionSetcpp"></a>
<div class="addfile"><h4>Added: trunk/Source/JavaScriptCore/b3/air/AirBlockInsertionSet.cpp (0 => 213714)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/b3/air/AirBlockInsertionSet.cpp (rev 0)
+++ trunk/Source/JavaScriptCore/b3/air/AirBlockInsertionSet.cpp 2017-03-10 17:49:42 UTC (rev 213714)
</span><span class="lines">@@ -0,0 +1,49 @@
</span><ins>+/*
+ * Copyright (C) 2017 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+ * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+ * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+ * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include "config.h"
+#include "AirBlockInsertionSet.h"
+
+#if ENABLE(B3_JIT)
+
+#include "AirBasicBlock.h"
+#include "AirCode.h"
+#include <wtf/BubbleSort.h>
+
+namespace JSC { namespace B3 { namespace Air {
+
+BlockInsertionSet::BlockInsertionSet(Code &code)
+ : GenericBlockInsertionSet(code.m_blocks)
+{
+}
+
+BlockInsertionSet::~BlockInsertionSet()
+{
+}
+
+} } } // namespace JSC::B3::Air
+
+#endif // ENABLE(B3_JIT)
+
</ins></span></pre></div>
<a id="trunkSourceJavaScriptCoreb3airAirBlockInsertionSeth"></a>
<div class="addfile"><h4>Added: trunk/Source/JavaScriptCore/b3/air/AirBlockInsertionSet.h (0 => 213714)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/b3/air/AirBlockInsertionSet.h (rev 0)
+++ trunk/Source/JavaScriptCore/b3/air/AirBlockInsertionSet.h 2017-03-10 17:49:42 UTC (rev 213714)
</span><span class="lines">@@ -0,0 +1,52 @@
</span><ins>+/*
+ * Copyright (C) 2017 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+ * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+ * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+ * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#pragma once
+
+#if ENABLE(B3_JIT)
+
+#include "B3GenericBlockInsertionSet.h"
+#include "AirCode.h"
+#include <wtf/Insertion.h>
+#include <wtf/Vector.h>
+
+namespace JSC { namespace B3 { namespace Air {
+
+class InsertionSet;
+
+typedef GenericBlockInsertionSet<BasicBlock>::BlockInsertion BlockInsertion;
+
+class BlockInsertionSet : public GenericBlockInsertionSet<BasicBlock> {
+public:
+ BlockInsertionSet(Code&);
+ ~BlockInsertionSet();
+
+ // FIXME: We should eventually implement B3::BlockInsertionSet's splitForward().
+ // https://bugs.webkit.org/show_bug.cgi?id=169253
+};
+
+} } } // namespace JSC::B3::Air
+
+#endif // ENABLE(B3_JIT)
</ins></span></pre></div>
<a id="trunkSourceJavaScriptCoreb3airAirDumpAsJScpp"></a>
<div class="delfile"><h4>Deleted: trunk/Source/JavaScriptCore/b3/air/AirDumpAsJS.cpp (213713 => 213714)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/b3/air/AirDumpAsJS.cpp 2017-03-10 17:32:07 UTC (rev 213713)
+++ trunk/Source/JavaScriptCore/b3/air/AirDumpAsJS.cpp 2017-03-10 17:49:42 UTC (rev 213714)
</span><span class="lines">@@ -1,245 +0,0 @@
</span><del>-/*
- * Copyright (C) 2016-2017 Apple Inc. All rights reserved.
- *
- * Redistribution and use in source and binary forms, with or without
- * modification, are permitted provided that the following conditions
- * are met:
- * 1. Redistributions of source code must retain the above copyright
- * notice, this list of conditions and the following disclaimer.
- * 2. Redistributions in binary form must reproduce the above copyright
- * notice, this list of conditions and the following disclaimer in the
- * documentation and/or other materials provided with the distribution.
- *
- * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
- * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
- * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
- * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR
- * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
- * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
- * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
- * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
- * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
- * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
- * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
- */
-
-#include "config.h"
-#include "AirDumpAsJS.h"
-
-#if ENABLE(B3_JIT)
-
-#include "AirCode.h"
-#include "AirInstInlines.h"
-
-namespace JSC { namespace B3 { namespace Air {
-
-namespace {
-
-CString varNameForBlockAtIndex(unsigned index)
-{
- return toCString("bb", index);
-}
-
-CString varName(BasicBlock* block)
-{
- return varNameForBlockAtIndex(block->index());
-}
-
-CString varNameForStackSlotAtIndex(unsigned index)
-{
- return toCString("slot", index);
-}
-
-CString varName(StackSlot* slot)
-{
- return varNameForStackSlotAtIndex(slot->index());
-}
-
-CString varName(Reg reg)
-{
- return toCString("Reg.", reg.debugName());
-}
-
-CString varNameForTmpWithBankAndIndex(Bank bank, unsigned index)
-{
- return toCString(bank == FP ? "f" : "", "tmp", index);
-}
-
-CString varName(Tmp tmp)
-{
- if (tmp.isReg())
- return varName(tmp.reg());
- return varNameForTmpWithBankAndIndex(Arg(tmp).bank(), tmp.tmpIndex());
-}
-
-} // anonymous namespace
-
-void dumpAsJS(Code& code, PrintStream& out)
-{
- out.println("let code = new Code();");
-
- for (unsigned i = 0; i < code.size(); ++i)
- out.println("let ", varNameForBlockAtIndex(i), " = code.addBlock();");
-
- out.println("let hash;");
-
- for (unsigned i = 0; i < code.stackSlots().size(); ++i) {
- StackSlot* slot = code.stackSlots()[i];
- if (slot) {
- out.println("let ", varName(slot), " = code.addStackSlot(", slot->byteSize(), ", ", slot->kind(), ");");
- if (slot->offsetFromFP())
- out.println(varName(slot), ".setOffsetFromFP(", slot->offsetFromFP(), ");");
- out.println("hash = ", varName(slot), ".hash();");
- out.println("if (hash != ", slot->jsHash(), ")");
- out.println(" throw new Error(\"Bad hash: \" + hash);");
- } else
- out.println("code.addStackSlot(1, Spill);");
- }
-
- forEachBank(
- [&] (Bank bank) {
- for (unsigned i = code.numTmps(bank); i--;) {
- out.println(
- "let ", varNameForTmpWithBankAndIndex(bank, i), " = code.newTmp(", bank, ");");
- }
- });
-
- out.println("let inst;");
- out.println("let arg;");
-
- for (BasicBlock* block : code) {
- for (FrequentedBlock successor : block->successors()) {
- out.println(
- varName(block), ".successors.push(new FrequentedBlock(",
- varName(successor.block()), ", ", successor.frequency(), "));");
- }
-
- for (BasicBlock* predecessor : block->predecessors())
- out.println(varName(block), ".predecessors.push(", varName(predecessor), ");");
-
- for (Inst& inst : *block) {
- // FIXME: This should do something for flags.
- // https://bugs.webkit.org/show_bug.cgi?id=162751
- out.println("inst = new Inst(", inst.kind.opcode, ");");
-
- inst.forEachArg(
- [&] (Arg& arg, Arg::Role, Bank, Width) {
- switch (arg.kind()) {
- case Arg::Invalid:
- RELEASE_ASSERT_NOT_REACHED();
- break;
-
- case Arg::Tmp:
- out.println("arg = Arg.createTmp(", varName(arg.tmp()), ");");
- break;
-
- case Arg::Imm:
- out.println("arg = Arg.createImm(", arg.value(), ");");
- break;
-
- case Arg::BigImm:
- out.println(
- "arg = Arg.createBigImm(",
- static_cast<int32_t>(arg.value()), ", ",
- static_cast<int32_t>(arg.value() >> 32), ");");
- break;
-
- case Arg::BitImm:
- out.println("arg = Arg.createBitImm(", arg.value(), ");");
- break;
-
- case Arg::BitImm64:
- out.println(
- "arg = Arg.createBitImm64(",
- static_cast<int32_t>(arg.value()), ", ",
- static_cast<int32_t>(arg.value() >> 32), ");");
- break;
-
- case Arg::Addr:
- out.println(
- "arg = Arg.createAddr(", varName(arg.base()), ", ", arg.offset(), ");");
- break;
-
- case Arg::Stack:
- out.println(
- "arg = Arg.createStack(", varName(arg.stackSlot()), ", ", arg.offset(), ");");
- break;
-
- case Arg::CallArg:
- out.println("arg = Arg.createCallArg(", arg.offset(), ");");
- break;
-
- case Arg::Index:
- out.println(
- "arg = Arg.createIndex(", varName(arg.base()), ", ",
- varName(arg.index()), ", ", arg.scale(), ", ", arg.offset(), ");");
- break;
-
- case Arg::RelCond:
- out.println("arg = Arg.createRelCond(", arg.asRelationalCondition(), ");");
- break;
-
- case Arg::ResCond:
- out.println("arg = Arg.createResCond(", arg.asResultCondition(), ");");
- break;
-
- case Arg::DoubleCond:
- out.println("arg = Arg.createDoubleCond(", arg.asDoubleCondition(), ");");
- break;
-
- case Arg::Special:
- out.println("arg = Arg.createSpecial();");
- break;
-
- case Arg::WidthArg:
- out.println("arg = Arg.createWidthArg(", arg.width(), ");");
- break;
- }
-
- out.println("inst.args.push(arg);");
- });
-
- if (inst.kind.opcode == Patch) {
- if (inst.hasNonArgEffects())
- out.println("inst.patchHasNonArgEffects = true;");
-
- out.println("inst.extraEarlyClobberedRegs = new Set();");
- out.println("inst.extraClobberedRegs = new Set();");
- inst.extraEarlyClobberedRegs().forEach(
- [&] (Reg reg) {
- out.println("inst.extraEarlyClobberedRegs.add(", varName(reg), ");");
- });
- inst.extraClobberedRegs().forEach(
- [&] (Reg reg) {
- out.println("inst.extraClobberedRegs.add(", varName(reg), ");");
- });
-
- out.println("inst.patchArgData = [];");
- inst.forEachArg(
- [&] (Arg&, Arg::Role role, Bank bank, Width width) {
- out.println(
- "inst.patchArgData.push({role: Arg.", role, ", type: ", bank,
- ", width: ", width, "});");
- });
- }
-
- if (inst.kind.opcode == CCall || inst.kind.opcode == ColdCCall) {
- out.println("inst.cCallType = ", inst.origin->type());
- out.println("inst.cCallArgTypes = [];");
- for (unsigned i = 1; i < inst.origin->numChildren(); ++i)
- out.println("inst.cCallArgTypes.push(", inst.origin->child(i)->type(), ");");
- }
-
- out.println("hash = inst.hash();");
- out.println("if (hash != ", inst.jsHash(), ")");
- out.println(" throw new Error(\"Bad hash: \" + hash);");
-
- out.println(varName(block), ".append(inst);");
- }
- }
-}
-
-} } } // namespace JSC::B3::Air
-
-#endif // ENABLE(B3_JIT)
-
</del></span></pre></div>
<a id="trunkSourceJavaScriptCoreb3airAirDumpAsJSh"></a>
<div class="delfile"><h4>Deleted: trunk/Source/JavaScriptCore/b3/air/AirDumpAsJS.h (213713 => 213714)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/b3/air/AirDumpAsJS.h 2017-03-10 17:32:07 UTC (rev 213713)
+++ trunk/Source/JavaScriptCore/b3/air/AirDumpAsJS.h 2017-03-10 17:49:42 UTC (rev 213714)
</span><span class="lines">@@ -1,43 +0,0 @@
</span><del>-/*
- * Copyright (C) 2016 Apple Inc. All rights reserved.
- *
- * Redistribution and use in source and binary forms, with or without
- * modification, are permitted provided that the following conditions
- * are met:
- * 1. Redistributions of source code must retain the above copyright
- * notice, this list of conditions and the following disclaimer.
- * 2. Redistributions in binary form must reproduce the above copyright
- * notice, this list of conditions and the following disclaimer in the
- * documentation and/or other materials provided with the distribution.
- *
- * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
- * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
- * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
- * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR
- * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
- * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
- * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
- * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
- * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
- * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
- * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
- */
-
-#pragma once
-
-#if ENABLE(B3_JIT)
-
-#include <wtf/PrintStream.h>
-
-namespace JSC { namespace B3 { namespace Air {
-
-class Code;
-
-// This is used for benchmarking. Various operations on Air are interesting from a benchmarking
-// standpoint. We can write some Air phases in JS and then use that to benchmark JS. The benchmark
-// is called JSAir, and it's in PerformanceTests/JSAir.
-void dumpAsJS(Code&, PrintStream&);
-
-} } } // namespace JSC::B3::Air
-
-#endif // ENABLE(B3_JIT)
</del></span></pre></div>
<a id="trunkSourceJavaScriptCoreb3airAirEliminateDeadCodecpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/b3/air/AirEliminateDeadCode.cpp (213713 => 213714)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/b3/air/AirEliminateDeadCode.cpp 2017-03-10 17:32:07 UTC (rev 213713)
+++ trunk/Source/JavaScriptCore/b3/air/AirEliminateDeadCode.cpp 2017-03-10 17:49:42 UTC (rev 213714)
</span><span class="lines">@@ -118,6 +118,8 @@
</span><span class="cx"> changed = false;
</span><span class="cx"> for (unsigned blockIndex = code.size(); blockIndex--;) {
</span><span class="cx"> BasicBlock* block = code[blockIndex];
</span><ins>+ if (!block)
+ continue;
</ins><span class="cx"> for (unsigned instIndex = block->size(); instIndex--;)
</span><span class="cx"> handleInst(block->at(instIndex));
</span><span class="cx"> }
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreb3airAirGeneratecpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/b3/air/AirGenerate.cpp (213713 => 213714)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/b3/air/AirGenerate.cpp 2017-03-10 17:32:07 UTC (rev 213713)
+++ trunk/Source/JavaScriptCore/b3/air/AirGenerate.cpp 2017-03-10 17:49:42 UTC (rev 213714)
</span><span class="lines">@@ -1,5 +1,5 @@
</span><span class="cx"> /*
</span><del>- * Copyright (C) 2015-2016 Apple Inc. All rights reserved.
</del><ins>+ * Copyright (C) 2015-2017 Apple Inc. All rights reserved.
</ins><span class="cx"> *
</span><span class="cx"> * Redistribution and use in source and binary forms, with or without
</span><span class="cx"> * modification, are permitted provided that the following conditions
</span><span class="lines">@@ -31,7 +31,6 @@
</span><span class="cx"> #include "AirAllocateRegistersByGraphColoring.h"
</span><span class="cx"> #include "AirAllocateStack.h"
</span><span class="cx"> #include "AirCode.h"
</span><del>-#include "AirDumpAsJS.h"
</del><span class="cx"> #include "AirEliminateDeadCode.h"
</span><span class="cx"> #include "AirFixObviousSpills.h"
</span><span class="cx"> #include "AirFixPartialRegisterStalls.h"
</span><span class="lines">@@ -62,6 +61,12 @@
</span><span class="cx"> {
</span><span class="cx"> TimingScope timingScope("Air::prepareForGeneration");
</span><span class="cx">
</span><ins>+ // If we're doing super verbose dumping, the phase scope of any phase will already do a dump.
+ if (shouldDumpIR(AirMode) && !shouldDumpIRAtEachPhase(AirMode)) {
+ dataLog("Initial air:\n");
+ dataLog(code);
+ }
+
</ins><span class="cx"> // We don't expect the incoming code to have predecessors computed.
</span><span class="cx"> code.resetReachability();
</span><span class="cx">
</span><span class="lines">@@ -68,11 +73,7 @@
</span><span class="cx"> if (shouldValidateIR())
</span><span class="cx"> validate(code);
</span><span class="cx">
</span><del>- // If we're doing super verbose dumping, the phase scope of any phase will already do a dump.
- if (shouldDumpIR(AirMode) && !shouldDumpIRAtEachPhase(AirMode)) {
- dataLog("Initial air:\n");
- dataLog(code);
- }
</del><ins>+ simplifyCFG(code);
</ins><span class="cx">
</span><span class="cx"> lowerMacros(code);
</span><span class="cx">
</span><span class="lines">@@ -107,23 +108,11 @@
</span><span class="cx"> // does things like identify which callee-saves we're using and saves them.
</span><span class="cx"> handleCalleeSaves(code);
</span><span class="cx">
</span><del>- if (Options::dumpAirAsJSBeforeAllocateStack()) {
- dataLog("Dumping Air as JS before allocateStack:\n");
- dumpAsJS(code, WTF::dataFile());
- dataLog("Air hash: ", code.jsHash(), "\n");
- }
-
</del><span class="cx"> // This turns all Stack and CallArg Args into Addr args that use the frame pointer. It does
</span><span class="cx"> // this by first-fit allocating stack slots. It should be pretty darn close to optimal, so we
</span><span class="cx"> // shouldn't have to worry about this very much.
</span><span class="cx"> allocateStack(code);
</span><span class="cx">
</span><del>- if (Options::dumpAirAfterAllocateStack()) {
- dataLog("Dumping Air after allocateStack:\n");
- dataLog(code);
- dataLog("Air hash: ", code.jsHash(), "\n");
- }
-
</del><span class="cx"> // If we coalesced moves then we can unbreak critical edges. This is the main reason for this
</span><span class="cx"> // phase.
</span><span class="cx"> simplifyCFG(code);
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreb3airAirInstInlinesh"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/b3/air/AirInstInlines.h (213713 => 213714)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/b3/air/AirInstInlines.h 2017-03-10 17:32:07 UTC (rev 213713)
+++ trunk/Source/JavaScriptCore/b3/air/AirInstInlines.h 2017-03-10 17:49:42 UTC (rev 213714)
</span><span class="lines">@@ -277,6 +277,73 @@
</span><span class="cx"> return isX86DivHelperValid(inst);
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+inline bool isAtomicStrongCASValid(const Inst& inst)
+{
+#if CPU(X86) || CPU(X86_64)
+ switch (inst.args.size()) {
+ case 3:
+ return inst.args[0] == Tmp(X86Registers::eax);
+ case 5:
+ return inst.args[1] == Tmp(X86Registers::eax);
+ default:
+ return false;
+ }
+#else // CPU(X86) || CPU(X86_64)
+ UNUSED_PARAM(inst);
+ return false;
+#endif // CPU(X86) || CPU(X86_64)
+}
+
+inline bool isBranchAtomicStrongCASValid(const Inst& inst)
+{
+#if CPU(X86) || CPU(X86_64)
+ return inst.args[1] == Tmp(X86Registers::eax);
+#else // CPU(X86) || CPU(X86_64)
+ UNUSED_PARAM(inst);
+ return false;
+#endif // CPU(X86) || CPU(X86_64)
+}
+
+inline bool isAtomicStrongCAS8Valid(const Inst& inst)
+{
+ return isAtomicStrongCASValid(inst);
+}
+
+inline bool isAtomicStrongCAS16Valid(const Inst& inst)
+{
+ return isAtomicStrongCASValid(inst);
+}
+
+inline bool isAtomicStrongCAS32Valid(const Inst& inst)
+{
+ return isAtomicStrongCASValid(inst);
+}
+
+inline bool isAtomicStrongCAS64Valid(const Inst& inst)
+{
+ return isAtomicStrongCASValid(inst);
+}
+
+inline bool isBranchAtomicStrongCAS8Valid(const Inst& inst)
+{
+ return isBranchAtomicStrongCASValid(inst);
+}
+
+inline bool isBranchAtomicStrongCAS16Valid(const Inst& inst)
+{
+ return isBranchAtomicStrongCASValid(inst);
+}
+
+inline bool isBranchAtomicStrongCAS32Valid(const Inst& inst)
+{
+ return isBranchAtomicStrongCASValid(inst);
+}
+
+inline bool isBranchAtomicStrongCAS64Valid(const Inst& inst)
+{
+ return isBranchAtomicStrongCASValid(inst);
+}
+
</ins><span class="cx"> } } } // namespace JSC::B3::Air
</span><span class="cx">
</span><span class="cx"> #endif // ENABLE(B3_JIT)
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreb3airAirOpcodeopcodes"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/b3/air/AirOpcode.opcodes (213713 => 213714)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/b3/air/AirOpcode.opcodes 2017-03-10 17:32:07 UTC (rev 213713)
+++ trunk/Source/JavaScriptCore/b3/air/AirOpcode.opcodes 2017-03-10 17:49:42 UTC (rev 213714)
</span><span class="lines">@@ -1,4 +1,4 @@
</span><del>-# Copyright (C) 2015-2016 Apple Inc. All rights reserved.
</del><ins>+# Copyright (C) 2015-2017 Apple Inc. All rights reserved.
</ins><span class="cx"> #
</span><span class="cx"> # Redistribution and use in source and binary forms, with or without
</span><span class="cx"> # modification, are permitted provided that the following conditions
</span><span class="lines">@@ -117,6 +117,7 @@
</span><span class="cx"> x86: Imm, Index
</span><span class="cx"> Imm, Tmp
</span><span class="cx"> x86: Addr, Tmp
</span><ins>+ x86: Index, Tmp
</ins><span class="cx"> x86: Tmp, Addr
</span><span class="cx"> x86: Tmp, Index
</span><span class="cx">
</span><span class="lines">@@ -135,9 +136,12 @@
</span><span class="cx"> 64: Add64 U:G:64, UD:G:64
</span><span class="cx"> Tmp, Tmp
</span><span class="cx"> x86: Imm, Addr
</span><ins>+ x86: Imm, Index
</ins><span class="cx"> Imm, Tmp
</span><span class="cx"> x86: Addr, Tmp
</span><ins>+ x86: Index, Tmp
</ins><span class="cx"> x86: Tmp, Addr
</span><ins>+ x86: Tmp, Index
</ins><span class="cx">
</span><span class="cx"> 64: Add64 U:G:64, U:G:64, D:G:64
</span><span class="cx"> Imm, Tmp, Tmp
</span><span class="lines">@@ -166,9 +170,12 @@
</span><span class="cx"> Sub32 U:G:32, UZD:G:32
</span><span class="cx"> Tmp, Tmp
</span><span class="cx"> x86: Imm, Addr
</span><ins>+ x86: Imm, Index
</ins><span class="cx"> Imm, Tmp
</span><span class="cx"> x86: Addr, Tmp
</span><ins>+ x86: Index, Tmp
</ins><span class="cx"> x86: Tmp, Addr
</span><ins>+ x86: Tmp, Index
</ins><span class="cx">
</span><span class="cx"> arm64: Sub32 U:G:32, U:G:32, D:G:32
</span><span class="cx"> Tmp, Tmp, Tmp
</span><span class="lines">@@ -176,9 +183,12 @@
</span><span class="cx"> 64: Sub64 U:G:64, UD:G:64
</span><span class="cx"> Tmp, Tmp
</span><span class="cx"> x86: Imm, Addr
</span><ins>+ x86: Imm, Index
</ins><span class="cx"> Imm, Tmp
</span><span class="cx"> x86: Addr, Tmp
</span><ins>+ x86: Index, Tmp
</ins><span class="cx"> x86: Tmp, Addr
</span><ins>+ x86: Tmp, Index
</ins><span class="cx">
</span><span class="cx"> arm64: Sub64 U:G:64, U:G:64, D:G:64
</span><span class="cx"> Tmp, Tmp, Tmp
</span><span class="lines">@@ -204,9 +214,12 @@
</span><span class="cx"> Neg32 UZD:G:32
</span><span class="cx"> Tmp
</span><span class="cx"> x86: Addr
</span><ins>+ x86: Index
</ins><span class="cx">
</span><span class="cx"> 64: Neg64 UD:G:64
</span><span class="cx"> Tmp
</span><ins>+ x86: Addr
+ x86: Index
</ins><span class="cx">
</span><span class="cx"> arm64: NegateDouble U:F:64, D:F:64
</span><span class="cx"> Tmp, Tmp
</span><span class="lines">@@ -330,8 +343,11 @@
</span><span class="cx"> Tmp, Tmp
</span><span class="cx"> x86: Imm, Tmp
</span><span class="cx"> x86: Tmp, Addr
</span><ins>+ x86: Tmp, Index
</ins><span class="cx"> x86: Addr, Tmp
</span><ins>+ x86: Index, Tmp
</ins><span class="cx"> x86: Imm, Addr
</span><ins>+ x86: Imm, Index
</ins><span class="cx">
</span><span class="cx"> 64: And64 U:G:64, U:G:64, D:G:64
</span><span class="cx"> Tmp, Tmp, Tmp
</span><span class="lines">@@ -339,7 +355,13 @@
</span><span class="cx">
</span><span class="cx"> x86_64: And64 U:G:64, UD:G:64
</span><span class="cx"> Tmp, Tmp
</span><del>- x86: Imm, Tmp
</del><ins>+ Imm, Tmp
+ Imm, Addr
+ Imm, Index
+ Tmp, Addr
+ Tmp, Index
+ Addr, Tmp
+ Index, Tmp
</ins><span class="cx">
</span><span class="cx"> AndDouble U:F:64, U:F:64, D:F:64
</span><span class="cx"> Tmp, Tmp, Tmp
</span><span class="lines">@@ -459,8 +481,11 @@
</span><span class="cx"> Tmp, Tmp
</span><span class="cx"> x86: Imm, Tmp
</span><span class="cx"> x86: Tmp, Addr
</span><ins>+ x86: Tmp, Index
</ins><span class="cx"> x86: Addr, Tmp
</span><ins>+ x86: Index, Tmp
</ins><span class="cx"> x86: Imm, Addr
</span><ins>+ x86: Imm, Index
</ins><span class="cx">
</span><span class="cx"> 64: Or64 U:G:64, U:G:64, D:G:64
</span><span class="cx"> Tmp, Tmp, Tmp
</span><span class="lines">@@ -469,6 +494,12 @@
</span><span class="cx"> 64: Or64 U:G:64, UD:G:64
</span><span class="cx"> Tmp, Tmp
</span><span class="cx"> x86: Imm, Tmp
</span><ins>+ x86: Imm, Addr
+ x86: Imm, Index
+ x86: Tmp, Addr
+ x86: Tmp, Index
+ x86: Addr, Tmp
+ x86: Index, Tmp
</ins><span class="cx">
</span><span class="cx"> Xor32 U:G:32, U:G:32, ZD:G:32
</span><span class="cx"> Tmp, Tmp, Tmp
</span><span class="lines">@@ -480,8 +511,11 @@
</span><span class="cx"> Tmp, Tmp
</span><span class="cx"> x86: Imm, Tmp
</span><span class="cx"> x86: Tmp, Addr
</span><ins>+ x86: Tmp, Index
</ins><span class="cx"> x86: Addr, Tmp
</span><ins>+ x86: Index, Tmp
</ins><span class="cx"> x86: Imm, Addr
</span><ins>+ x86: Imm, Index
</ins><span class="cx">
</span><span class="cx"> 64: Xor64 U:G:64, U:G:64, D:G:64
</span><span class="cx"> Tmp, Tmp, Tmp
</span><span class="lines">@@ -490,6 +524,11 @@
</span><span class="cx"> 64: Xor64 U:G:64, UD:G:64
</span><span class="cx"> Tmp, Tmp
</span><span class="cx"> x86: Tmp, Addr
</span><ins>+ x86: Tmp, Index
+ x86: Addr, Tmp
+ x86: Index, Tmp
+ x86: Imm, Addr
+ x86: Imm, Index
</ins><span class="cx"> x86: Imm, Tmp
</span><span class="cx">
</span><span class="cx"> arm64: Not32 U:G:32, ZD:G:32
</span><span class="lines">@@ -498,13 +537,15 @@
</span><span class="cx"> x86: Not32 UZD:G:32
</span><span class="cx"> Tmp
</span><span class="cx"> Addr
</span><ins>+ Index
</ins><span class="cx">
</span><span class="cx"> arm64: Not64 U:G:64, D:G:64
</span><span class="cx"> Tmp, Tmp
</span><span class="cx">
</span><del>-x86: Not64 UD:G:64
</del><ins>+x86_64: Not64 UD:G:64
</ins><span class="cx"> Tmp
</span><span class="cx"> Addr
</span><ins>+ Index
</ins><span class="cx">
</span><span class="cx"> arm64: AbsDouble U:F:64, D:F:64
</span><span class="cx"> Tmp, Tmp
</span><span class="lines">@@ -668,6 +709,11 @@
</span><span class="cx"> Addr, Tmp
</span><span class="cx"> Index, Tmp
</span><span class="cx">
</span><ins>+LoadAcq8 U:G:8, ZD:G:32 /effects
+ x86: Addr, Tmp
+ x86: Index, Tmp
+ SimpleAddr, Tmp
+
</ins><span class="cx"> Store8 U:G:8, D:G:8
</span><span class="cx"> Tmp, Index
</span><span class="cx"> Tmp, Addr
</span><span class="lines">@@ -674,22 +720,368 @@
</span><span class="cx"> x86: Imm, Index
</span><span class="cx"> x86: Imm, Addr
</span><span class="cx">
</span><ins>+StoreRel8 U:G:8, D:G:8 /effects
+ x86: Tmp, Index
+ x86: Tmp, Addr
+ x86: Imm, Index
+ x86: Imm, Addr
+ Tmp, SimpleAddr
+
</ins><span class="cx"> Load8SignedExtendTo32 U:G:8, ZD:G:32
</span><span class="cx"> Addr, Tmp
</span><span class="cx"> Index, Tmp
</span><span class="cx">
</span><ins>+LoadAcq8SignedExtendTo32 U:G:8, ZD:G:32 /effects
+ x86: Addr, Tmp
+ x86: Index, Tmp
+ SimpleAddr, Tmp
+
</ins><span class="cx"> Load16 U:G:16, ZD:G:32
</span><span class="cx"> Addr, Tmp
</span><span class="cx"> Index, Tmp
</span><span class="cx">
</span><ins>+LoadAcq16 U:G:16, ZD:G:32 /effects
+ x86: Addr, Tmp
+ x86: Index, Tmp
+ SimpleAddr, Tmp
+
</ins><span class="cx"> Load16SignedExtendTo32 U:G:16, ZD:G:32
</span><span class="cx"> Addr, Tmp
</span><span class="cx"> Index, Tmp
</span><span class="cx">
</span><ins>+LoadAcq16SignedExtendTo32 U:G:16, ZD:G:32 /effects
+ x86: Addr, Tmp
+ x86: Index, Tmp
+ SimpleAddr, Tmp
+
</ins><span class="cx"> Store16 U:G:16, D:G:16
</span><span class="cx"> Tmp, Index
</span><span class="cx"> Tmp, Addr
</span><ins>+ x86: Imm, Index
+ x86: Imm, Addr
</ins><span class="cx">
</span><ins>+StoreRel16 U:G:16, D:G:16 /effects
+ x86: Tmp, Index
+ x86: Tmp, Addr
+ x86: Imm, Index
+ x86: Imm, Addr
+ Tmp, SimpleAddr
+
+LoadAcq32 U:G:32, ZD:G:32 /effects
+ x86: Addr, Tmp
+ x86: Index, Tmp
+ SimpleAddr, Tmp
+
+StoreRel32 U:G:32, ZD:G:32 /effects
+ x86: Tmp, Addr
+ x86: Tmp, Index
+ x86: Imm, Addr
+ x86: Imm, Index
+ Tmp, SimpleAddr
+
+64: LoadAcq64 U:G:64, ZD:G:64 /effects
+ x86: Addr, Tmp
+ x86: Index, Tmp
+ SimpleAddr, Tmp
+
+64: StoreRel64 U:G:64, ZD:G:64 /effects
+ x86: Tmp, Addr
+ x86: Tmp, Index
+ x86: Imm, Addr
+ x86: Imm, Index
+ Tmp, SimpleAddr
+
+# The first operand is rax.
+# FIXME: This formulation means that the boolean result cannot be put in eax, even though all users
+# of this would be OK with that.
+# https://bugs.webkit.org/show_bug.cgi?id=169254
+x86: AtomicStrongCAS8 U:G:32, UD:G:8, U:G:8, UD:G:8, ZD:G:8 /effects
+ StatusCond, Tmp*, Tmp, Addr, Tmp
+ StatusCond, Tmp*, Tmp, Index, Tmp
+
+x86: AtomicStrongCAS16 U:G:32, UD:G:16, U:G:32, UD:G:16, ZD:G:8 /effects
+ StatusCond, Tmp*, Tmp, Addr, Tmp
+ StatusCond, Tmp*, Tmp, Index, Tmp
+
+x86: AtomicStrongCAS32 U:G:32, UD:G:32, U:G:32, UD:G:32, ZD:G:8 /effects
+ StatusCond, Tmp*, Tmp, Addr, Tmp
+ StatusCond, Tmp*, Tmp, Index, Tmp
+
+x86_64: AtomicStrongCAS64 U:G:32, UD:G:64, U:G:64, UD:G:64, ZD:G:8 /effects
+ StatusCond, Tmp*, Tmp, Addr, Tmp
+ StatusCond, Tmp*, Tmp, Index, Tmp
+
+x86: AtomicStrongCAS8 UD:G:8, U:G:8, UD:G:8 /effects
+ Tmp*, Tmp, Addr
+ Tmp*, Tmp, Index
+
+x86: AtomicStrongCAS16 UD:G:16, U:G:32, UD:G:16 /effects
+ Tmp*, Tmp, Addr
+ Tmp*, Tmp, Index
+
+x86: AtomicStrongCAS32 UD:G:32, U:G:32, UD:G:32 /effects
+ Tmp*, Tmp, Addr
+ Tmp*, Tmp, Index
+
+x86_64: AtomicStrongCAS64 UD:G:64, U:G:64, UD:G:64 /effects
+ Tmp*, Tmp, Addr
+ Tmp*, Tmp, Index
+
+x86: BranchAtomicStrongCAS8 U:G:32, UD:G:8, U:G:8, UD:G:8 /branch /effects
+ StatusCond, Tmp*, Tmp, Addr
+ StatusCond, Tmp*, Tmp, Index
+
+x86: BranchAtomicStrongCAS16 U:G:32, UD:G:16, U:G:32, UD:G:16 /branch /effects
+ StatusCond, Tmp*, Tmp, Addr
+ StatusCond, Tmp*, Tmp, Index
+
+x86: BranchAtomicStrongCAS32 U:G:32, UD:G:32, U:G:32, UD:G:32 /branch /effects
+ StatusCond, Tmp*, Tmp, Addr
+ StatusCond, Tmp*, Tmp, Index
+
+x86_64: BranchAtomicStrongCAS64 U:G:32, UD:G:64, U:G:64, UD:G:64 /branch /effects
+ StatusCond, Tmp*, Tmp, Addr
+ StatusCond, Tmp*, Tmp, Index
+
+x86: AtomicAdd8 U:G:8, UD:G:8 /effects
+ Imm, Addr
+ Imm, Index
+ Tmp, Addr
+ Tmp, Index
+
+x86: AtomicAdd16 U:G:16, UD:G:16 /effects
+ Imm, Addr
+ Imm, Index
+ Tmp, Addr
+ Tmp, Index
+
+x86: AtomicAdd32 U:G:32, UD:G:32 /effects
+ Imm, Addr
+ Imm, Index
+ Tmp, Addr
+ Tmp, Index
+
+x86_64: AtomicAdd64 U:G:64, UD:G:64 /effects
+ Imm, Addr
+ Imm, Index
+ Tmp, Addr
+ Tmp, Index
+
+x86: AtomicSub8 U:G:8, UD:G:8 /effects
+ Imm, Addr
+ Imm, Index
+ Tmp, Addr
+ Tmp, Index
+
+x86: AtomicSub16 U:G:16, UD:G:16 /effects
+ Imm, Addr
+ Imm, Index
+ Tmp, Addr
+ Tmp, Index
+
+x86: AtomicSub32 U:G:32, UD:G:32 /effects
+ Imm, Addr
+ Imm, Index
+ Tmp, Addr
+ Tmp, Index
+
+x86_64: AtomicSub64 U:G:64, UD:G:64 /effects
+ Imm, Addr
+ Imm, Index
+ Tmp, Addr
+ Tmp, Index
+
+x86: AtomicAnd8 U:G:8, UD:G:8 /effects
+ Imm, Addr
+ Imm, Index
+ Tmp, Addr
+ Tmp, Index
+
+x86: AtomicAnd16 U:G:16, UD:G:16 /effects
+ Imm, Addr
+ Imm, Index
+ Tmp, Addr
+ Tmp, Index
+
+x86: AtomicAnd32 U:G:32, UD:G:32 /effects
+ Imm, Addr
+ Imm, Index
+ Tmp, Addr
+ Tmp, Index
+
+x86_64: AtomicAnd64 U:G:64, UD:G:64 /effects
+ Imm, Addr
+ Imm, Index
+ Tmp, Addr
+ Tmp, Index
+
+x86: AtomicOr8 U:G:8, UD:G:8 /effects
+ Imm, Addr
+ Imm, Index
+ Tmp, Addr
+ Tmp, Index
+
+x86: AtomicOr16 U:G:16, UD:G:16 /effects
+ Imm, Addr
+ Imm, Index
+ Tmp, Addr
+ Tmp, Index
+
+x86: AtomicOr32 U:G:32, UD:G:32 /effects
+ Imm, Addr
+ Imm, Index
+ Tmp, Addr
+ Tmp, Index
+
+x86_64: AtomicOr64 U:G:64, UD:G:64 /effects
+ Imm, Addr
+ Imm, Index
+ Tmp, Addr
+ Tmp, Index
+
+x86: AtomicXor8 U:G:8, UD:G:8 /effects
+ Imm, Addr
+ Imm, Index
+ Tmp, Addr
+ Tmp, Index
+
+x86: AtomicXor16 U:G:16, UD:G:16 /effects
+ Imm, Addr
+ Imm, Index
+ Tmp, Addr
+ Tmp, Index
+
+x86: AtomicXor32 U:G:32, UD:G:32 /effects
+ Imm, Addr
+ Imm, Index
+ Tmp, Addr
+ Tmp, Index
+
+x86_64: AtomicXor64 U:G:64, UD:G:64 /effects
+ Imm, Addr
+ Imm, Index
+ Tmp, Addr
+ Tmp, Index
+
+x86: AtomicNeg8 UD:G:8 /effects
+ Addr
+ Index
+
+x86: AtomicNeg16 UD:G:16 /effects
+ Addr
+ Index
+
+x86: AtomicNeg32 UD:G:32 /effects
+ Addr
+ Index
+
+x86_64: AtomicNeg64 UD:G:64 /effects
+ Addr
+ Index
+
+x86: AtomicNot8 UD:G:8 /effects
+ Addr
+ Index
+
+x86: AtomicNot16 UD:G:16 /effects
+ Addr
+ Index
+
+x86: AtomicNot32 UD:G:32 /effects
+ Addr
+ Index
+
+x86_64: AtomicNot64 UD:G:64 /effects
+ Addr
+ Index
+
+x86: AtomicXchgAdd8 UD:G:8, UD:G:8 /effects
+ Tmp, Addr
+ Tmp, Index
+
+x86: AtomicXchgAdd16 UD:G:16, UD:G:16 /effects
+ Tmp, Addr
+ Tmp, Index
+
+x86: AtomicXchgAdd32 UD:G:32, UD:G:32 /effects
+ Tmp, Addr
+ Tmp, Index
+
+x86_64: AtomicXchgAdd64 UD:G:64, UD:G:64 /effects
+ Tmp, Addr
+ Tmp, Index
+
+x86: AtomicXchg8 UD:G:8, UD:G:8 /effects
+ Tmp, Addr
+ Tmp, Index
+
+x86: AtomicXchg16 UD:G:16, UD:G:16 /effects
+ Tmp, Addr
+ Tmp, Index
+
+x86: AtomicXchg32 UD:G:32, UD:G:32 /effects
+ Tmp, Addr
+ Tmp, Index
+
+x86_64: AtomicXchg64 UD:G:64, UD:G:64 /effects
+ Tmp, Addr
+ Tmp, Index
+
+arm64: LoadLink8 U:G:8, ZD:G:8 /effects
+ SimpleAddr, Tmp
+
+arm64: LoadLinkAcq8 U:G:8, ZD:G:8 /effects
+ SimpleAddr, Tmp
+
+# Super confusing fact: this returns 0 to mean success, 1 to mean failure.
+arm64: StoreCond8 U:G:8, D:G:8, EZD:G:8 /effects
+ Tmp, SimpleAddr, Tmp
+
+arm64: StoreCondRel8 U:G:8, D:G:8, EZD:G:8 /effects
+ Tmp, SimpleAddr, Tmp
+
+arm64: LoadLink16 U:G:16, ZD:G:16 /effects
+ SimpleAddr, Tmp
+
+arm64: LoadLinkAcq16 U:G:16, ZD:G:16 /effects
+ SimpleAddr, Tmp
+
+arm64: StoreCond16 U:G:16, D:G:16, EZD:G:8 /effects
+ Tmp, SimpleAddr, Tmp
+
+arm64: StoreCondRel16 U:G:16, D:G:16, EZD:G:8 /effects
+ Tmp, SimpleAddr, Tmp
+
+arm64: LoadLink32 U:G:32, ZD:G:32 /effects
+ SimpleAddr, Tmp
+
+arm64: LoadLinkAcq32 U:G:32, ZD:G:32 /effects
+ SimpleAddr, Tmp
+
+arm64: StoreCond32 U:G:32, D:G:32, EZD:G:8 /effects
+ Tmp, SimpleAddr, Tmp
+
+arm64: StoreCondRel32 U:G:32, D:G:32, EZD:G:8 /effects
+ Tmp, SimpleAddr, Tmp
+
+arm64: LoadLink64 U:G:64, ZD:G:64 /effects
+ SimpleAddr, Tmp
+
+arm64: LoadLinkAcq64 U:G:64, ZD:G:64 /effects
+ SimpleAddr, Tmp
+
+arm64: StoreCond64 U:G:64, D:G:64, EZD:G:8 /effects
+ Tmp, SimpleAddr, Tmp
+
+arm64: StoreCondRel64 U:G:64, D:G:64, EZD:G:8 /effects
+ Tmp, SimpleAddr, Tmp
+
+arm64: Depend32 U:G:32, ZD:G:32
+ Tmp, Tmp
+
+arm64: Depend64 U:G:64, ZD:G:64
+ Tmp, Tmp
+
</ins><span class="cx"> Compare32 U:G:32, U:G:32, U:G:32, ZD:G:32
</span><span class="cx"> RelCond, Tmp, Tmp, Tmp
</span><span class="cx"> RelCond, Tmp, Imm, Tmp
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreb3airAirOptimizeBlockOrdercpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/b3/air/AirOptimizeBlockOrder.cpp (213713 => 213714)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/b3/air/AirOptimizeBlockOrder.cpp 2017-03-10 17:32:07 UTC (rev 213713)
+++ trunk/Source/JavaScriptCore/b3/air/AirOptimizeBlockOrder.cpp 2017-03-10 17:49:42 UTC (rev 213714)
</span><span class="lines">@@ -1,5 +1,5 @@
</span><span class="cx"> /*
</span><del>- * Copyright (C) 2015-2016 Apple Inc. All rights reserved.
</del><ins>+ * Copyright (C) 2015-2017 Apple Inc. All rights reserved.
</ins><span class="cx"> *
</span><span class="cx"> * Redistribution and use in source and binary forms, with or without
</span><span class="cx"> * modification, are permitted provided that the following conditions
</span><span class="lines">@@ -177,6 +177,10 @@
</span><span class="cx"> case BranchSub64:
</span><span class="cx"> case BranchNeg32:
</span><span class="cx"> case BranchNeg64:
</span><ins>+ case BranchAtomicStrongCAS8:
+ case BranchAtomicStrongCAS16:
+ case BranchAtomicStrongCAS32:
+ case BranchAtomicStrongCAS64:
</ins><span class="cx"> if (code.findNextBlock(block) == block->successorBlock(0) && branch.args[0].isInvertible()) {
</span><span class="cx"> std::swap(block->successor(0), block->successor(1));
</span><span class="cx"> branch.args[0] = branch.args[0].inverted();
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreb3airAirPadInterferencecpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/b3/air/AirPadInterference.cpp (213713 => 213714)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/b3/air/AirPadInterference.cpp 2017-03-10 17:32:07 UTC (rev 213713)
+++ trunk/Source/JavaScriptCore/b3/air/AirPadInterference.cpp 2017-03-10 17:49:42 UTC (rev 213714)
</span><span class="lines">@@ -48,6 +48,7 @@
</span><span class="cx"> [&] (Arg&, Arg::Role role, Bank, Width) {
</span><span class="cx"> switch (role) {
</span><span class="cx"> case Arg::EarlyDef:
</span><ins>+ case Arg::EarlyZDef:
</ins><span class="cx"> hasEarlyDef = true;
</span><span class="cx"> break;
</span><span class="cx"> case Arg::LateUse:
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreb3airAirSpillEverythingcpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/b3/air/AirSpillEverything.cpp (213713 => 213714)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/b3/air/AirSpillEverything.cpp 2017-03-10 17:32:07 UTC (rev 213713)
+++ trunk/Source/JavaScriptCore/b3/air/AirSpillEverything.cpp 2017-03-10 17:49:42 UTC (rev 213714)
</span><span class="lines">@@ -155,6 +155,7 @@
</span><span class="cx"> case Arg::LateColdUse:
</span><span class="cx"> case Arg::Scratch:
</span><span class="cx"> case Arg::EarlyDef:
</span><ins>+ case Arg::EarlyZDef:
</ins><span class="cx"> for (Reg reg : code.regsInPriorityOrder(bank)) {
</span><span class="cx"> if (!setBefore.get(reg) && !setAfter.get(reg)) {
</span><span class="cx"> setAfter.set(reg);
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreb3airopcode_generatorrb"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/b3/air/opcode_generator.rb (213713 => 213714)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/b3/air/opcode_generator.rb 2017-03-10 17:32:07 UTC (rev 213713)
+++ trunk/Source/JavaScriptCore/b3/air/opcode_generator.rb 2017-03-10 17:49:42 UTC (rev 213714)
</span><span class="lines">@@ -51,8 +51,8 @@
</span><span class="cx"> @bank = bank
</span><span class="cx"> @width = width
</span><span class="cx"> end
</span><del>-
- def widthCode
</del><ins>+
+ def self.widthCode(width)
</ins><span class="cx"> if width == "Ptr"
</span><span class="cx"> "pointerWidth()"
</span><span class="cx"> else
</span><span class="lines">@@ -59,6 +59,41 @@
</span><span class="cx"> "Width#{width}"
</span><span class="cx"> end
</span><span class="cx"> end
</span><ins>+
+ def widthCode
+ Arg.widthCode(width)
+ end
+
+ def self.roleCode(role)
+ case role
+ when "U"
+ "Use"
+ when "D"
+ "Def"
+ when "ZD"
+ "ZDef"
+ when "UD"
+ "UseDef"
+ when "UZD"
+ "UseZDef"
+ when "UA"
+ "UseAddr"
+ when "S"
+ "Scratch"
+ when "ED"
+ "EarlyDef"
+ when "EZD"
+ "EarlyZDef"
+ when "LU"
+ "LateUse"
+ else
+ raise
+ end
+ end
+
+ def roleCode
+ Arg.roleCode(role)
+ end
</ins><span class="cx"> end
</span><span class="cx">
</span><span class="cx"> class Overload
</span><span class="lines">@@ -182,7 +217,7 @@
</span><span class="cx"> end
</span><span class="cx">
</span><span class="cx"> def isRole(token)
</span><del>- token =~ /\A((U)|(D)|(UD)|(ZD)|(UZD)|(UA)|(S))\Z/
</del><ins>+ token =~ /\A((U)|(D)|(UD)|(ZD)|(UZD)|(UA)|(S)|(ED)|(EZD)|(LU))\Z/
</ins><span class="cx"> end
</span><span class="cx">
</span><span class="cx"> def isGF(token)
</span><span class="lines">@@ -190,7 +225,7 @@
</span><span class="cx"> end
</span><span class="cx">
</span><span class="cx"> def isKind(token)
</span><del>- token =~ /\A((Tmp)|(Imm)|(BigImm)|(BitImm)|(BitImm64)|(Addr)|(Index)|(RelCond)|(ResCond)|(DoubleCond))\Z/
</del><ins>+ token =~ /\A((Tmp)|(Imm)|(BigImm)|(BitImm)|(BitImm64)|(SimpleAddr)|(Addr)|(Index)|(RelCond)|(ResCond)|(DoubleCond)|(StatusCond))\Z/
</ins><span class="cx"> end
</span><span class="cx">
</span><span class="cx"> def isArch(token)
</span><span class="lines">@@ -264,7 +299,7 @@
</span><span class="cx">
</span><span class="cx"> def consumeKind
</span><span class="cx"> result = token.string
</span><del>- parseError("Expected kind (Imm, BigImm, BitImm, BitImm64, Tmp, Addr, Index, RelCond, ResCond, or DoubleCond)") unless isKind(result)
</del><ins>+ parseError("Expected kind (Imm, BigImm, BitImm, BitImm64, Tmp, SimpleAddr, Addr, Index, RelCond, ResCond, DoubleCond, or StatusCond)") unless isKind(result)
</ins><span class="cx"> advance
</span><span class="cx"> result
</span><span class="cx"> end
</span><span class="lines">@@ -634,28 +669,7 @@
</span><span class="cx"> else
</span><span class="cx"> overload.signature.each_with_index {
</span><span class="cx"> | arg, index |
</span><del>-
- role = nil
- case arg.role
- when "U"
- role = "Use"
- when "D"
- role = "Def"
- when "ZD"
- role = "ZDef"
- when "UD"
- role = "UseDef"
- when "UZD"
- role = "UseZDef"
- when "UA"
- role = "UseAddr"
- when "S"
- role = "Scratch"
- else
- raise
- end
-
- outp.puts "functor(args[#{index}], Arg::#{role}, #{arg.bank}P, #{arg.widthCode});"
</del><ins>+ outp.puts "functor(args[#{index}], Arg::#{arg.roleCode}, #{arg.bank}P, #{arg.widthCode});"
</ins><span class="cx"> }
</span><span class="cx"> end
</span><span class="cx"> }
</span><span class="lines">@@ -804,6 +818,9 @@
</span><span class="cx"> when "BitImm64"
</span><span class="cx"> outp.puts "if (!Arg::isValidBitImm64Form(args[#{index}].value()))"
</span><span class="cx"> outp.puts "OPGEN_RETURN(false);"
</span><ins>+ when "SimpleAddr"
+ outp.puts "if (!args[#{index}].tmp().isGP())"
+ outp.puts "OPGEN_RETURN(false);"
</ins><span class="cx"> when "Addr"
</span><span class="cx"> if arg.role == "UA"
</span><span class="cx"> outp.puts "if (args[#{index}].isStack() && args[#{index}].stackSlot()->isSpill())"
</span><span class="lines">@@ -819,6 +836,7 @@
</span><span class="cx"> when "RelCond"
</span><span class="cx"> when "ResCond"
</span><span class="cx"> when "DoubleCond"
</span><ins>+ when "StatusCond"
</ins><span class="cx"> else
</span><span class="cx"> raise "Unexpected kind: #{kind.name}"
</span><span class="cx"> end
</span><span class="lines">@@ -1095,7 +1113,7 @@
</span><span class="cx"> outp.print "args[#{index}].asTrustedImm32()"
</span><span class="cx"> when "BigImm", "BitImm64"
</span><span class="cx"> outp.print "args[#{index}].asTrustedImm64()"
</span><del>- when "Addr"
</del><ins>+ when "SimpleAddr", "Addr"
</ins><span class="cx"> outp.print "args[#{index}].asAddress()"
</span><span class="cx"> when "Index"
</span><span class="cx"> outp.print "args[#{index}].asBaseIndex()"
</span><span class="lines">@@ -1105,6 +1123,8 @@
</span><span class="cx"> outp.print "args[#{index}].asResultCondition()"
</span><span class="cx"> when "DoubleCond"
</span><span class="cx"> outp.print "args[#{index}].asDoubleCondition()"
</span><ins>+ when "StatusCond"
+ outp.print "args[#{index}].asStatusCondition()"
</ins><span class="cx"> end
</span><span class="cx"> }
</span><span class="cx">
</span><span class="lines">@@ -1148,27 +1168,7 @@
</span><span class="cx"> outp.puts "case #{overload.signature.length}:" if needOverloadSwitch
</span><span class="cx"> overload.signature.each_with_index {
</span><span class="cx"> | arg, index |
</span><del>- role = nil
- case arg.role
- when "U"
- role = "Use"
- when "D"
- role = "Def"
- when "ZD"
- role = "ZDef"
- when "UD"
- role = "UseDef"
- when "UZD"
- role = "UseZDef"
- when "UA"
- role = "UseAddr"
- when "S"
- role = "Scratch"
- else
- raise
- end
-
- outp.puts "inst.visitArg(#{index}, func, Arg.#{role}, #{arg.bank}P, #{arg.width});"
</del><ins>+ outp.puts "inst.visitArg(#{index}, func, Arg.#{arg.roleCode}, #{arg.bank}P, #{arg.width});"
</ins><span class="cx"> }
</span><span class="cx"> outp.puts "break;"
</span><span class="cx"> }
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreb3testb3cpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/b3/testb3.cpp (213713 => 213714)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/b3/testb3.cpp 2017-03-10 17:32:07 UTC (rev 213713)
+++ trunk/Source/JavaScriptCore/b3/testb3.cpp 2017-03-10 17:49:42 UTC (rev 213714)
</span><span class="lines">@@ -30,6 +30,7 @@
</span><span class="cx"> #include "AirValidate.h"
</span><span class="cx"> #include "AllowMacroScratchRegisterUsage.h"
</span><span class="cx"> #include "B3ArgumentRegValue.h"
</span><ins>+#include "B3AtomicValue.h"
</ins><span class="cx"> #include "B3BasicBlockInlines.h"
</span><span class="cx"> #include "B3CCallValue.h"
</span><span class="cx"> #include "B3Compilation.h"
</span><span class="lines">@@ -44,6 +45,7 @@
</span><span class="cx"> #include "B3MathExtras.h"
</span><span class="cx"> #include "B3MemoryValue.h"
</span><span class="cx"> #include "B3MoveConstants.h"
</span><ins>+#include "B3NativeTraits.h"
</ins><span class="cx"> #include "B3Procedure.h"
</span><span class="cx"> #include "B3ReduceStrength.h"
</span><span class="cx"> #include "B3SlotBaseValue.h"
</span><span class="lines">@@ -297,6 +299,24 @@
</span><span class="cx"> CHECK(compileAndRun<int>(proc) == 42);
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+void testLoadAcq42()
+{
+ Procedure proc;
+ BasicBlock* root = proc.addBlock();
+ int x = 42;
+ root->appendNewControlValue(
+ proc, Return, Origin(),
+ root->appendNew<MemoryValue>(
+ proc, Load, Int32, Origin(),
+ root->appendNew<ConstPtrValue>(proc, Origin(), &x),
+ 0, HeapRange(42), HeapRange(42)));
+
+ auto code = compile(proc);
+ if (isARM64())
+ checkUsesInstruction(*code, "lda");
+ CHECK(invoke<int>(*code) == 42);
+}
+
</ins><span class="cx"> void testLoadWithOffsetImpl(int32_t offset64, int32_t offset32)
</span><span class="cx"> {
</span><span class="cx"> {
</span><span class="lines">@@ -5748,6 +5768,35 @@
</span><span class="cx"> CHECK(slot == 37 + amount);
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+void testStoreRelAddLoadAcq32(int amount)
+{
+ Procedure proc;
+ BasicBlock* root = proc.addBlock();
+ int slot = 37;
+ ConstPtrValue* slotPtr = root->appendNew<ConstPtrValue>(proc, Origin(), &slot);
+ root->appendNew<MemoryValue>(
+ proc, Store, Origin(),
+ root->appendNew<Value>(
+ proc, Add, Origin(),
+ root->appendNew<MemoryValue>(
+ proc, Load, Int32, Origin(), slotPtr, 0, HeapRange(42), HeapRange(42)),
+ root->appendNew<Value>(
+ proc, Trunc, Origin(),
+ root->appendNew<ArgumentRegValue>(proc, Origin(), GPRInfo::argumentGPR0))),
+ slotPtr, 0, HeapRange(42), HeapRange(42));
+ root->appendNewControlValue(
+ proc, Return, Origin(),
+ root->appendNew<Const32Value>(proc, Origin(), 0));
+
+ auto code = compile(proc);
+ if (isARM64()) {
+ checkUsesInstruction(*code, "lda");
+ checkUsesInstruction(*code, "stl");
+ }
+ CHECK(!invoke<int>(*code, amount));
+ CHECK(slot == 37 + amount);
+}
+
</ins><span class="cx"> void testStoreAddLoadImm32(int amount)
</span><span class="cx"> {
</span><span class="cx"> Procedure proc;
</span><span class="lines">@@ -5792,6 +5841,74 @@
</span><span class="cx"> CHECK(slot == 37 + amount);
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+void testStoreRelAddLoadAcq8(int amount, B3::Opcode loadOpcode)
+{
+ Procedure proc;
+ BasicBlock* root = proc.addBlock();
+ int8_t slot = 37;
+ ConstPtrValue* slotPtr = root->appendNew<ConstPtrValue>(proc, Origin(), &slot);
+ root->appendNew<MemoryValue>(
+ proc, Store8, Origin(),
+ root->appendNew<Value>(
+ proc, Add, Origin(),
+ root->appendNew<MemoryValue>(
+ proc, loadOpcode, Origin(), slotPtr, 0, HeapRange(42), HeapRange(42)),
+ root->appendNew<Value>(
+ proc, Trunc, Origin(),
+ root->appendNew<ArgumentRegValue>(proc, Origin(), GPRInfo::argumentGPR0))),
+ slotPtr, 0, HeapRange(42), HeapRange(42));
+ root->appendNewControlValue(
+ proc, Return, Origin(),
+ root->appendNew<Const32Value>(proc, Origin(), 0));
+
+ auto code = compile(proc);
+ if (isARM64()) {
+ checkUsesInstruction(*code, "lda");
+ checkUsesInstruction(*code, "stl");
+ }
+ CHECK(!invoke<int>(*code, amount));
+ CHECK(slot == 37 + amount);
+}
+
+void testStoreRelAddFenceLoadAcq8(int amount, B3::Opcode loadOpcode)
+{
+ Procedure proc;
+ BasicBlock* root = proc.addBlock();
+ int8_t slot = 37;
+ ConstPtrValue* slotPtr = root->appendNew<ConstPtrValue>(proc, Origin(), &slot);
+ Value* loadedValue = root->appendNew<MemoryValue>(
+ proc, loadOpcode, Origin(), slotPtr, 0, HeapRange(42), HeapRange(42));
+ PatchpointValue* patchpoint = root->appendNew<PatchpointValue>(proc, Void, Origin());
+ patchpoint->clobber(RegisterSet::macroScratchRegisters());
+ patchpoint->setGenerator(
+ [&] (CCallHelpers& jit, const StackmapGenerationParams&) {
+ AllowMacroScratchRegisterUsage allowScratch(jit);
+ jit.store8(CCallHelpers::TrustedImm32(0xbeef), &slot);
+ });
+ patchpoint->effects = Effects::none();
+ patchpoint->effects.fence = true;
+ root->appendNew<MemoryValue>(
+ proc, Store8, Origin(),
+ root->appendNew<Value>(
+ proc, Add, Origin(),
+ loadedValue,
+ root->appendNew<Value>(
+ proc, Trunc, Origin(),
+ root->appendNew<ArgumentRegValue>(proc, Origin(), GPRInfo::argumentGPR0))),
+ slotPtr, 0, HeapRange(42), HeapRange(42));
+ root->appendNewControlValue(
+ proc, Return, Origin(),
+ root->appendNew<Const32Value>(proc, Origin(), 0));
+
+ auto code = compile(proc);
+ if (isARM64()) {
+ checkUsesInstruction(*code, "lda");
+ checkUsesInstruction(*code, "stl");
+ }
+ CHECK(!invoke<int>(*code, amount));
+ CHECK(slot == 37 + amount);
+}
+
</ins><span class="cx"> void testStoreAddLoadImm8(int amount, B3::Opcode loadOpcode)
</span><span class="cx"> {
</span><span class="cx"> Procedure proc;
</span><span class="lines">@@ -5836,6 +5953,35 @@
</span><span class="cx"> CHECK(slot == 37 + amount);
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+void testStoreRelAddLoadAcq16(int amount, B3::Opcode loadOpcode)
+{
+ Procedure proc;
+ BasicBlock* root = proc.addBlock();
+ int16_t slot = 37;
+ ConstPtrValue* slotPtr = root->appendNew<ConstPtrValue>(proc, Origin(), &slot);
+ root->appendNew<MemoryValue>(
+ proc, Store16, Origin(),
+ root->appendNew<Value>(
+ proc, Add, Origin(),
+ root->appendNew<MemoryValue>(
+ proc, loadOpcode, Origin(), slotPtr, 0, HeapRange(42), HeapRange(42)),
+ root->appendNew<Value>(
+ proc, Trunc, Origin(),
+ root->appendNew<ArgumentRegValue>(proc, Origin(), GPRInfo::argumentGPR0))),
+ slotPtr, 0, HeapRange(42), HeapRange(42));
+ root->appendNewControlValue(
+ proc, Return, Origin(),
+ root->appendNew<Const32Value>(proc, Origin(), 0));
+
+ auto code = compile(proc);
+ if (isARM64()) {
+ checkUsesInstruction(*code, "lda");
+ checkUsesInstruction(*code, "stl");
+ }
+ CHECK(!invoke<int>(*code, amount));
+ CHECK(slot == 37 + amount);
+}
+
</ins><span class="cx"> void testStoreAddLoadImm16(int amount, B3::Opcode loadOpcode)
</span><span class="cx"> {
</span><span class="cx"> Procedure proc;
</span><span class="lines">@@ -5878,6 +6024,33 @@
</span><span class="cx"> CHECK(slot == 37000000000ll + amount);
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+void testStoreRelAddLoadAcq64(int amount)
+{
+ Procedure proc;
+ BasicBlock* root = proc.addBlock();
+ int64_t slot = 37000000000ll;
+ ConstPtrValue* slotPtr = root->appendNew<ConstPtrValue>(proc, Origin(), &slot);
+ root->appendNew<MemoryValue>(
+ proc, Store, Origin(),
+ root->appendNew<Value>(
+ proc, Add, Origin(),
+ root->appendNew<MemoryValue>(
+ proc, Load, Int64, Origin(), slotPtr, 0, HeapRange(42), HeapRange(42)),
+ root->appendNew<ArgumentRegValue>(proc, Origin(), GPRInfo::argumentGPR0)),
+ slotPtr, 0, HeapRange(42), HeapRange(42));
+ root->appendNewControlValue(
+ proc, Return, Origin(),
+ root->appendNew<Const32Value>(proc, Origin(), 0));
+
+ auto code = compile(proc);
+ if (isARM64()) {
+ checkUsesInstruction(*code, "lda");
+ checkUsesInstruction(*code, "stl");
+ }
+ CHECK(!invoke<int>(*code, amount));
+ CHECK(slot == 37000000000ll + amount);
+}
+
</ins><span class="cx"> void testStoreAddLoadImm64(int64_t amount)
</span><span class="cx"> {
</span><span class="cx"> Procedure proc;
</span><span class="lines">@@ -13791,7 +13964,7 @@
</span><span class="cx"> compileAndRun<int>(proc);
</span><span class="cx"> unsigned storeCount = 0;
</span><span class="cx"> for (Value* value : proc.values()) {
</span><del>- if (MemoryValue::isStore(value->opcode()))
</del><ins>+ if (isStore(value->opcode()))
</ins><span class="cx"> storeCount++;
</span><span class="cx"> }
</span><span class="cx"> CHECK_EQ(storeCount, 2u);
</span><span class="lines">@@ -13997,6 +14170,7 @@
</span><span class="cx"> root->appendNew<Value>(proc, Return, Origin(), result);
</span><span class="cx">
</span><span class="cx"> auto code = compile(proc);
</span><ins>+ CHECK_EQ(invoke<intptr_t>(*code, 1, 2), (1 + 2) + 100);
</ins><span class="cx"> checkDisassembly(
</span><span class="cx"> *code,
</span><span class="cx"> [&] (const char* disassembly) -> bool {
</span><span class="lines">@@ -14004,7 +14178,6 @@
</span><span class="cx"> || strstr(disassembly, "lea 0x64(%rsi,%rdi), %rax");
</span><span class="cx"> },
</span><span class="cx"> "Expected to find something like lea 0x64(%rdi,%rsi), %rax but didn't!");
</span><del>- CHECK_EQ(invoke<intptr_t>(*code, 1, 2), (1 + 2) + 100);
</del><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> void testX86LeaAddShlRight()
</span><span class="lines">@@ -14041,6 +14214,7 @@
</span><span class="cx"> root->appendNew<Value>(proc, Return, Origin(), result);
</span><span class="cx">
</span><span class="cx"> auto code = compile(proc);
</span><ins>+ CHECK_EQ(invoke<intptr_t>(*code, 1, 2), 1 + 2);
</ins><span class="cx"> checkDisassembly(
</span><span class="cx"> *code,
</span><span class="cx"> [&] (const char* disassembly) -> bool {
</span><span class="lines">@@ -14048,7 +14222,6 @@
</span><span class="cx"> || strstr(disassembly, "lea (%rsi,%rdi), %rax");
</span><span class="cx"> },
</span><span class="cx"> "Expected to find something like lea (%rdi,%rsi), %rax but didn't!");
</span><del>- CHECK_EQ(invoke<intptr_t>(*code, 1, 2), 1 + 2);
</del><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> void testX86LeaAddShlLeftScale2()
</span><span class="lines">@@ -14277,6 +14450,684 @@
</span><span class="cx"> CHECK(found);
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+template<typename T>
+void testAtomicWeakCAS()
+{
+ Type type = NativeTraits<T>::type;
+ Width width = NativeTraits<T>::width;
+
+ auto checkMyDisassembly = [&] (Compilation& compilation, bool fenced) {
+ if (isX86()) {
+ checkUsesInstruction(compilation, "lock");
+ checkUsesInstruction(compilation, "cmpxchg");
+ } else {
+ if (fenced) {
+ checkUsesInstruction(compilation, "ldax");
+ checkUsesInstruction(compilation, "stlx");
+ } else {
+ checkUsesInstruction(compilation, "ldx");
+ checkUsesInstruction(compilation, "stx");
+ }
+ }
+ };
+
+ {
+ Procedure proc;
+ BasicBlock* root = proc.addBlock();
+ BasicBlock* reloop = proc.addBlock();
+ BasicBlock* done = proc.addBlock();
+
+ Value* ptr = root->appendNew<ArgumentRegValue>(proc, Origin(), GPRInfo::argumentGPR0);
+ root->appendNew<Value>(proc, Jump, Origin());
+ root->setSuccessors(reloop);
+
+ reloop->appendNew<Value>(
+ proc, Branch, Origin(),
+ reloop->appendNew<AtomicValue>(
+ proc, AtomicWeakCAS, Origin(), width,
+ reloop->appendIntConstant(proc, Origin(), type, 42),
+ reloop->appendIntConstant(proc, Origin(), type, 0xbeef),
+ ptr));
+ reloop->setSuccessors(done, reloop);
+
+ done->appendNew<Value>(proc, Return, Origin());
+
+ auto code = compile(proc);
+ T value[2];
+ value[0] = 42;
+ value[1] = 13;
+ invoke<void>(*code, value);
+ CHECK_EQ(value[0], static_cast<T>(0xbeef));
+ CHECK_EQ(value[1], 13);
+ checkMyDisassembly(*code, true);
+ }
+
+ {
+ Procedure proc;
+ BasicBlock* root = proc.addBlock();
+ BasicBlock* reloop = proc.addBlock();
+ BasicBlock* done = proc.addBlock();
+
+ Value* ptr = root->appendNew<ArgumentRegValue>(proc, Origin(), GPRInfo::argumentGPR0);
+ root->appendNew<Value>(proc, Jump, Origin());
+ root->setSuccessors(reloop);
+
+ reloop->appendNew<Value>(
+ proc, Branch, Origin(),
+ reloop->appendNew<AtomicValue>(
+ proc, AtomicWeakCAS, Origin(), width,
+ reloop->appendIntConstant(proc, Origin(), type, 42),
+ reloop->appendIntConstant(proc, Origin(), type, 0xbeef),
+ ptr, 0, HeapRange(42), HeapRange()));
+ reloop->setSuccessors(done, reloop);
+
+ done->appendNew<Value>(proc, Return, Origin());
+
+ auto code = compile(proc);
+ T value[2];
+ value[0] = 42;
+ value[1] = 13;
+ invoke<void>(*code, value);
+ CHECK_EQ(value[0], static_cast<T>(0xbeef));
+ CHECK_EQ(value[1], 13);
+ checkMyDisassembly(*code, false);
+ }
+
+ {
+ Procedure proc;
+ BasicBlock* root = proc.addBlock();
+ BasicBlock* succ = proc.addBlock();
+ BasicBlock* fail = proc.addBlock();
+
+ Value* ptr = root->appendNew<ArgumentRegValue>(proc, Origin(), GPRInfo::argumentGPR0);
+ root->appendNew<Value>(
+ proc, Branch, Origin(),
+ root->appendNew<AtomicValue>(
+ proc, AtomicWeakCAS, Origin(), width,
+ root->appendIntConstant(proc, Origin(), type, 42),
+ root->appendIntConstant(proc, Origin(), type, 0xbeef),
+ ptr));
+ root->setSuccessors(succ, fail);
+
+ succ->appendNew<MemoryValue>(
+ proc, storeOpcode(GP, width), Origin(),
+ succ->appendIntConstant(proc, Origin(), type, 100),
+ ptr);
+ succ->appendNew<Value>(proc, Return, Origin());
+
+ fail->appendNew<Value>(proc, Return, Origin());
+
+ auto code = compile(proc);
+ T value[2];
+ value[0] = 42;
+ value[1] = 13;
+ while (value[0] == 42)
+ invoke<void>(*code, value);
+ CHECK_EQ(value[0], static_cast<T>(100));
+ CHECK_EQ(value[1], 13);
+ value[0] = static_cast<T>(300);
+ invoke<void>(*code, value);
+ CHECK_EQ(value[0], static_cast<T>(300));
+ CHECK_EQ(value[1], 13);
+ checkMyDisassembly(*code, true);
+ }
+
+ {
+ Procedure proc;
+ BasicBlock* root = proc.addBlock();
+ BasicBlock* succ = proc.addBlock();
+ BasicBlock* fail = proc.addBlock();
+
+ Value* ptr = root->appendNew<ArgumentRegValue>(proc, Origin(), GPRInfo::argumentGPR0);
+ root->appendNew<Value>(
+ proc, Branch, Origin(),
+ root->appendNew<Value>(
+ proc, Equal, Origin(),
+ root->appendNew<AtomicValue>(
+ proc, AtomicWeakCAS, Origin(), width,
+ root->appendIntConstant(proc, Origin(), type, 42),
+ root->appendIntConstant(proc, Origin(), type, 0xbeef),
+ ptr),
+ root->appendIntConstant(proc, Origin(), Int32, 0)));
+ root->setSuccessors(fail, succ);
+
+ succ->appendNew<MemoryValue>(
+ proc, storeOpcode(GP, width), Origin(),
+ succ->appendIntConstant(proc, Origin(), type, 100),
+ ptr);
+ succ->appendNew<Value>(proc, Return, Origin());
+
+ fail->appendNew<Value>(proc, Return, Origin());
+
+ auto code = compile(proc);
+ T value[2];
+ value[0] = 42;
+ value[1] = 13;
+ while (value[0] == 42)
+ invoke<void>(*code, value);
+ CHECK_EQ(value[0], static_cast<T>(100));
+ CHECK_EQ(value[1], 13);
+ value[0] = static_cast<T>(300);
+ invoke<void>(*code, value);
+ CHECK_EQ(value[0], static_cast<T>(300));
+ CHECK_EQ(value[1], 13);
+ checkMyDisassembly(*code, true);
+ }
+
+ {
+ Procedure proc;
+ BasicBlock* root = proc.addBlock();
+ root->appendNew<Value>(
+ proc, Return, Origin(),
+ root->appendNew<AtomicValue>(
+ proc, AtomicWeakCAS, Origin(), width,
+ root->appendIntConstant(proc, Origin(), type, 42),
+ root->appendIntConstant(proc, Origin(), type, 0xbeef),
+ root->appendNew<ArgumentRegValue>(proc, Origin(), GPRInfo::argumentGPR0)));
+
+ auto code = compile(proc);
+ T value[2];
+ value[0] = 42;
+ value[1] = 13;
+ while (!invoke<bool>(*code, value)) { }
+ CHECK_EQ(value[0], static_cast<T>(0xbeef));
+ CHECK_EQ(value[1], 13);
+
+ value[0] = static_cast<T>(300);
+ CHECK(!invoke<bool>(*code, value));
+ CHECK_EQ(value[0], static_cast<T>(300));
+ CHECK_EQ(value[1], 13);
+ checkMyDisassembly(*code, true);
+ }
+
+ {
+ Procedure proc;
+ BasicBlock* root = proc.addBlock();
+ root->appendNew<Value>(
+ proc, Return, Origin(),
+ root->appendNew<Value>(
+ proc, Equal, Origin(),
+ root->appendNew<AtomicValue>(
+ proc, AtomicWeakCAS, Origin(), width,
+ root->appendIntConstant(proc, Origin(), type, 42),
+ root->appendIntConstant(proc, Origin(), type, 0xbeef),
+ root->appendNew<ArgumentRegValue>(proc, Origin(), GPRInfo::argumentGPR0)),
+ root->appendNew<Const32Value>(proc, Origin(), 0)));
+
+ auto code = compile(proc);
+ T value[2];
+ value[0] = 42;
+ value[1] = 13;
+ while (invoke<bool>(*code, value)) { }
+ CHECK_EQ(value[0], static_cast<T>(0xbeef));
+ CHECK_EQ(value[1], 13);
+
+ value[0] = static_cast<T>(300);
+ CHECK(invoke<bool>(*code, value));
+ CHECK_EQ(value[0], static_cast<T>(300));
+ CHECK_EQ(value[1], 13);
+ checkMyDisassembly(*code, true);
+ }
+
+ {
+ Procedure proc;
+ BasicBlock* root = proc.addBlock();
+ root->appendNew<Value>(
+ proc, Return, Origin(),
+ root->appendNew<AtomicValue>(
+ proc, AtomicWeakCAS, Origin(), width,
+ root->appendIntConstant(proc, Origin(), type, 42),
+ root->appendIntConstant(proc, Origin(), type, 0xbeef),
+ root->appendNew<ArgumentRegValue>(proc, Origin(), GPRInfo::argumentGPR0),
+ 42));
+
+ auto code = compile(proc);
+ T value[2];
+ value[0] = 42;
+ value[1] = 13;
+ while (!invoke<bool>(*code, bitwise_cast<intptr_t>(value) - 42)) { }
+ CHECK_EQ(value[0], static_cast<T>(0xbeef));
+ CHECK_EQ(value[1], 13);
+
+ value[0] = static_cast<T>(300);
+ CHECK(!invoke<bool>(*code, bitwise_cast<intptr_t>(value) - 42));
+ CHECK_EQ(value[0], static_cast<T>(300));
+ CHECK_EQ(value[1], 13);
+ checkMyDisassembly(*code, true);
+ }
+}
+
+template<typename T>
+void testAtomicStrongCAS()
+{
+ Type type = NativeTraits<T>::type;
+ Width width = NativeTraits<T>::width;
+
+ auto checkMyDisassembly = [&] (Compilation& compilation, bool fenced) {
+ if (isX86()) {
+ checkUsesInstruction(compilation, "lock");
+ checkUsesInstruction(compilation, "cmpxchg");
+ } else {
+ if (fenced) {
+ checkUsesInstruction(compilation, "ldax");
+ checkUsesInstruction(compilation, "stlx");
+ } else {
+ checkUsesInstruction(compilation, "ldx");
+ checkUsesInstruction(compilation, "stx");
+ }
+ }
+ };
+
+ {
+ Procedure proc;
+ BasicBlock* root = proc.addBlock();
+ BasicBlock* succ = proc.addBlock();
+ BasicBlock* fail = proc.addBlock();
+
+ Value* ptr = root->appendNew<ArgumentRegValue>(proc, Origin(), GPRInfo::argumentGPR0);
+ root->appendNew<Value>(
+ proc, Branch, Origin(),
+ root->appendNew<Value>(
+ proc, Equal, Origin(),
+ root->appendNew<AtomicValue>(
+ proc, AtomicStrongCAS, Origin(), width,
+ root->appendIntConstant(proc, Origin(), type, 42),
+ root->appendIntConstant(proc, Origin(), type, 0xbeef),
+ ptr),
+ root->appendIntConstant(proc, Origin(), type, 42)));
+ root->setSuccessors(succ, fail);
+
+ succ->appendNew<MemoryValue>(
+ proc, storeOpcode(GP, width), Origin(),
+ succ->appendIntConstant(proc, Origin(), type, 100),
+ ptr);
+ succ->appendNew<Value>(proc, Return, Origin());
+
+ fail->appendNew<Value>(proc, Return, Origin());
+
+ auto code = compile(proc);
+ T value[2];
+ value[0] = 42;
+ value[1] = 13;
+ invoke<void>(*code, value);
+ CHECK_EQ(value[0], static_cast<T>(100));
+ CHECK_EQ(value[1], 13);
+ value[0] = static_cast<T>(300);
+ invoke<void>(*code, value);
+ CHECK_EQ(value[0], static_cast<T>(300));
+ CHECK_EQ(value[1], 13);
+ checkMyDisassembly(*code, true);
+ }
+
+ {
+ Procedure proc;
+ BasicBlock* root = proc.addBlock();
+ BasicBlock* succ = proc.addBlock();
+ BasicBlock* fail = proc.addBlock();
+
+ Value* ptr = root->appendNew<ArgumentRegValue>(proc, Origin(), GPRInfo::argumentGPR0);
+ root->appendNew<Value>(
+ proc, Branch, Origin(),
+ root->appendNew<Value>(
+ proc, Equal, Origin(),
+ root->appendNew<AtomicValue>(
+ proc, AtomicStrongCAS, Origin(), width,
+ root->appendIntConstant(proc, Origin(), type, 42),
+ root->appendIntConstant(proc, Origin(), type, 0xbeef),
+ ptr, 0, HeapRange(42), HeapRange()),
+ root->appendIntConstant(proc, Origin(), type, 42)));
+ root->setSuccessors(succ, fail);
+
+ succ->appendNew<MemoryValue>(
+ proc, storeOpcode(GP, width), Origin(),
+ succ->appendIntConstant(proc, Origin(), type, 100),
+ ptr);
+ succ->appendNew<Value>(proc, Return, Origin());
+
+ fail->appendNew<Value>(proc, Return, Origin());
+
+ auto code = compile(proc);
+ T value[2];
+ value[0] = 42;
+ value[1] = 13;
+ invoke<void>(*code, value);
+ CHECK_EQ(value[0], static_cast<T>(100));
+ CHECK_EQ(value[1], 13);
+ value[0] = static_cast<T>(300);
+ invoke<void>(*code, value);
+ CHECK_EQ(value[0], static_cast<T>(300));
+ CHECK_EQ(value[1], 13);
+ checkMyDisassembly(*code, false);
+ }
+
+ {
+ Procedure proc;
+ BasicBlock* root = proc.addBlock();
+ BasicBlock* succ = proc.addBlock();
+ BasicBlock* fail = proc.addBlock();
+
+ Value* ptr = root->appendNew<ArgumentRegValue>(proc, Origin(), GPRInfo::argumentGPR0);
+ root->appendNew<Value>(
+ proc, Branch, Origin(),
+ root->appendNew<Value>(
+ proc, NotEqual, Origin(),
+ root->appendNew<AtomicValue>(
+ proc, AtomicStrongCAS, Origin(), width,
+ root->appendIntConstant(proc, Origin(), type, 42),
+ root->appendIntConstant(proc, Origin(), type, 0xbeef),
+ ptr),
+ root->appendIntConstant(proc, Origin(), type, 42)));
+ root->setSuccessors(fail, succ);
+
+ succ->appendNew<MemoryValue>(
+ proc, storeOpcode(GP, width), Origin(),
+ succ->appendIntConstant(proc, Origin(), type, 100),
+ ptr);
+ succ->appendNew<Value>(proc, Return, Origin());
+
+ fail->appendNew<Value>(proc, Return, Origin());
+
+ auto code = compile(proc);
+ T value[2];
+ value[0] = 42;
+ value[1] = 13;
+ invoke<void>(*code, value);
+ CHECK_EQ(value[0], static_cast<T>(100));
+ CHECK_EQ(value[1], 13);
+ value[0] = static_cast<T>(300);
+ invoke<void>(*code, value);
+ CHECK_EQ(value[0], static_cast<T>(300));
+ CHECK_EQ(value[1], 13);
+ checkMyDisassembly(*code, true);
+ }
+
+ {
+ Procedure proc;
+ BasicBlock* root = proc.addBlock();
+ root->appendNew<Value>(
+ proc, Return, Origin(),
+ root->appendNew<AtomicValue>(
+ proc, AtomicStrongCAS, Origin(), width,
+ root->appendIntConstant(proc, Origin(), type, 42),
+ root->appendIntConstant(proc, Origin(), type, 0xbeef),
+ root->appendNew<ArgumentRegValue>(proc, Origin(), GPRInfo::argumentGPR0)));
+
+ auto code = compile(proc);
+ T value[2];
+ value[0] = 42;
+ value[1] = 13;
+ CHECK_EQ(invoke<typename NativeTraits<T>::CanonicalType>(*code, value), 42);
+ CHECK_EQ(value[0], static_cast<T>(0xbeef));
+ CHECK_EQ(value[1], 13);
+ value[0] = static_cast<T>(300);
+ CHECK_EQ(invoke<typename NativeTraits<T>::CanonicalType>(*code, value), static_cast<typename NativeTraits<T>::CanonicalType>(static_cast<T>(300)));
+ CHECK_EQ(value[0], static_cast<T>(300));
+ CHECK_EQ(value[1], 13);
+ value[0] = static_cast<T>(-1);
+ CHECK_EQ(invoke<typename NativeTraits<T>::CanonicalType>(*code, value), static_cast<typename NativeTraits<T>::CanonicalType>(static_cast<T>(-1)));
+ CHECK_EQ(value[0], static_cast<T>(-1));
+ CHECK_EQ(value[1], 13);
+ checkMyDisassembly(*code, true);
+ }
+
+ {
+ Procedure proc;
+ BasicBlock* root = proc.addBlock();
+ root->appendNew<Value>(
+ proc, Return, Origin(),
+ root->appendNew<Value>(
+ proc, Equal, Origin(),
+ root->appendNew<AtomicValue>(
+ proc, AtomicStrongCAS, Origin(), width,
+ root->appendIntConstant(proc, Origin(), type, 42),
+ root->appendIntConstant(proc, Origin(), type, 0xbeef),
+ root->appendNew<ArgumentRegValue>(proc, Origin(), GPRInfo::argumentGPR0)),
+ root->appendIntConstant(proc, Origin(), type, 42)));
+
+ auto code = compile(proc);
+ T value[2];
+ value[0] = 42;
+ value[1] = 13;
+ CHECK(invoke<bool>(*code, value));
+ CHECK_EQ(value[0], static_cast<T>(0xbeef));
+ CHECK_EQ(value[1], 13);
+ value[0] = static_cast<T>(300);
+ CHECK(!invoke<bool>(*code, value));
+ CHECK_EQ(value[0], static_cast<T>(300));
+ CHECK_EQ(value[1], 13);
+ checkMyDisassembly(*code, true);
+ }
+
+ {
+ Procedure proc;
+ BasicBlock* root = proc.addBlock();
+ root->appendNew<Value>(
+ proc, Return, Origin(),
+ root->appendNew<Value>(
+ proc, Equal, Origin(),
+ root->appendNew<Value>(
+ proc, NotEqual, Origin(),
+ root->appendNew<AtomicValue>(
+ proc, AtomicStrongCAS, Origin(), width,
+ root->appendIntConstant(proc, Origin(), type, 42),
+ root->appendIntConstant(proc, Origin(), type, 0xbeef),
+ root->appendNew<ArgumentRegValue>(proc, Origin(), GPRInfo::argumentGPR0)),
+ root->appendIntConstant(proc, Origin(), type, 42)),
+ root->appendNew<Const32Value>(proc, Origin(), 0)));
+
+ auto code = compile(proc);
+ T value[2];
+ value[0] = 42;
+ value[1] = 13;
+ CHECK(invoke<bool>(*code, value));
+ CHECK_EQ(value[0], static_cast<T>(0xbeef));
+ CHECK_EQ(value[1], 13);
+ value[0] = static_cast<T>(300);
+ CHECK(!invoke<bool>(*code, &value));
+ CHECK_EQ(value[0], static_cast<T>(300));
+ CHECK_EQ(value[1], 13);
+ checkMyDisassembly(*code, true);
+ }
+}
+
+template<typename T>
+void testAtomicXchg(B3::Opcode opcode)
+{
+ Type type = NativeTraits<T>::type;
+ Width width = NativeTraits<T>::width;
+
+ auto doTheMath = [&] (T& memory, T operand) -> T {
+ T oldValue = memory;
+ switch (opcode) {
+ case AtomicXchgAdd:
+ memory += operand;
+ break;
+ case AtomicXchgAnd:
+ memory &= operand;
+ break;
+ case AtomicXchgOr:
+ memory |= operand;
+ break;
+ case AtomicXchgSub:
+ memory -= operand;
+ break;
+ case AtomicXchgXor:
+ memory ^= operand;
+ break;
+ case AtomicXchg:
+ memory = operand;
+ break;
+ default:
+ RELEASE_ASSERT_NOT_REACHED();
+ }
+ return oldValue;
+ };
+
+ auto oldValue = [&] (T memory, T operand) -> T {
+ return doTheMath(memory, operand);
+ };
+
+ auto newValue = [&] (T memory, T operand) -> T {
+ doTheMath(memory, operand);
+ return memory;
+ };
+
+ auto checkMyDisassembly = [&] (Compilation& compilation, bool fenced) {
+ if (isX86())
+ checkUsesInstruction(compilation, "lock");
+ else {
+ if (fenced) {
+ checkUsesInstruction(compilation, "ldax");
+ checkUsesInstruction(compilation, "stlx");
+ } else {
+ checkUsesInstruction(compilation, "ldx");
+ checkUsesInstruction(compilation, "stx");
+ }
+ }
+ };
+
+ {
+ Procedure proc;
+ BasicBlock* root = proc.addBlock();
+ root->appendNew<Value>(
+ proc, Return, Origin(),
+ root->appendNew<AtomicValue>(
+ proc, opcode, Origin(), width,
+ root->appendIntConstant(proc, Origin(), type, 1),
+ root->appendNew<ArgumentRegValue>(proc, Origin(), GPRInfo::argumentGPR0)));
+
+ auto code = compile(proc);
+ T value[2];
+ value[0] = 5;
+ value[1] = 100;
+ CHECK_EQ(invoke<T>(*code, value), oldValue(5, 1));
+ CHECK_EQ(value[0], newValue(5, 1));
+ CHECK_EQ(value[1], 100);
+ checkMyDisassembly(*code, true);
+ }
+
+ {
+ Procedure proc;
+ BasicBlock* root = proc.addBlock();
+ root->appendNew<Value>(
+ proc, Return, Origin(),
+ root->appendNew<AtomicValue>(
+ proc, opcode, Origin(), width,
+ root->appendIntConstant(proc, Origin(), type, 42),
+ root->appendNew<ArgumentRegValue>(proc, Origin(), GPRInfo::argumentGPR0)));
+
+ auto code = compile(proc);
+ T value[2];
+ value[0] = 5;
+ value[1] = 100;
+ CHECK_EQ(invoke<T>(*code, value), oldValue(5, 42));
+ CHECK_EQ(value[0], newValue(5, 42));
+ CHECK_EQ(value[1], 100);
+ checkMyDisassembly(*code, true);
+ }
+
+ {
+ Procedure proc;
+ BasicBlock* root = proc.addBlock();
+ root->appendNew<AtomicValue>(
+ proc, opcode, Origin(), width,
+ root->appendIntConstant(proc, Origin(), type, 42),
+ root->appendNew<ArgumentRegValue>(proc, Origin(), GPRInfo::argumentGPR0));
+ root->appendNew<Value>(proc, Return, Origin());
+
+ auto code = compile(proc);
+ T value[2];
+ value[0] = 5;
+ value[1] = 100;
+ invoke<T>(*code, value);
+ CHECK_EQ(value[0], newValue(5, 42));
+ CHECK_EQ(value[1], 100);
+ checkMyDisassembly(*code, true);
+ }
+
+ {
+ Procedure proc;
+ BasicBlock* root = proc.addBlock();
+ root->appendNew<AtomicValue>(
+ proc, opcode, Origin(), width,
+ root->appendIntConstant(proc, Origin(), type, 42),
+ root->appendNew<ArgumentRegValue>(proc, Origin(), GPRInfo::argumentGPR0),
+ 0, HeapRange(42), HeapRange());
+ root->appendNew<Value>(proc, Return, Origin());
+
+ auto code = compile(proc);
+ T value[2];
+ value[0] = 5;
+ value[1] = 100;
+ invoke<T>(*code, value);
+ CHECK_EQ(value[0], newValue(5, 42));
+ CHECK_EQ(value[1], 100);
+ checkMyDisassembly(*code, false);
+ }
+}
+
+void testDepend32()
+{
+ Procedure proc;
+ BasicBlock* root = proc.addBlock();
+ Value* ptr = root->appendNew<ArgumentRegValue>(proc, Origin(), GPRInfo::argumentGPR0);
+ Value* first = root->appendNew<MemoryValue>(proc, Load, Int32, Origin(), ptr, 0);
+ Value* second = root->appendNew<MemoryValue>(
+ proc, Load, Int32, Origin(),
+ root->appendNew<Value>(
+ proc, Add, Origin(), ptr,
+ root->appendNew<Value>(
+ proc, ZExt32, Origin(),
+ root->appendNew<Value>(proc, Depend, Origin(), first))),
+ 4);
+ root->appendNew<Value>(
+ proc, Return, Origin(),
+ root->appendNew<Value>(proc, Add, Origin(), first, second));
+
+ int32_t values[2];
+ values[0] = 42;
+ values[1] = 0xbeef;
+
+ auto code = compile(proc);
+ if (isARM64())
+ checkUsesInstruction(*code, "eor");
+ else if (isX86()) {
+ checkDoesNotUseInstruction(*code, "mfence");
+ checkDoesNotUseInstruction(*code, "lock");
+ }
+ CHECK_EQ(invoke<int32_t>(*code, values), 42 + 0xbeef);
+}
+
+void testDepend64()
+{
+ Procedure proc;
+ BasicBlock* root = proc.addBlock();
+ Value* ptr = root->appendNew<ArgumentRegValue>(proc, Origin(), GPRInfo::argumentGPR0);
+ Value* first = root->appendNew<MemoryValue>(proc, Load, Int64, Origin(), ptr, 0);
+ Value* second = root->appendNew<MemoryValue>(
+ proc, Load, Int64, Origin(),
+ root->appendNew<Value>(
+ proc, Add, Origin(), ptr,
+ root->appendNew<Value>(proc, Depend, Origin(), first)),
+ 8);
+ root->appendNew<Value>(
+ proc, Return, Origin(),
+ root->appendNew<Value>(proc, Add, Origin(), first, second));
+
+ int64_t values[2];
+ values[0] = 42;
+ values[1] = 0xbeef;
+
+ auto code = compile(proc);
+ if (isARM64())
+ checkUsesInstruction(*code, "eor");
+ else if (isX86()) {
+ checkDoesNotUseInstruction(*code, "mfence");
+ checkDoesNotUseInstruction(*code, "lock");
+ }
+ CHECK_EQ(invoke<int64_t>(*code, values), 42 + 0xbeef);
+}
+
</ins><span class="cx"> void testWasmBoundsCheck(unsigned offset)
</span><span class="cx"> {
</span><span class="cx"> Procedure proc;
</span><span class="lines">@@ -14370,6 +15221,13 @@
</span><span class="cx"> return -zero();
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+#define RUN_NOW(test) do { \
+ if (!shouldRun(#test)) \
+ break; \
+ dataLog(#test "...\n"); \
+ test; \
+ dataLog(#test ": OK!\n"); \
+ } while (false)
</ins><span class="cx"> #define RUN(test) do { \
</span><span class="cx"> if (!shouldRun(#test)) \
</span><span class="cx"> break; \
</span><span class="lines">@@ -14421,12 +15279,11 @@
</span><span class="cx"> return !filter || !!strcasestr(testName, filter);
</span><span class="cx"> };
</span><span class="cx">
</span><del>- // We run this test first because it fiddles with some
- // JSC options.
- testTerminalPatchpointThatNeedsToBeSpilled2();
</del><ins>+ RUN_NOW(testTerminalPatchpointThatNeedsToBeSpilled2());
</ins><span class="cx">
</span><span class="cx"> RUN(test42());
</span><span class="cx"> RUN(testLoad42());
</span><ins>+ RUN(testLoadAcq42());
</ins><span class="cx"> RUN(testLoadOffsetImm9Max());
</span><span class="cx"> RUN(testLoadOffsetImm9MaxPlusOne());
</span><span class="cx"> RUN(testLoadOffsetImm9MaxPlusTwo());
</span><span class="lines">@@ -15099,16 +15956,23 @@
</span><span class="cx"> RUN(testNeg32(52));
</span><span class="cx"> RUN(testNegPtr(53));
</span><span class="cx"> RUN(testStoreAddLoad32(46));
</span><ins>+ RUN(testStoreRelAddLoadAcq32(46));
</ins><span class="cx"> RUN(testStoreAddLoadImm32(46));
</span><span class="cx"> RUN(testStoreAddLoad64(4600));
</span><ins>+ RUN(testStoreRelAddLoadAcq64(4600));
</ins><span class="cx"> RUN(testStoreAddLoadImm64(4600));
</span><span class="cx"> RUN(testStoreAddLoad8(4, Load8Z));
</span><ins>+ RUN(testStoreRelAddLoadAcq8(4, Load8Z));
+ RUN(testStoreRelAddFenceLoadAcq8(4, Load8Z));
</ins><span class="cx"> RUN(testStoreAddLoadImm8(4, Load8Z));
</span><span class="cx"> RUN(testStoreAddLoad8(4, Load8S));
</span><ins>+ RUN(testStoreRelAddLoadAcq8(4, Load8S));
</ins><span class="cx"> RUN(testStoreAddLoadImm8(4, Load8S));
</span><span class="cx"> RUN(testStoreAddLoad16(6, Load16Z));
</span><ins>+ RUN(testStoreRelAddLoadAcq16(6, Load16Z));
</ins><span class="cx"> RUN(testStoreAddLoadImm16(6, Load16Z));
</span><span class="cx"> RUN(testStoreAddLoad16(6, Load16S));
</span><ins>+ RUN(testStoreRelAddLoadAcq16(6, Load16S));
</ins><span class="cx"> RUN(testStoreAddLoadImm16(6, Load16S));
</span><span class="cx"> RUN(testStoreAddLoad32Index(46));
</span><span class="cx"> RUN(testStoreAddLoadImm32Index(46));
</span><span class="lines">@@ -15831,6 +16695,41 @@
</span><span class="cx"> RUN(testLoadBaseIndexShift2());
</span><span class="cx"> RUN(testLoadBaseIndexShift32());
</span><span class="cx"> RUN(testOptimizeMaterialization());
</span><ins>+
+ RUN(testAtomicWeakCAS<int8_t>());
+ RUN(testAtomicWeakCAS<int16_t>());
+ RUN(testAtomicWeakCAS<int32_t>());
+ RUN(testAtomicWeakCAS<int64_t>());
+ RUN(testAtomicStrongCAS<int8_t>());
+ RUN(testAtomicStrongCAS<int16_t>());
+ RUN(testAtomicStrongCAS<int32_t>());
+ RUN(testAtomicStrongCAS<int64_t>());
+ RUN(testAtomicXchg<int8_t>(AtomicXchgAdd));
+ RUN(testAtomicXchg<int16_t>(AtomicXchgAdd));
+ RUN(testAtomicXchg<int32_t>(AtomicXchgAdd));
+ RUN(testAtomicXchg<int64_t>(AtomicXchgAdd));
+ RUN(testAtomicXchg<int8_t>(AtomicXchgAnd));
+ RUN(testAtomicXchg<int16_t>(AtomicXchgAnd));
+ RUN(testAtomicXchg<int32_t>(AtomicXchgAnd));
+ RUN(testAtomicXchg<int64_t>(AtomicXchgAnd));
+ RUN(testAtomicXchg<int8_t>(AtomicXchgOr));
+ RUN(testAtomicXchg<int16_t>(AtomicXchgOr));
+ RUN(testAtomicXchg<int32_t>(AtomicXchgOr));
+ RUN(testAtomicXchg<int64_t>(AtomicXchgOr));
+ RUN(testAtomicXchg<int8_t>(AtomicXchgSub));
+ RUN(testAtomicXchg<int16_t>(AtomicXchgSub));
+ RUN(testAtomicXchg<int32_t>(AtomicXchgSub));
+ RUN(testAtomicXchg<int64_t>(AtomicXchgSub));
+ RUN(testAtomicXchg<int8_t>(AtomicXchgXor));
+ RUN(testAtomicXchg<int16_t>(AtomicXchgXor));
+ RUN(testAtomicXchg<int32_t>(AtomicXchgXor));
+ RUN(testAtomicXchg<int64_t>(AtomicXchgXor));
+ RUN(testAtomicXchg<int8_t>(AtomicXchg));
+ RUN(testAtomicXchg<int16_t>(AtomicXchg));
+ RUN(testAtomicXchg<int32_t>(AtomicXchg));
+ RUN(testAtomicXchg<int64_t>(AtomicXchg));
+ RUN(testDepend32());
+ RUN(testDepend64());
</ins><span class="cx">
</span><span class="cx"> RUN(testWasmBoundsCheck(0));
</span><span class="cx"> RUN(testWasmBoundsCheck(100));
</span></span></pre></div>
<a id="trunkSourceJavaScriptCoreruntimeOptionsh"></a>
<div class="modfile"><h4>Modified: trunk/Source/JavaScriptCore/runtime/Options.h (213713 => 213714)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/JavaScriptCore/runtime/Options.h 2017-03-10 17:32:07 UTC (rev 213713)
+++ trunk/Source/JavaScriptCore/runtime/Options.h 2017-03-10 17:49:42 UTC (rev 213714)
</span><span class="lines">@@ -416,9 +416,6 @@
</span><span class="cx"> v(bool, dumpModuleLoadingState, false, Normal, nullptr) \
</span><span class="cx"> v(bool, exposeInternalModuleLoader, false, Normal, "expose the internal module loader object to the global space for debugging") \
</span><span class="cx"> \
</span><del>- v(bool, dumpAirAsJSBeforeAllocateStack, false, Normal, nullptr) \
- v(bool, dumpAirAfterAllocateStack, false, Normal, nullptr) \
- \
</del><span class="cx"> v(bool, useSuperSampler, false, Normal, nullptr) \
</span><span class="cx"> \
</span><span class="cx"> v(bool, reportLLIntStats, false, Configurable, "Reports LLInt statistics") \
</span></span></pre></div>
<a id="trunkWebsiteswebkitorgChangeLog"></a>
<div class="modfile"><h4>Modified: trunk/Websites/webkit.org/ChangeLog (213713 => 213714)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Websites/webkit.org/ChangeLog 2017-03-10 17:32:07 UTC (rev 213713)
+++ trunk/Websites/webkit.org/ChangeLog 2017-03-10 17:49:42 UTC (rev 213714)
</span><span class="lines">@@ -1,3 +1,14 @@
</span><ins>+2017-03-04 Filip Pizlo <fpizlo@apple.com>
+
+ B3 should have comprehensive support for atomic operations
+ https://bugs.webkit.org/show_bug.cgi?id=162349
+
+ Reviewed by Keith Miller.
+
+ Document the new opcodes!
+
+ * docs/b3/intermediate-representation.html:
+
</ins><span class="cx"> 2017-03-09 Jon Davis <jond@apple.com>
</span><span class="cx">
</span><span class="cx"> Prevent HTML in rich link previews
</span></span></pre></div>
<a id="trunkWebsiteswebkitorgdocsb3intermediaterepresentationhtml"></a>
<div class="modfile"><h4>Modified: trunk/Websites/webkit.org/docs/b3/intermediate-representation.html (213713 => 213714)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Websites/webkit.org/docs/b3/intermediate-representation.html 2017-03-10 17:32:07 UTC (rev 213713)
+++ trunk/Websites/webkit.org/docs/b3/intermediate-representation.html 2017-03-10 17:49:42 UTC (rev 213714)
</span><span class="lines">@@ -148,6 +148,28 @@
</span><span class="cx">
</span><span class="cx"> <p>Each basic block contains a Vector&lt;Value*&gt; as the contents of the block. Control
</span><span class="cx"> flow inside the block is implicit based on the order of Values in the vector.</p>
</span><ins>+
+ <h2>Memory model</h2>
+
+ <p>B3 uses a secondary address space of <i>abstract heaps</i> to reason about aliasing and
+ concurrency. This address space is 32-bit and we point at it using the HeapRange type. If you
+ never supply HeapRanges to your memory operations (the default), they will have [0, UINT_MAX]
+ as their range, indicating that the operation may conservatively write to all 2<sup>32</sup>
+ abstract heaps. Two memory operations are said to alias if they access the same abstract heaps.
+ A simple example of using HeapRanges is with loads and stores, but you can also supply
+ HeapRanges as bounds on the effects of calls and patchpoints.</p>
+
+ <p>Fences are modeled as phantom effects that we call <i>fence ranges</i>. For example, a load
+ fence could be represented as a patchpoint that writes to [0, UINT_MAX]. B3 models load fences,
+ store fences, store-load fences, acquire fences, and release fences using fence ranges. The
+ absence of a fence range means that there is no fence.</p>
+
+ <p>Acquire fences are modeled as phantom writes to memory, while release fences are modeled as
+ phantom reads from memory. By itself this does not permit B3 to do all of the reorderings that
+ acquire/release allow. B3 is therefore allowed to treat loads/stores with fence ranges more
+ precisely. The phantom write effects of a fenced load cannot write to anything that is read or
+ written after the fence. The phantom read effects of a fenced store cannot read anything that
+ is written after the fence.</p>
</ins><span class="cx">
</span><span class="cx"> <h2>Opcodes</h2>
</span><span class="cx">
</span><span class="lines">@@ -395,51 +417,126 @@
</span><span class="cx"> <dt>Int32 Load8Z(IntPtr, offset)</dt>
</span><span class="cx"> <dd>Loads a byte from the address, which is computed by adding the compile-time 32-bit
</span><span class="cx"> signed integer offset to the child value. Zero extends the loaded byte, so the high 24
</span><del>- bits are all zero. Must use the MemoryValue class. May have the Traps flag set.</dd>
</del><ins>+ bits are all zero. Must use the MemoryValue class. May have the Traps flag set. May set
+ a fence range to turn this into a load acquire.</dd>
</ins><span class="cx">
</span><span class="cx"> <dt>Int32 Load8S(IntPtr, offset)</dt>
</span><span class="cx"> <dd>Loads a byte from the address, which is computed by adding the compile-time 32-bit
</span><span class="cx"> signed integer offset to the child value. Sign extends the loaded byte. Must use the
</span><del>- MemoryValue class. May have the Traps flag set.</dd>
</del><ins>+ MemoryValue class. May have the Traps flag set. May set
+ a fence range to turn this into a load acquire.</dd>
</ins><span class="cx">
</span><span class="cx"> <dt>Int32 Load16Z(IntPtr, offset)</dt>
</span><span class="cx"> <dd>Loads a 16-bit integer from the address, which is computed by adding the compile-time
</span><span class="cx"> 32-bit signed integer offset to the child value. Zero extends the loaded 16-bit
</span><span class="cx"> integer, so the high 16 bits are all zero. Misaligned loads are not penalized. Must
</span><del>- use the MemoryValue class. May have the Traps flag set.</dd>
</del><ins>+ use the MemoryValue class. May have the Traps flag set. May set
+ a fence range to turn this into a load acquire.</dd>
</ins><span class="cx">
</span><span class="cx"> <dt>Int32 Load16S(IntPtr, offset)</dt>
</span><span class="cx"> <dd>Loads a 16-bit integer from the address, which is computed by adding the compile-time
</span><span class="cx"> 32-bit signed integer offset to the child value. Sign extends the loaded 16-bit
</span><span class="cx"> integer. Misaligned loads are not penalized. Must use the MemoryValue class. May have the
</span><del>- Traps flag set.</dd>
</del><ins>+ Traps flag set. May set a fence range to turn this into a load acquire.</dd>
</ins><span class="cx">
</span><span class="cx"> <dt>T Load(IntPtr, offset)</dt>
</span><span class="cx"> <dd>Valid for any type except Void. Loads a value of that type from the address, which is
</span><span class="cx"> computed by adding the compile-time 32-bit signed integer offset to the child value.
</span><span class="cx"> Misaligned loads are not penalized. Must use the MemoryValue class. May have the Traps
</span><del>- flag set.</dd>
</del><ins>+ flag set. May set a fence range to turn this into a load acquire.</dd>
</ins><span class="cx">
</span><span class="cx"> <dt>Void Store8(Int32, IntPtr, offset)</dt>
</span><span class="cx"> <dd>Stores a the low byte of the first child into the address computed by adding the
</span><span class="cx"> compile-time 32-bit signed integer offset to the second child. Must use the MemoryValue
</span><del>- class. May have the Traps flag set.</dd>
</del><ins>+ class. May have the Traps flag set. May set a fence range to turn this into a store release.</dd>
</ins><span class="cx">
</span><span class="cx"> <dt>Void Store16(Int32, IntPtr, offset)</dt>
</span><span class="cx"> <dd>Stores a the low 16 bits of the first child into the address computed by adding the
</span><span class="cx"> compile-time 32-bit signed integer offset to the second child. Misaligned stores are
</span><del>- not penalized. Must use the MemoryValue class. May have the Traps flag set.</dd>
</del><ins>+ not penalized. Must use the MemoryValue class. May have the Traps flag set. May set a
+ fence range to turn this into a store release.</dd>
</ins><span class="cx">
</span><span class="cx"> <dt>Void Store(T, IntPtr, offset)</dt>
</span><span class="cx"> <dd>Stores the value in the first child into the address computed by adding the
</span><span class="cx"> compile-time 32-bit signed integer offset to the second child. Misaligned stores are
</span><del>- not penalized. Must use the MemoryValue class. May have the Traps flag set.</dd>
</del><ins>+ not penalized. Must use the MemoryValue class. May have the Traps flag set. May set a
+ fence range to turn this into a store release.</dd>
+
+ <dt>Int32 AtomicWeakCAS(T expectedValue, T newValue, IntPtr ptr)</dt>
+ <dd>Performs a weak <a href="https://en.wikipedia.org/wiki/Compare-and-swap">CAS
+ (compare-and-swap)</a>. Returns a boolean (0 or 1) to indicate the result (failure or
+ success, respectively). May spuriously fail, in which case it will do nothing and return
+ 0. You can supply a fence range to indicate that the CAS is fenced. Fenced CAS has both
+ acquire and release fences, and claims to both read and write the full fence range in
+ addition to whatever primary range is supplied for the CAS itself. AtomicWeakCAS only
+ has fencing behavior in the case that it succeeds. Atomic operations take a width parameter,
+ allowing them to operate over 8-bit, 16-bit, 32-bit, or 64-bit integer memory locations.</dd>
</ins><span class="cx">
</span><ins>+ <dt>T AtomicStrongCAS(T expectedValue, T newValue, IntPtr ptr)</dt>
+ <dd>Performs a strong <a href="https://en.wikipedia.org/wiki/Compare-and-swap">CAS
+ (compare-and-swap)</a>. Returns the old value before the CAS. The instruction selector
+ is smart about Equal(AtomicStrongCAS(expected, ...), expected) patterns, so this opcode
+ is also the best way to do a strong CAS that returns a boolean - just use Equal to compare
+ the result to expected. You can supply a fence range to indicate that the CAS is fenced.
+ Fenced CAS has both acquire and release fences, and claims to both read and write the full
+ fence range in addition to whatever primary range is supplied for the CAS itself.
+ AtomicStrongCAS has the same fencing on failure and success. Atomic operations take a width
+ parameter, allowing them to operate over 8-bit, 16-bit, 32-bit, or 64-bit integer memory
+ locations. Returns a sign-extended result for 8-bit or 16-bit widths.</dd>
+
+ <dt>T AtomicXchgAdd(T, IntPtr)</dt>
+ <dd>Atomically adds a value to the memory location and returns the old value. Is allowed to
+ have a fence range, which causes it to have acquire/release fencing. Atomic operations take
+ a width parameter, allowing them to operate over 8-bit, 16-bit, 32-bit, or 64-bit integer memory
+ locations. Returns a sign-extended result for 8-bit or 16-bit widths.</dd>
+
+ <dt>T AtomicXchgAnd(T, IntPtr)</dt>
+ <dd>Atomically bitwise-ands a value to the memory location and returns the old value. Is allowed to
+ have a fence range, which causes it to have acquire/release fencing. Atomic operations take
+ a width parameter, allowing them to operate over 8-bit, 16-bit, 32-bit, or 64-bit integer memory
+ locations. Returns a sign-extended result for 8-bit or 16-bit widths.</dd>
+
+ <dt>T AtomicXchgOr(T, IntPtr)</dt>
+ <dd>Atomically bitwise-ors a value to the memory location and returns the old value. Is allowed to
+ have a fence range, which causes it to have acquire/release fencing. Atomic operations take
+ a width parameter, allowing them to operate over 8-bit, 16-bit, 32-bit, or 64-bit integer memory
+ locations. Returns a sign-extended result for 8-bit or 16-bit widths.</dd>
+
+ <dt>T AtomicXchgSub(T, IntPtr)</dt>
+ <dd>Atomically subtracts a value to the memory location and returns the old value. Is allowed to
+ have a fence range, which causes it to have acquire/release fencing. Atomic operations take
+ a width parameter, allowing them to operate over 8-bit, 16-bit, 32-bit, or 64-bit integer memory
+ locations. Returns a sign-extended result for 8-bit or 16-bit widths.</dd>
+
+ <dt>T AtomicXchgXor(T, IntPtr)</dt>
+ <dd>Atomically bitwise-xors a value to the memory location and returns the old value. Is allowed to
+ have a fence range, which causes it to have acquire/release fencing. Atomic operations take
+ a width parameter, allowing them to operate over 8-bit, 16-bit, 32-bit, or 64-bit integer memory
+ locations. Returns a sign-extended result for 8-bit or 16-bit widths.</dd>
+
+ <dt>T AtomicXchg(T, IntPtr)</dt>
+ <dd>Atomically stores a value to the memory location and returns the old value. Is allowed to
+ have a fence range, which causes it to have acquire/release fencing. Atomic operations take
+ a width parameter, allowing them to operate over 8-bit, 16-bit, 32-bit, or 64-bit integer memory
+ locations. Returns a sign-extended result for 8-bit or 16-bit widths.</dd>
+
+ <dt>T Depend(T)</dt>
+ <dd>Only valid for integer types. This is guaranteed to return zero by xoring the input with
+ itself, and the B3 compiler guarantees that it will not leverage this knowledge for any
+ optimization. On x86, this is lowered to a Fence with read=Bottom, write=Top (i.e. a load-load
+ fence) and then it's constant-folded to zero. On ARM, this is preserved through the compiler
+ pipeline all the way to the MacroAssembler, which turns this into a <code>eor</code>. Using
+ Depend is the most efficient way to create load-load dependencies, regarldess of platform. It
+ allows B3's CSE to work for surrounding loads and it even supports CSEing the dependency
+ itself - so two identical load chains with no interleaving effects can be turned into one. On
+ ARM, it also avoids emitting an expensive <code>dmb ish</code> load-load fence. Because of the
+ benefits for CSE, it's recommended to use Depend for load-load dependencies even if you're only
+ targeting x86.</dd>
+
</ins><span class="cx"> <dt>IntPtr WasmAddress(IntPtr, pinnedGPR)</dt>
</span><span class="cx"> <dd>This is used to compute the address of a wasm memory load from a pinned base GPR.
</span><span class="cx"> Must use the WasmAddressValue class.</dd>
</span><span class="cx">
</span><del>-
</del><span class="cx"> <dt>Void Fence()</dt>
</span><span class="cx"> <dd>Abstracts standalone data fences on x86 and ARM. Must use the FenceValue class, which has
</span><span class="cx"> two additional members that configure the precise meaning of the fence:
</span></span></pre>
</div>
</div>
</body>
</html>