[webkit-changes] [WebKit/WebKit] 41169e: Colocate ValueProfiles

[Keith Miller]

  Branch: refs/heads/main
  Home:   https://github.com/WebKit/WebKit
  Commit: 41169e7edae194d93e7c21bbd76b285d6128fa55
      https://github.com/WebKit/WebKit/commit/41169e7edae194d93e7c21bbd76b285d6128fa55
  Author: Keith Miller <keith_miller at apple.com>
  Date:   2023-09-05 (Tue, 05 Sep 2023)

  Changed paths:
    M JSTests/wasm/stress/simple-inline-stacktrace.js
    M LayoutTests/js/script-tests/regress-139548.js
    M Source/JavaScriptCore/bytecode/BytecodeGeneratorification.cpp
    M Source/JavaScriptCore/bytecode/BytecodeList.rb
    M Source/JavaScriptCore/bytecode/BytecodeOperandsForCheckpoint.h
    M Source/JavaScriptCore/bytecode/CodeBlock.cpp
    M Source/JavaScriptCore/bytecode/CodeBlock.h
    M Source/JavaScriptCore/bytecode/CodeBlockInlines.h
    M Source/JavaScriptCore/bytecode/MetadataTable.cpp
    M Source/JavaScriptCore/bytecode/MetadataTable.h
    M Source/JavaScriptCore/bytecode/UnlinkedCodeBlock.cpp
    M Source/JavaScriptCore/bytecode/UnlinkedCodeBlock.h
    M Source/JavaScriptCore/bytecode/UnlinkedCodeBlockGenerator.cpp
    M Source/JavaScriptCore/bytecode/UnlinkedMetadataTable.cpp
    M Source/JavaScriptCore/bytecode/UnlinkedMetadataTable.h
    M Source/JavaScriptCore/bytecode/UnlinkedMetadataTableInlines.h
    M Source/JavaScriptCore/bytecompiler/BytecodeGenerator.cpp
    M Source/JavaScriptCore/bytecompiler/BytecodeGenerator.h
    M Source/JavaScriptCore/jit/JITInlines.h
    M Source/JavaScriptCore/jit/JITPropertyAccess.cpp
    M Source/JavaScriptCore/llint/LLIntSlowPaths.cpp
    M Source/JavaScriptCore/llint/LowLevelInterpreter.asm
    M Source/JavaScriptCore/llint/LowLevelInterpreter32_64.asm
    M Source/JavaScriptCore/llint/LowLevelInterpreter64.asm
    M Source/JavaScriptCore/runtime/CachedTypes.cpp
    M Source/JavaScriptCore/runtime/CommonSlowPaths.cpp

  Log Message:
  -----------
  Colocate ValueProfiles
https://bugs.webkit.org/show_bug.cgi?id=261151
rdar://114387937

Reviewed by Yusuke Suzuki.

We spend a lot of time in updateAllNonLazyValueProfilePredictions during our
benchmark runs. In order to improve the throughput of that function we should
move all ValueProfiles into their own buffer. This improves the memory
locality of ValueProfiles. From my testing this is ~.5% progression on
Speedometer2/3 and JetStream2.

In order to not need a new register for ValueProfiles in the LLInt/Baseline we
"butterfly" the MetadataTable so that negative indicies point to ValueProfiles
and positive indicies point to the old MetadataTable contents.

* Source/JavaScriptCore/bytecode/BytecodeGeneratorification.cpp:
(JSC::BytecodeGeneratorification::run):
* Source/JavaScriptCore/bytecode/BytecodeList.rb:
* Source/JavaScriptCore/bytecode/BytecodeOperandsForCheckpoint.h:
(JSC::valueProfileOffsetFor):
(JSC::valueProfileForImpl): Deleted.
(JSC::hasValueProfileFor): Deleted.
(JSC::valueProfileFor): Deleted.
* Source/JavaScriptCore/bytecode/CodeBlock.cpp:
(JSC::CodeBlock::finishCreation):
(JSC::CodeBlock::estimatedSize):
(JSC::CodeBlock::visitChildren):
(JSC::CodeBlock::tryGetValueProfileForBytecodeIndex):
* Source/JavaScriptCore/bytecode/CodeBlock.h:
(JSC::CodeBlock::valueProfileForOffset):
* Source/JavaScriptCore/bytecode/CodeBlockInlines.h:
(JSC::CodeBlock::forEachValueProfile):
* Source/JavaScriptCore/bytecode/MetadataTable.cpp:
(JSC::MetadataTable::sizeInBytesForGC):
(JSC::MetadataTable::sizeInBytes): Deleted.
* Source/JavaScriptCore/bytecode/MetadataTable.h:
(JSC::MetadataTable::forEachValueProfile):
(JSC::MetadataTable::valueProfilesEnd):
(JSC::MetadataTable::valueProfileForOffset):
(JSC::MetadataTable::totalSize const):
(JSC::MetadataTable::buffer):
* Source/JavaScriptCore/bytecode/UnlinkedCodeBlock.cpp:
(JSC::UnlinkedCodeBlock::visitChildrenImpl):
(JSC::UnlinkedCodeBlock::estimatedSize):
(JSC::UnlinkedCodeBlock::allocateSharedProfiles):
* Source/JavaScriptCore/bytecode/UnlinkedCodeBlock.h:
(JSC::UnlinkedCodeBlock::metadataSizeInBytes):
* Source/JavaScriptCore/bytecode/UnlinkedCodeBlockGenerator.cpp:
(JSC::UnlinkedCodeBlockGenerator::finalize):
* Source/JavaScriptCore/bytecode/UnlinkedMetadataTable.cpp:
(JSC::UnlinkedMetadataTable::finalize):
* Source/JavaScriptCore/bytecode/UnlinkedMetadataTable.h:
(JSC::UnlinkedMetadataTable::numValueProfiles const):
(JSC::UnlinkedMetadataTable::create):
(JSC::UnlinkedMetadataTable::totalSize const):
(JSC::UnlinkedMetadataTable::buffer const):
(JSC::UnlinkedMetadataTable::preprocessBuffer const):
(JSC::UnlinkedMetadataTable::offsetTable16 const):
(JSC::UnlinkedMetadataTable::offsetTable32 const):
* Source/JavaScriptCore/bytecode/UnlinkedMetadataTableInlines.h:
(JSC::UnlinkedMetadataTable::UnlinkedMetadataTable):
(JSC::UnlinkedMetadataTable::~UnlinkedMetadataTable):
(JSC::UnlinkedMetadataTable::addValueProfile):
(JSC::UnlinkedMetadataTable::sizeInBytesForGC):
(JSC::UnlinkedMetadataTable::link):
(JSC::UnlinkedMetadataTable::unlink):
(JSC::UnlinkedMetadataTable::sizeInBytes): Deleted.
* Source/JavaScriptCore/bytecompiler/BytecodeGenerator.cpp:
(JSC::BytecodeGenerator::emitToObject):
(JSC::BytecodeGenerator::emitGetFromScope):
(JSC::BytecodeGenerator::emitTryGetById):
(JSC::BytecodeGenerator::emitGetById):
(JSC::BytecodeGenerator::emitDirectGetById):
(JSC::BytecodeGenerator::emitGetByVal):
(JSC::BytecodeGenerator::emitGetPrototypeOf):
(JSC::BytecodeGenerator::emitGetPrivateName):
(JSC::BytecodeGenerator::emitGetInternalField):
(JSC::BytecodeGenerator::emitGetPrivateBrand):
(JSC::BytecodeGenerator::emitGetArgument):
(JSC::BytecodeGenerator::emitNewArrayWithSpecies):
(JSC::BytecodeGenerator::emitCall):
(JSC::BytecodeGenerator::emitCallVarargs):
(JSC::BytecodeGenerator::emitConstruct):
(JSC::BytecodeGenerator::emitIteratorOpen):
(JSC::BytecodeGenerator::emitIteratorNext):
(JSC::rewriteOp):
(JSC::BytecodeGenerator::emitToThis):
* Source/JavaScriptCore/bytecompiler/BytecodeGenerator.h:
(JSC::BytecodeGenerator::nextValueProfileIndex):
* Source/JavaScriptCore/jit/JITInlines.h:
(JSC::JIT::emitValueProfilingSite):
* Source/JavaScriptCore/jit/JITPropertyAccess.cpp:
(JSC::JIT::emit_op_get_from_scope):
(JSC::JIT::generateOpGetFromScopeThunk):
(JSC::JIT::slow_op_get_from_scopeGenerator):
* Source/JavaScriptCore/llint/LLIntSlowPaths.cpp:
(JSC::LLInt::LLINT_SLOW_PATH_DECL):
* Source/JavaScriptCore/llint/LowLevelInterpreter.asm:
* Source/JavaScriptCore/runtime/CachedTypes.cpp:
(JSC::CachedMetadataTable::encode):
(JSC::CachedMetadataTable::decode const):
* Source/JavaScriptCore/runtime/CommonSlowPaths.cpp:
(JSC::iteratorOpenTryFastImpl):
(JSC::iteratorNextTryFastImpl):

Canonical link: https://commits.webkit.org/267656@main