<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
"http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head><meta http-equiv="content-type" content="text/html; charset=utf-8" />
<title>[199453] releases/WebKitGTK/webkit-2.12/Source/bmalloc</title>
</head>
<body>
<style type="text/css"><!--
#msg dl.meta { border: 1px #006 solid; background: #369; padding: 6px; color: #fff; }
#msg dl.meta dt { float: left; width: 6em; font-weight: bold; }
#msg dt:after { content:':';}
#msg dl, #msg dt, #msg ul, #msg li, #header, #footer, #logmsg { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt; }
#msg dl a { font-weight: bold}
#msg dl a:link { color:#fc3; }
#msg dl a:active { color:#ff0; }
#msg dl a:visited { color:#cc6; }
h3 { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt; font-weight: bold; }
#msg pre { overflow: auto; background: #ffc; border: 1px #fa0 solid; padding: 6px; }
#logmsg { background: #ffc; border: 1px #fa0 solid; padding: 1em 1em 0 1em; }
#logmsg p, #logmsg pre, #logmsg blockquote { margin: 0 0 1em 0; }
#logmsg p, #logmsg li, #logmsg dt, #logmsg dd { line-height: 14pt; }
#logmsg h1, #logmsg h2, #logmsg h3, #logmsg h4, #logmsg h5, #logmsg h6 { margin: .5em 0; }
#logmsg h1:first-child, #logmsg h2:first-child, #logmsg h3:first-child, #logmsg h4:first-child, #logmsg h5:first-child, #logmsg h6:first-child { margin-top: 0; }
#logmsg ul, #logmsg ol { padding: 0; list-style-position: inside; margin: 0 0 0 1em; }
#logmsg ul { text-indent: -1em; padding-left: 1em; }#logmsg ol { text-indent: -1.5em; padding-left: 1.5em; }
#logmsg > ul, #logmsg > ol { margin: 0 0 1em 0; }
#logmsg pre { background: #eee; padding: 1em; }
#logmsg blockquote { border: 1px solid #fa0; border-left-width: 10px; padding: 1em 1em 0 1em; background: white;}
#logmsg dl { margin: 0; }
#logmsg dt { font-weight: bold; }
#logmsg dd { margin: 0; padding: 0 0 0.5em 0; }
#logmsg dd:before { content:'\00bb';}
#logmsg table { border-spacing: 0px; border-collapse: collapse; border-top: 4px solid #fa0; border-bottom: 1px solid #fa0; background: #fff; }
#logmsg table th { text-align: left; font-weight: normal; padding: 0.2em 0.5em; border-top: 1px dotted #fa0; }
#logmsg table td { text-align: right; border-top: 1px dotted #fa0; padding: 0.2em 0.5em; }
#logmsg table thead th { text-align: center; border-bottom: 1px solid #fa0; }
#logmsg table th.Corner { text-align: left; }
#logmsg hr { border: none 0; border-top: 2px dashed #fa0; height: 1px; }
#header, #footer { color: #fff; background: #636; border: 1px #300 solid; padding: 6px; }
#patch { width: 100%; }
#patch h4 {font-family: verdana,arial,helvetica,sans-serif;font-size:10pt;padding:8px;background:#369;color:#fff;margin:0;}
#patch .propset h4, #patch .binary h4 {margin:0;}
#patch pre {padding:0;line-height:1.2em;margin:0;}
#patch .diff {width:100%;background:#eee;padding: 0 0 10px 0;overflow:auto;}
#patch .propset .diff, #patch .binary .diff {padding:10px 0;}
#patch span {display:block;padding:0 10px;}
#patch .modfile, #patch .addfile, #patch .delfile, #patch .propset, #patch .binary, #patch .copfile {border:1px solid #ccc;margin:10px 0;}
#patch ins {background:#dfd;text-decoration:none;display:block;padding:0 10px;}
#patch del {background:#fdd;text-decoration:none;display:block;padding:0 10px;}
#patch .lines, .info {color:#888;background:#fff;}
--></style>
<div id="msg">
<dl class="meta">
<dt>Revision</dt> <dd><a href="http://trac.webkit.org/projects/webkit/changeset/199453">199453</a></dd>
<dt>Author</dt> <dd>carlosgc@webkit.org</dd>
<dt>Date</dt> <dd>2016-04-13 02:39:26 -0700 (Wed, 13 Apr 2016)</dd>
</dl>
<h3>Log Message</h3>
<pre>Merge <a href="http://trac.webkit.org/projects/webkit/changeset/199115">r199115</a> - bmalloc: handle aligned allocations on the fast path
https://bugs.webkit.org/show_bug.cgi?id=156302
Reviewed by Michael Saboff.
This helps keep the JavaScriptCore GC on the fast path, and it also
helps avoid fragmentation on our website stress test:
nimlang 209,584kB 198,076kB ^ 1.06x smaller
* bmalloc/Allocator.cpp:
(bmalloc::Allocator::allocate): Because we arrange for power-of-two size
classes to allocate at power-of-two alignments, we can allocate any
small aligned request on the small path.
* bmalloc/Chunk.h:
(bmalloc::Chunk::bytes):
(bmalloc::Chunk::lines):
(bmalloc::Chunk::pages):
(bmalloc::Chunk::boundaryTags):
(bmalloc::Chunk::objectType): Moved some code around to provide better
API.
(bmalloc::Chunk::Chunk): Moved this code to VMHeap.
(bmalloc::Chunk::offset):
(bmalloc::Chunk::object): Use our new bytes() helper function.
* bmalloc/VMHeap.cpp:
(bmalloc::VMHeap::allocateChunk): Moved code here from Chunk.
(bmalloc::VMHeap::allocateSmallChunk): Ensure that power-of-two page
sizes always begin allocation at the same alignment. Power-of-two object
sizes always request power-of-two page sizes (since that's the least
wasteful option), so if we also ensure that power-of-two page sizes get
power-of-two alignment, then everything is aligned for all small objects.</pre>
<h3>Modified Paths</h3>
<ul>
<li><a href="#releasesWebKitGTKwebkit212SourcebmallocChangeLog">releases/WebKitGTK/webkit-2.12/Source/bmalloc/ChangeLog</a></li>
<li><a href="#releasesWebKitGTKwebkit212SourcebmallocbmallocAllocatorcpp">releases/WebKitGTK/webkit-2.12/Source/bmalloc/bmalloc/Allocator.cpp</a></li>
<li><a href="#releasesWebKitGTKwebkit212SourcebmallocbmallocChunkh">releases/WebKitGTK/webkit-2.12/Source/bmalloc/bmalloc/Chunk.h</a></li>
<li><a href="#releasesWebKitGTKwebkit212SourcebmallocbmallocVMHeapcpp">releases/WebKitGTK/webkit-2.12/Source/bmalloc/bmalloc/VMHeap.cpp</a></li>
</ul>
</div>
<div id="patch">
<h3>Diff</h3>
<a id="releasesWebKitGTKwebkit212SourcebmallocChangeLog"></a>
<div class="modfile"><h4>Modified: releases/WebKitGTK/webkit-2.12/Source/bmalloc/ChangeLog (199452 => 199453)</h4>
<pre class="diff"><span>
<span class="info">--- releases/WebKitGTK/webkit-2.12/Source/bmalloc/ChangeLog 2016-04-13 09:39:10 UTC (rev 199452)
+++ releases/WebKitGTK/webkit-2.12/Source/bmalloc/ChangeLog 2016-04-13 09:39:26 UTC (rev 199453)
</span><span class="lines">@@ -1,3 +1,42 @@
</span><ins>+2016-04-06 Geoffrey Garen <ggaren@apple.com>
+
+ bmalloc: handle aligned allocations on the fast path
+ https://bugs.webkit.org/show_bug.cgi?id=156302
+
+ Reviewed by Michael Saboff.
+
+ This helps keep the JavaScriptCore GC on the fast path, and it also
+ helps avoid fragmentation on our website stress test:
+
+ nimlang 209,584kB 198,076kB ^ 1.06x smaller
+
+ * bmalloc/Allocator.cpp:
+ (bmalloc::Allocator::allocate): Because we arrange for power-of-two size
+ classes to allocate at power-of-two alignments, we can allocate any
+ small aligned request on the small path.
+
+ * bmalloc/Chunk.h:
+ (bmalloc::Chunk::bytes):
+ (bmalloc::Chunk::lines):
+ (bmalloc::Chunk::pages):
+ (bmalloc::Chunk::boundaryTags):
+ (bmalloc::Chunk::objectType): Moved some code around to provide better
+ API.
+
+ (bmalloc::Chunk::Chunk): Moved this code to VMHeap.
+
+ (bmalloc::Chunk::offset):
+ (bmalloc::Chunk::object): Use our new bytes() helper function.
+
+ * bmalloc/VMHeap.cpp:
+ (bmalloc::VMHeap::allocateChunk): Moved code here from Chunk.
+
+ (bmalloc::VMHeap::allocateSmallChunk): Ensure that power-of-two page
+ sizes always begin allocation at the same alignment. Power-of-two object
+ sizes always request power-of-two page sizes (since that's the least
+ wasteful option), so if we also ensure that power-of-two page sizes get
+ power-of-two alignment, then everything is aligned for all small objects.
+
</ins><span class="cx"> 2016-04-03 Geoffrey Garen <ggaren@apple.com>
</span><span class="cx">
</span><span class="cx"> bmalloc: segregate small and large objects again, and allocate more objects on the small path
</span></span></pre></div>
<a id="releasesWebKitGTKwebkit212SourcebmallocbmallocAllocatorcpp"></a>
<div class="modfile"><h4>Modified: releases/WebKitGTK/webkit-2.12/Source/bmalloc/bmalloc/Allocator.cpp (199452 => 199453)</h4>
<pre class="diff"><span>
<span class="info">--- releases/WebKitGTK/webkit-2.12/Source/bmalloc/bmalloc/Allocator.cpp 2016-04-13 09:39:10 UTC (rev 199452)
+++ releases/WebKitGTK/webkit-2.12/Source/bmalloc/bmalloc/Allocator.cpp 2016-04-13 09:39:26 UTC (rev 199453)
</span><span class="lines">@@ -77,16 +77,13 @@
</span><span class="cx"> return nullptr;
</span><span class="cx"> return result;
</span><span class="cx"> }
</span><del>-
- if (size <= smallMax && alignment <= smallLineSize) {
- size_t alignmentMask = alignment - 1;
- while (void* p = allocate(size)) {
- if (!test(p, alignmentMask))
- return p;
- m_deallocator.deallocate(p);
- }
- }
</del><span class="cx">
</span><ins>+ if (!size)
+ size = alignment;
+
+ if (size <= smallMax && alignment <= smallMax)
+ return allocate(roundUpToMultipleOf(alignment, size));
+
</ins><span class="cx"> if (size <= largeMax && alignment <= largeMax) {
</span><span class="cx"> size = std::max(largeMin, roundUpToMultipleOf<largeAlignment>(size));
</span><span class="cx"> alignment = roundUpToMultipleOf<largeAlignment>(alignment);
</span></span></pre></div>
<a id="releasesWebKitGTKwebkit212SourcebmallocbmallocChunkh"></a>
<div class="modfile"><h4>Modified: releases/WebKitGTK/webkit-2.12/Source/bmalloc/bmalloc/Chunk.h (199452 => 199453)</h4>
<pre class="diff"><span>
<span class="info">--- releases/WebKitGTK/webkit-2.12/Source/bmalloc/bmalloc/Chunk.h 2016-04-13 09:39:10 UTC (rev 199452)
+++ releases/WebKitGTK/webkit-2.12/Source/bmalloc/bmalloc/Chunk.h 2016-04-13 09:39:26 UTC (rev 199453)
</span><span class="lines">@@ -39,6 +39,23 @@
</span><span class="cx"> namespace bmalloc {
</span><span class="cx">
</span><span class="cx"> class Chunk {
</span><ins>+ // Our metadata layout includes a left and right edge sentinel.
+ // Metadata takes up enough space to leave at least the first two
+ // boundary tag slots unused.
+ //
+ // So, boundary tag space looks like this:
+ //
+ // [OOXXXXX...]
+ //
+ // And BoundaryTag::get subtracts one, producing:
+ //
+ // [OXXXXX...O].
+ //
+ // We use the X's for boundary tags and the O's for edge sentinels.
+
+ static const size_t boundaryTagCount = chunkSize / largeMin;
+ static_assert(boundaryTagCount > 2, "Chunk must have space for two sentinel boundary tags");
+
</ins><span class="cx"> public:
</span><span class="cx"> static Chunk* get(void*);
</span><span class="cx">
</span><span class="lines">@@ -53,33 +70,14 @@
</span><span class="cx"> SmallPage* page(size_t offset);
</span><span class="cx"> SmallLine* line(size_t offset);
</span><span class="cx">
</span><ins>+ char* bytes() { return reinterpret_cast<char*>(this); }
</ins><span class="cx"> SmallLine* lines() { return m_lines.begin(); }
</span><span class="cx"> SmallPage* pages() { return m_pages.begin(); }
</span><ins>+ std::array<BoundaryTag, boundaryTagCount>& boundaryTags() { return m_boundaryTags; }
</ins><span class="cx">
</span><del>- char* begin() { return roundUpToMultipleOf(vmPageSizePhysical(), m_memory); }
- char* end() { return reinterpret_cast<char*>(this) + chunkSize; }
- size_t size() { return end() - begin(); }
-
</del><span class="cx"> ObjectType objectType() { return m_objectType; }
</span><span class="cx">
</span><span class="cx"> private:
</span><del>- static const size_t boundaryTagCount = chunkSize / largeMin;
- static_assert(boundaryTagCount > 2, "Chunk must have space for two sentinel boundary tags");
-
- // Our metadata layout includes a left and right edge sentinel.
- // Metadata takes up enough space to leave at least the first two
- // boundary tag slots unused.
- //
- // So, boundary tag space looks like this:
- //
- // [OOXXXXX...]
- //
- // And BoundaryTag::get subtracts one, producing:
- //
- // [OXXXXX...O].
- //
- // We use the X's for boundary tags and the O's for edge sentinels.
-
</del><span class="cx"> union {
</span><span class="cx"> // The first few bytes of metadata cover the metadata region, so they're
</span><span class="cx"> // not used. We can steal them to store m_objectType.
</span><span class="lines">@@ -93,7 +91,6 @@
</span><span class="cx"> std::array<SmallPage, chunkSize / smallPageSize> m_pages;
</span><span class="cx"> std::array<BoundaryTag, boundaryTagCount> m_boundaryTags;
</span><span class="cx"> };
</span><del>- char m_memory[];
</del><span class="cx"> };
</span><span class="cx">
</span><span class="cx"> static_assert(sizeof(Chunk) + largeMax <= chunkSize, "largeMax is too big");
</span><span class="lines">@@ -104,33 +101,6 @@
</span><span class="cx"> inline Chunk::Chunk(std::lock_guard<StaticMutex>&, ObjectType objectType)
</span><span class="cx"> : m_objectType(objectType)
</span><span class="cx"> {
</span><del>- if (objectType != ObjectType::Large)
- return;
-
- Range range(begin(), size());
- BASSERT(range.size() <= largeObjectMax);
-
- BeginTag* beginTag = Chunk::beginTag(range.begin());
- beginTag->setRange(range);
- beginTag->setFree(true);
- beginTag->setVMState(VMState::Virtual);
-
- EndTag* endTag = Chunk::endTag(range.begin(), range.size());
- endTag->init(beginTag);
-
- // Mark the left and right edges of our range as allocated. This naturally
- // prevents merging logic from overflowing left (into metadata) or right
- // (beyond our chunk), without requiring special-case checks.
-
- EndTag* leftSentinel = beginTag->prev();
- BASSERT(leftSentinel >= m_boundaryTags.begin());
- BASSERT(leftSentinel < m_boundaryTags.end());
- leftSentinel->initSentinel();
-
- BeginTag* rightSentinel = endTag->next();
- BASSERT(rightSentinel >= m_boundaryTags.begin());
- BASSERT(rightSentinel < m_boundaryTags.end());
- rightSentinel->initSentinel();
</del><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> inline Chunk* Chunk::get(void* object)
</span><span class="lines">@@ -161,13 +131,13 @@
</span><span class="cx"> inline size_t Chunk::offset(void* object)
</span><span class="cx"> {
</span><span class="cx"> BASSERT(object >= this);
</span><del>- BASSERT(object < reinterpret_cast<char*>(this) + chunkSize);
- return static_cast<char*>(object) - reinterpret_cast<char*>(this);
</del><ins>+ BASSERT(object < bytes() + chunkSize);
+ return static_cast<char*>(object) - bytes();
</ins><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> inline char* Chunk::object(size_t offset)
</span><span class="cx"> {
</span><del>- return reinterpret_cast<char*>(this) + offset;
</del><ins>+ return bytes() + offset;
</ins><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> inline SmallPage* Chunk::page(size_t offset)
</span></span></pre></div>
<a id="releasesWebKitGTKwebkit212SourcebmallocbmallocVMHeapcpp"></a>
<div class="modfile"><h4>Modified: releases/WebKitGTK/webkit-2.12/Source/bmalloc/bmalloc/VMHeap.cpp (199452 => 199453)</h4>
<pre class="diff"><span>
<span class="info">--- releases/WebKitGTK/webkit-2.12/Source/bmalloc/bmalloc/VMHeap.cpp 2016-04-13 09:39:10 UTC (rev 199452)
+++ releases/WebKitGTK/webkit-2.12/Source/bmalloc/bmalloc/VMHeap.cpp 2016-04-13 09:39:26 UTC (rev 199453)
</span><span class="lines">@@ -44,7 +44,35 @@
</span><span class="cx"> m_zone.addChunk(chunk);
</span><span class="cx"> #endif
</span><span class="cx">
</span><del>- return LargeObject(chunk->begin());
</del><ins>+ size_t alignment = largeAlignment;
+ size_t metadataSize = roundUpToMultipleOf(alignment, sizeof(Chunk));
+
+ Range range(chunk->bytes() + metadataSize, chunkSize - metadataSize);
+ BASSERT(range.size() <= largeObjectMax);
+
+ BeginTag* beginTag = Chunk::beginTag(range.begin());
+ beginTag->setRange(range);
+ beginTag->setFree(true);
+ beginTag->setVMState(VMState::Virtual);
+
+ EndTag* endTag = Chunk::endTag(range.begin(), range.size());
+ endTag->init(beginTag);
+
+ // Mark the left and right edges of our range as allocated. This naturally
+ // prevents merging logic from overflowing left (into metadata) or right
+ // (beyond our chunk), without requiring special-case checks.
+
+ EndTag* leftSentinel = beginTag->prev();
+ BASSERT(leftSentinel >= chunk->boundaryTags().begin());
+ BASSERT(leftSentinel < chunk->boundaryTags().end());
+ leftSentinel->initSentinel();
+
+ BeginTag* rightSentinel = endTag->next();
+ BASSERT(rightSentinel >= chunk->boundaryTags().begin());
+ BASSERT(rightSentinel < chunk->boundaryTags().end());
+ rightSentinel->initSentinel();
+
+ return LargeObject(range.begin());
</ins><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> void VMHeap::allocateSmallChunk(std::lock_guard<StaticMutex>& lock, size_t pageClass)
</span><span class="lines">@@ -59,9 +87,16 @@
</span><span class="cx"> size_t pageSize = bmalloc::pageSize(pageClass);
</span><span class="cx"> size_t smallPageCount = pageSize / smallPageSize;
</span><span class="cx">
</span><del>- Object begin(chunk->begin());
- Object end(begin + chunk->size());
</del><ins>+ // If our page size is a power of two, we align to it in order to guarantee
+ // that we can service aligned allocation requests at the same power of two.
+ size_t alignment = vmPageSizePhysical();
+ if (isPowerOfTwo(pageSize))
+ alignment = pageSize;
+ size_t metadataSize = roundUpToMultipleOf(alignment, sizeof(Chunk));
</ins><span class="cx">
</span><ins>+ Object begin(chunk, metadataSize);
+ Object end(chunk, chunkSize);
+
</ins><span class="cx"> for (Object it = begin; it + pageSize <= end; it = it + pageSize) {
</span><span class="cx"> SmallPage* page = it.page();
</span><span class="cx"> new (page) SmallPage;
</span></span></pre>
</div>
</div>
</body>
</html>