<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
"http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head><meta http-equiv="content-type" content="text/html; charset=utf-8" />
<title>[167502] trunk/Source/bmalloc</title>
</head>
<body>
<style type="text/css"><!--
#msg dl.meta { border: 1px #006 solid; background: #369; padding: 6px; color: #fff; }
#msg dl.meta dt { float: left; width: 6em; font-weight: bold; }
#msg dt:after { content:':';}
#msg dl, #msg dt, #msg ul, #msg li, #header, #footer, #logmsg { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt; }
#msg dl a { font-weight: bold}
#msg dl a:link { color:#fc3; }
#msg dl a:active { color:#ff0; }
#msg dl a:visited { color:#cc6; }
h3 { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt; font-weight: bold; }
#msg pre { overflow: auto; background: #ffc; border: 1px #fa0 solid; padding: 6px; }
#logmsg { background: #ffc; border: 1px #fa0 solid; padding: 1em 1em 0 1em; }
#logmsg p, #logmsg pre, #logmsg blockquote { margin: 0 0 1em 0; }
#logmsg p, #logmsg li, #logmsg dt, #logmsg dd { line-height: 14pt; }
#logmsg h1, #logmsg h2, #logmsg h3, #logmsg h4, #logmsg h5, #logmsg h6 { margin: .5em 0; }
#logmsg h1:first-child, #logmsg h2:first-child, #logmsg h3:first-child, #logmsg h4:first-child, #logmsg h5:first-child, #logmsg h6:first-child { margin-top: 0; }
#logmsg ul, #logmsg ol { padding: 0; list-style-position: inside; margin: 0 0 0 1em; }
#logmsg ul { text-indent: -1em; padding-left: 1em; }#logmsg ol { text-indent: -1.5em; padding-left: 1.5em; }
#logmsg > ul, #logmsg > ol { margin: 0 0 1em 0; }
#logmsg pre { background: #eee; padding: 1em; }
#logmsg blockquote { border: 1px solid #fa0; border-left-width: 10px; padding: 1em 1em 0 1em; background: white;}
#logmsg dl { margin: 0; }
#logmsg dt { font-weight: bold; }
#logmsg dd { margin: 0; padding: 0 0 0.5em 0; }
#logmsg dd:before { content:'\00bb';}
#logmsg table { border-spacing: 0px; border-collapse: collapse; border-top: 4px solid #fa0; border-bottom: 1px solid #fa0; background: #fff; }
#logmsg table th { text-align: left; font-weight: normal; padding: 0.2em 0.5em; border-top: 1px dotted #fa0; }
#logmsg table td { text-align: right; border-top: 1px dotted #fa0; padding: 0.2em 0.5em; }
#logmsg table thead th { text-align: center; border-bottom: 1px solid #fa0; }
#logmsg table th.Corner { text-align: left; }
#logmsg hr { border: none 0; border-top: 2px dashed #fa0; height: 1px; }
#header, #footer { color: #fff; background: #636; border: 1px #300 solid; padding: 6px; }
#patch { width: 100%; }
#patch h4 {font-family: verdana,arial,helvetica,sans-serif;font-size:10pt;padding:8px;background:#369;color:#fff;margin:0;}
#patch .propset h4, #patch .binary h4 {margin:0;}
#patch pre {padding:0;line-height:1.2em;margin:0;}
#patch .diff {width:100%;background:#eee;padding: 0 0 10px 0;overflow:auto;}
#patch .propset .diff, #patch .binary .diff {padding:10px 0;}
#patch span {display:block;padding:0 10px;}
#patch .modfile, #patch .addfile, #patch .delfile, #patch .propset, #patch .binary, #patch .copfile {border:1px solid #ccc;margin:10px 0;}
#patch ins {background:#dfd;text-decoration:none;display:block;padding:0 10px;}
#patch del {background:#fdd;text-decoration:none;display:block;padding:0 10px;}
#patch .lines, .info {color:#888;background:#fff;}
--></style>
<div id="msg">
<dl class="meta">
<dt>Revision</dt> <dd><a href="http://trac.webkit.org/projects/webkit/changeset/167502">167502</a></dd>
<dt>Author</dt> <dd>ggaren@apple.com</dd>
<dt>Date</dt> <dd>2014-04-18 13:17:59 -0700 (Fri, 18 Apr 2014)</dd>
</dl>
<h3>Log Message</h3>
<pre>bmalloc: Added an XSmall line size
https://bugs.webkit.org/show_bug.cgi?id=131851
Reviewed by Sam Weinig.
Reduces malloc footprint on Membuster recordings by 10%.
This is a throughput regression, but we're still way ahead of TCMalloc.
I have some ideas for how to recover the regression -- but I wanted to
get this win in first.
Full set of benchmark results:
bmalloc> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks --measure-heap nopatch:~/scratch/Build-nopatch/Release/ patch:~/webkit/WebKitBuild/Release/
nopatch patch Δ
Peak Memory:
reddit_memory_warning 7,896kB 7,532kB ^ 1.05x smaller
flickr_memory_warning 12,968kB 12,324kB ^ 1.05x smaller
theverge_memory_warning 16,672kB 15,200kB ^ 1.1x smaller
<geometric mean> 11,952kB 11,216kB ^ 1.07x smaller
<arithmetic mean> 12,512kB 11,685kB ^ 1.07x smaller
<harmonic mean> 11,375kB 10,726kB ^ 1.06x smaller
Memory at End:
reddit_memory_warning 7,320kB 6,856kB ^ 1.07x smaller
flickr_memory_warning 10,848kB 9,692kB ^ 1.12x smaller
theverge_memory_warning 16,380kB 14,872kB ^ 1.1x smaller
<geometric mean> 10,916kB 9,961kB ^ 1.1x smaller
<arithmetic mean> 11,516kB 10,473kB ^ 1.1x smaller
<harmonic mean> 10,350kB 9,485kB ^ 1.09x smaller
MallocBench> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks nopatch:~/scratch/Build-nopatch/Release/ patch:~/webkit/WebKitBuild/Release/
nopatch patch Δ
Execution Time:
churn 127ms 151ms ! 1.19x slower
list_allocate 130ms 164ms ! 1.26x slower
tree_allocate 109ms 127ms ! 1.17x slower
tree_churn 115ms 120ms ! 1.04x slower
facebook 240ms 259ms ! 1.08x slower
fragment 91ms 131ms ! 1.44x slower
fragment_iterate 105ms 106ms ! 1.01x slower
message_one 260ms 259ms ^ 1.0x faster
message_many 149ms 154ms ! 1.03x slower
medium 194ms 248ms ! 1.28x slower
big 157ms 160ms ! 1.02x slower
<geometric mean> 144ms 163ms ! 1.13x slower
<arithmetic mean> 152ms 171ms ! 1.12x slower
<harmonic mean> 137ms 156ms ! 1.14x slower
MallocBench> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks nopatch:~/scratch/Build-nopatch/Release/ patch:~/webkit/WebKitBuild/Release/
nopatch patch Δ
Execution Time:
churn 126ms 148ms ! 1.17x slower
churn --parallel 62ms 76ms ! 1.23x slower
list_allocate 130ms 164ms ! 1.26x slower
list_allocate --parallel 120ms 175ms ! 1.46x slower
tree_allocate 111ms 127ms ! 1.14x slower
tree_allocate --parallel 95ms 135ms ! 1.42x slower
tree_churn 115ms 124ms ! 1.08x slower
tree_churn --parallel 107ms 126ms ! 1.18x slower
facebook 240ms 276ms ! 1.15x slower
facebook --parallel 802ms 1,088ms ! 1.36x slower
fragment 92ms 130ms ! 1.41x slower
fragment --parallel 66ms 124ms ! 1.88x slower
fragment_iterate 109ms 127ms ! 1.17x slower
fragment_iterate --parallel 55ms 64ms ! 1.16x slower
message_one 260ms 260ms
message_many 170ms 238ms ! 1.4x slower
medium 185ms 250ms ! 1.35x slower
medium --parallel 210ms 334ms ! 1.59x slower
big 150ms 169ms ! 1.13x slower
big --parallel 138ms 144ms ! 1.04x slower
<geometric mean> 135ms 170ms ! 1.26x slower
<arithmetic mean> 167ms 214ms ! 1.28x slower
<harmonic mean> 117ms 148ms ! 1.26x slower
MallocBench> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks TC:~/scratch/Build-TCMalloc/Release/ patch:~/webkit/WebKitBuild/Release/
TC patch Δ
Peak Memory:
reddit_memory_warning 13,836kB 13,436kB ^ 1.03x smaller
flickr_memory_warning 24,868kB 25,188kB ! 1.01x bigger
theverge_memory_warning 24,504kB 26,636kB ! 1.09x bigger
<geometric mean> 20,353kB 20,812kB ! 1.02x bigger
<arithmetic mean> 21,069kB 21,753kB ! 1.03x bigger
<harmonic mean> 19,570kB 19,780kB ! 1.01x bigger
Memory at End:
reddit_memory_warning 8,656kB 10,016kB ! 1.16x bigger
flickr_memory_warning 11,844kB 13,784kB ! 1.16x bigger
theverge_memory_warning 18,516kB 22,748kB ! 1.23x bigger
<geometric mean> 12,382kB 14,644kB ! 1.18x bigger
<arithmetic mean> 13,005kB 15,516kB ! 1.19x bigger
<harmonic mean> 11,813kB 13,867kB ! 1.17x bigger
MallocBench> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks TC:~/scratch/Build-TCMalloc/Release/ patch:~/webkit/WebKitBuild/Release/
TC patch Δ
Execution Time:
churn 416ms 148ms ^ 2.81x faster
list_allocate 463ms 164ms ^ 2.82x faster
tree_allocate 292ms 127ms ^ 2.3x faster
tree_churn 157ms 120ms ^ 1.31x faster
facebook 327ms 276ms ^ 1.18x faster
fragment 335ms 129ms ^ 2.6x faster
fragment_iterate 344ms 108ms ^ 3.19x faster
message_one 386ms 258ms ^ 1.5x faster
message_many 410ms 154ms ^ 2.66x faster
medium 391ms 245ms ^ 1.6x faster
big 261ms 167ms ^ 1.56x faster
<geometric mean> 332ms 164ms ^ 2.02x faster
<arithmetic mean> 344ms 172ms ^ 1.99x faster
<harmonic mean> 317ms 157ms ^ 2.02x faster
* bmalloc.xcodeproj/project.pbxproj:
* bmalloc/Allocator.cpp:
(bmalloc::Allocator::Allocator): Don't assume that each allocator's
index corresponds with its size. Instead, use the size selection function
explicitly. Now that we have XSmall, some small allocator entries are
unused.
(bmalloc::Allocator::scavenge):
(bmalloc::Allocator::log):
(bmalloc::Allocator::processXSmallAllocatorLog):
(bmalloc::Allocator::allocateSlowCase):
* bmalloc/Allocator.h:
(bmalloc::Allocator::xSmallAllocatorFor):
(bmalloc::Allocator::allocateFastCase):
* bmalloc/Chunk.h:
* bmalloc/Deallocator.cpp:
(bmalloc::Deallocator::scavenge):
(bmalloc::Deallocator::processObjectLog):
(bmalloc::Deallocator::deallocateSlowCase):
(bmalloc::Deallocator::deallocateXSmallLine):
(bmalloc::Deallocator::allocateXSmallLine):
* bmalloc/Deallocator.h:
(bmalloc::Deallocator::deallocateFastCase):
* bmalloc/Heap.cpp:
(bmalloc::Heap::scavenge):
(bmalloc::Heap::scavengeXSmallPages):
(bmalloc::Heap::allocateXSmallLineSlowCase):
* bmalloc/Heap.h:
(bmalloc::Heap::deallocateXSmallLine):
(bmalloc::Heap::allocateXSmallLine):
* bmalloc/LargeChunk.h:
(bmalloc::LargeChunk::get):
(bmalloc::LargeChunk::endTag):
* bmalloc/Line.h:
* bmalloc/MediumAllocator.h:
(bmalloc::MediumAllocator::allocate):
(bmalloc::MediumAllocator::refill):
* bmalloc/ObjectType.cpp:
(bmalloc::objectType):
* bmalloc/ObjectType.h:
(bmalloc::isXSmall):
(bmalloc::isSmall):
(bmalloc::isMedium):
(bmalloc::isLarge):
(bmalloc::isSmallOrMedium): Deleted.
* bmalloc/SegregatedFreeList.h: I boiler-plate copied existing code for
handling small objects. There's probably a reasonable way to share this
code in the future -- I'll look into that once it's stopped changing.
* bmalloc/Sizes.h: Tweaked size classes to make Membuster happy. This
is the main reason things got slower.
* bmalloc/SmallAllocator.h:
(bmalloc::SmallAllocator::allocate):
* bmalloc/SmallTraits.h:
* bmalloc/VMHeap.cpp:
(bmalloc::VMHeap::allocateXSmallChunk):
* bmalloc/VMHeap.h:
(bmalloc::VMHeap::allocateXSmallPage):
(bmalloc::VMHeap::deallocateXSmallPage):
* bmalloc/XSmallAllocator.h: Added.
(bmalloc::XSmallAllocator::isNull):
(bmalloc::XSmallAllocator::canAllocate):
(bmalloc::XSmallAllocator::XSmallAllocator):
(bmalloc::XSmallAllocator::line):
(bmalloc::XSmallAllocator::allocate):
(bmalloc::XSmallAllocator::objectCount):
(bmalloc::XSmallAllocator::derefCount):
(bmalloc::XSmallAllocator::refill):
* bmalloc/XSmallChunk.h: Added.
* bmalloc/XSmallLine.h: Added.
* bmalloc/XSmallPage.h: Added.
* bmalloc/XSmallTraits.h: Added.
* bmalloc/bmalloc.h:
(bmalloc::api::realloc): Boiler-plate copy, as above.</pre>
<h3>Modified Paths</h3>
<ul>
<li><a href="#trunkSourcebmallocChangeLog">trunk/Source/bmalloc/ChangeLog</a></li>
<li><a href="#trunkSourcebmallocbmallocAllocatorcpp">trunk/Source/bmalloc/bmalloc/Allocator.cpp</a></li>
<li><a href="#trunkSourcebmallocbmallocAllocatorh">trunk/Source/bmalloc/bmalloc/Allocator.h</a></li>
<li><a href="#trunkSourcebmallocbmallocChunkh">trunk/Source/bmalloc/bmalloc/Chunk.h</a></li>
<li><a href="#trunkSourcebmallocbmallocDeallocatorcpp">trunk/Source/bmalloc/bmalloc/Deallocator.cpp</a></li>
<li><a href="#trunkSourcebmallocbmallocDeallocatorh">trunk/Source/bmalloc/bmalloc/Deallocator.h</a></li>
<li><a href="#trunkSourcebmallocbmallocHeapcpp">trunk/Source/bmalloc/bmalloc/Heap.cpp</a></li>
<li><a href="#trunkSourcebmallocbmallocHeaph">trunk/Source/bmalloc/bmalloc/Heap.h</a></li>
<li><a href="#trunkSourcebmallocbmallocLargeChunkh">trunk/Source/bmalloc/bmalloc/LargeChunk.h</a></li>
<li><a href="#trunkSourcebmallocbmallocLineh">trunk/Source/bmalloc/bmalloc/Line.h</a></li>
<li><a href="#trunkSourcebmallocbmallocMediumAllocatorh">trunk/Source/bmalloc/bmalloc/MediumAllocator.h</a></li>
<li><a href="#trunkSourcebmallocbmallocObjectTypecpp">trunk/Source/bmalloc/bmalloc/ObjectType.cpp</a></li>
<li><a href="#trunkSourcebmallocbmallocObjectTypeh">trunk/Source/bmalloc/bmalloc/ObjectType.h</a></li>
<li><a href="#trunkSourcebmallocbmallocSegregatedFreeListh">trunk/Source/bmalloc/bmalloc/SegregatedFreeList.h</a></li>
<li><a href="#trunkSourcebmallocbmallocSizesh">trunk/Source/bmalloc/bmalloc/Sizes.h</a></li>
<li><a href="#trunkSourcebmallocbmallocSmallAllocatorh">trunk/Source/bmalloc/bmalloc/SmallAllocator.h</a></li>
<li><a href="#trunkSourcebmallocbmallocSmallTraitsh">trunk/Source/bmalloc/bmalloc/SmallTraits.h</a></li>
<li><a href="#trunkSourcebmallocbmallocVMHeapcpp">trunk/Source/bmalloc/bmalloc/VMHeap.cpp</a></li>
<li><a href="#trunkSourcebmallocbmallocVMHeaph">trunk/Source/bmalloc/bmalloc/VMHeap.h</a></li>
<li><a href="#trunkSourcebmallocbmallocbmalloch">trunk/Source/bmalloc/bmalloc/bmalloc.h</a></li>
<li><a href="#trunkSourcebmallocbmallocxcodeprojprojectpbxproj">trunk/Source/bmalloc/bmalloc.xcodeproj/project.pbxproj</a></li>
</ul>
<h3>Added Paths</h3>
<ul>
<li><a href="#trunkSourcebmallocbmallocXSmallAllocatorh">trunk/Source/bmalloc/bmalloc/XSmallAllocator.h</a></li>
<li><a href="#trunkSourcebmallocbmallocXSmallChunkh">trunk/Source/bmalloc/bmalloc/XSmallChunk.h</a></li>
<li><a href="#trunkSourcebmallocbmallocXSmallLineh">trunk/Source/bmalloc/bmalloc/XSmallLine.h</a></li>
<li><a href="#trunkSourcebmallocbmallocXSmallPageh">trunk/Source/bmalloc/bmalloc/XSmallPage.h</a></li>
<li><a href="#trunkSourcebmallocbmallocXSmallTraitsh">trunk/Source/bmalloc/bmalloc/XSmallTraits.h</a></li>
</ul>
</div>
<div id="patch">
<h3>Diff</h3>
<a id="trunkSourcebmallocChangeLog"></a>
<div class="modfile"><h4>Modified: trunk/Source/bmalloc/ChangeLog (167501 => 167502)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/bmalloc/ChangeLog 2014-04-18 20:07:50 UTC (rev 167501)
+++ trunk/Source/bmalloc/ChangeLog 2014-04-18 20:17:59 UTC (rev 167502)
</span><span class="lines">@@ -1,3 +1,205 @@
</span><ins>+2014-04-18 Geoffrey Garen <ggaren@apple.com>
+
+ bmalloc: Added an XSmall line size
+ https://bugs.webkit.org/show_bug.cgi?id=131851
+
+ Reviewed by Sam Weinig.
+
+ Reduces malloc footprint on Membuster recordings by 10%.
+
+ This is a throughput regression, but we're still way ahead of TCMalloc.
+ I have some ideas for how to recover the regression -- but I wanted to
+ get this win in first.
+
+ Full set of benchmark results:
+
+ bmalloc> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks --measure-heap nopatch:~/scratch/Build-nopatch/Release/ patch:~/webkit/WebKitBuild/Release/
+
+ nopatch patch Δ
+ Peak Memory:
+ reddit_memory_warning 7,896kB 7,532kB ^ 1.05x smaller
+ flickr_memory_warning 12,968kB 12,324kB ^ 1.05x smaller
+ theverge_memory_warning 16,672kB 15,200kB ^ 1.1x smaller
+
+ <geometric mean> 11,952kB 11,216kB ^ 1.07x smaller
+ <arithmetic mean> 12,512kB 11,685kB ^ 1.07x smaller
+ <harmonic mean> 11,375kB 10,726kB ^ 1.06x smaller
+
+ Memory at End:
+ reddit_memory_warning 7,320kB 6,856kB ^ 1.07x smaller
+ flickr_memory_warning 10,848kB 9,692kB ^ 1.12x smaller
+ theverge_memory_warning 16,380kB 14,872kB ^ 1.1x smaller
+
+ <geometric mean> 10,916kB 9,961kB ^ 1.1x smaller
+ <arithmetic mean> 11,516kB 10,473kB ^ 1.1x smaller
+ <harmonic mean> 10,350kB 9,485kB ^ 1.09x smaller
+
+ MallocBench> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks nopatch:~/scratch/Build-nopatch/Release/ patch:~/webkit/WebKitBuild/Release/
+
+ nopatch patch Δ
+ Execution Time:
+ churn 127ms 151ms ! 1.19x slower
+ list_allocate 130ms 164ms ! 1.26x slower
+ tree_allocate 109ms 127ms ! 1.17x slower
+ tree_churn 115ms 120ms ! 1.04x slower
+ facebook 240ms 259ms ! 1.08x slower
+ fragment 91ms 131ms ! 1.44x slower
+ fragment_iterate 105ms 106ms ! 1.01x slower
+ message_one 260ms 259ms ^ 1.0x faster
+ message_many 149ms 154ms ! 1.03x slower
+ medium 194ms 248ms ! 1.28x slower
+ big 157ms 160ms ! 1.02x slower
+
+ <geometric mean> 144ms 163ms ! 1.13x slower
+ <arithmetic mean> 152ms 171ms ! 1.12x slower
+ <harmonic mean> 137ms 156ms ! 1.14x slower
+
+ MallocBench> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks nopatch:~/scratch/Build-nopatch/Release/ patch:~/webkit/WebKitBuild/Release/
+
+ nopatch patch Δ
+ Execution Time:
+ churn 126ms 148ms ! 1.17x slower
+ churn --parallel 62ms 76ms ! 1.23x slower
+ list_allocate 130ms 164ms ! 1.26x slower
+ list_allocate --parallel 120ms 175ms ! 1.46x slower
+ tree_allocate 111ms 127ms ! 1.14x slower
+ tree_allocate --parallel 95ms 135ms ! 1.42x slower
+ tree_churn 115ms 124ms ! 1.08x slower
+ tree_churn --parallel 107ms 126ms ! 1.18x slower
+ facebook 240ms 276ms ! 1.15x slower
+ facebook --parallel 802ms 1,088ms ! 1.36x slower
+ fragment 92ms 130ms ! 1.41x slower
+ fragment --parallel 66ms 124ms ! 1.88x slower
+ fragment_iterate 109ms 127ms ! 1.17x slower
+ fragment_iterate --parallel 55ms 64ms ! 1.16x slower
+ message_one 260ms 260ms
+ message_many 170ms 238ms ! 1.4x slower
+ medium 185ms 250ms ! 1.35x slower
+ medium --parallel 210ms 334ms ! 1.59x slower
+ big 150ms 169ms ! 1.13x slower
+ big --parallel 138ms 144ms ! 1.04x slower
+
+ <geometric mean> 135ms 170ms ! 1.26x slower
+ <arithmetic mean> 167ms 214ms ! 1.28x slower
+ <harmonic mean> 117ms 148ms ! 1.26x slower
+
+ MallocBench> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks TC:~/scratch/Build-TCMalloc/Release/ patch:~/webkit/WebKitBuild/Release/
+
+ TC patch Δ
+ Peak Memory:
+ reddit_memory_warning 13,836kB 13,436kB ^ 1.03x smaller
+ flickr_memory_warning 24,868kB 25,188kB ! 1.01x bigger
+ theverge_memory_warning 24,504kB 26,636kB ! 1.09x bigger
+
+ <geometric mean> 20,353kB 20,812kB ! 1.02x bigger
+ <arithmetic mean> 21,069kB 21,753kB ! 1.03x bigger
+ <harmonic mean> 19,570kB 19,780kB ! 1.01x bigger
+
+ Memory at End:
+ reddit_memory_warning 8,656kB 10,016kB ! 1.16x bigger
+ flickr_memory_warning 11,844kB 13,784kB ! 1.16x bigger
+ theverge_memory_warning 18,516kB 22,748kB ! 1.23x bigger
+
+ <geometric mean> 12,382kB 14,644kB ! 1.18x bigger
+ <arithmetic mean> 13,005kB 15,516kB ! 1.19x bigger
+ <harmonic mean> 11,813kB 13,867kB ! 1.17x bigger
+
+ MallocBench> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks TC:~/scratch/Build-TCMalloc/Release/ patch:~/webkit/WebKitBuild/Release/
+
+ TC patch Δ
+ Execution Time:
+ churn 416ms 148ms ^ 2.81x faster
+ list_allocate 463ms 164ms ^ 2.82x faster
+ tree_allocate 292ms 127ms ^ 2.3x faster
+ tree_churn 157ms 120ms ^ 1.31x faster
+ facebook 327ms 276ms ^ 1.18x faster
+ fragment 335ms 129ms ^ 2.6x faster
+ fragment_iterate 344ms 108ms ^ 3.19x faster
+ message_one 386ms 258ms ^ 1.5x faster
+ message_many 410ms 154ms ^ 2.66x faster
+ medium 391ms 245ms ^ 1.6x faster
+ big 261ms 167ms ^ 1.56x faster
+
+ <geometric mean> 332ms 164ms ^ 2.02x faster
+ <arithmetic mean> 344ms 172ms ^ 1.99x faster
+ <harmonic mean> 317ms 157ms ^ 2.02x faster
+
+ * bmalloc.xcodeproj/project.pbxproj:
+ * bmalloc/Allocator.cpp:
+ (bmalloc::Allocator::Allocator): Don't assume that each allocator's
+ index corresponds with its size. Instead, use the size selection function
+ explicitly. Now that we have XSmall, some small allocator entries are
+ unused.
+
+ (bmalloc::Allocator::scavenge):
+ (bmalloc::Allocator::log):
+ (bmalloc::Allocator::processXSmallAllocatorLog):
+ (bmalloc::Allocator::allocateSlowCase):
+ * bmalloc/Allocator.h:
+ (bmalloc::Allocator::xSmallAllocatorFor):
+ (bmalloc::Allocator::allocateFastCase):
+ * bmalloc/Chunk.h:
+ * bmalloc/Deallocator.cpp:
+ (bmalloc::Deallocator::scavenge):
+ (bmalloc::Deallocator::processObjectLog):
+ (bmalloc::Deallocator::deallocateSlowCase):
+ (bmalloc::Deallocator::deallocateXSmallLine):
+ (bmalloc::Deallocator::allocateXSmallLine):
+ * bmalloc/Deallocator.h:
+ (bmalloc::Deallocator::deallocateFastCase):
+ * bmalloc/Heap.cpp:
+ (bmalloc::Heap::scavenge):
+ (bmalloc::Heap::scavengeXSmallPages):
+ (bmalloc::Heap::allocateXSmallLineSlowCase):
+ * bmalloc/Heap.h:
+ (bmalloc::Heap::deallocateXSmallLine):
+ (bmalloc::Heap::allocateXSmallLine):
+ * bmalloc/LargeChunk.h:
+ (bmalloc::LargeChunk::get):
+ (bmalloc::LargeChunk::endTag):
+ * bmalloc/Line.h:
+ * bmalloc/MediumAllocator.h:
+ (bmalloc::MediumAllocator::allocate):
+ (bmalloc::MediumAllocator::refill):
+ * bmalloc/ObjectType.cpp:
+ (bmalloc::objectType):
+ * bmalloc/ObjectType.h:
+ (bmalloc::isXSmall):
+ (bmalloc::isSmall):
+ (bmalloc::isMedium):
+ (bmalloc::isLarge):
+ (bmalloc::isSmallOrMedium): Deleted.
+ * bmalloc/SegregatedFreeList.h: I boiler-plate copied existing code for
+ handling small objects. There's probably a reasonable way to share this
+ code in the future -- I'll look into that once it's stopped changing.
+
+ * bmalloc/Sizes.h: Tweaked size classes to make Membuster happy. This
+ is the main reason things got slower.
+
+ * bmalloc/SmallAllocator.h:
+ (bmalloc::SmallAllocator::allocate):
+ * bmalloc/SmallTraits.h:
+ * bmalloc/VMHeap.cpp:
+ (bmalloc::VMHeap::allocateXSmallChunk):
+ * bmalloc/VMHeap.h:
+ (bmalloc::VMHeap::allocateXSmallPage):
+ (bmalloc::VMHeap::deallocateXSmallPage):
+ * bmalloc/XSmallAllocator.h: Added.
+ (bmalloc::XSmallAllocator::isNull):
+ (bmalloc::XSmallAllocator::canAllocate):
+ (bmalloc::XSmallAllocator::XSmallAllocator):
+ (bmalloc::XSmallAllocator::line):
+ (bmalloc::XSmallAllocator::allocate):
+ (bmalloc::XSmallAllocator::objectCount):
+ (bmalloc::XSmallAllocator::derefCount):
+ (bmalloc::XSmallAllocator::refill):
+ * bmalloc/XSmallChunk.h: Added.
+ * bmalloc/XSmallLine.h: Added.
+ * bmalloc/XSmallPage.h: Added.
+ * bmalloc/XSmallTraits.h: Added.
+ * bmalloc/bmalloc.h:
+ (bmalloc::api::realloc): Boiler-plate copy, as above.
+
</ins><span class="cx"> 2014-04-14 Geoffrey Garen <ggaren@apple.com>
</span><span class="cx">
</span><span class="cx"> MallocBench should scavenge explicitly instead of waiting
</span></span></pre></div>
<a id="trunkSourcebmallocbmallocAllocatorcpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/bmalloc/bmalloc/Allocator.cpp (167501 => 167502)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/bmalloc/bmalloc/Allocator.cpp 2014-04-18 20:07:50 UTC (rev 167501)
+++ trunk/Source/bmalloc/bmalloc/Allocator.cpp 2014-04-18 20:17:59 UTC (rev 167502)
</span><span class="lines">@@ -43,10 +43,11 @@
</span><span class="cx"> , m_mediumAllocatorLog()
</span><span class="cx"> {
</span><span class="cx"> unsigned short size = alignment;
</span><del>- for (auto& allocator : m_smallAllocators) {
- allocator = SmallAllocator(size);
- size += alignment;
- }
</del><ins>+ for ( ; size <= xSmallMax; size += alignment)
+ xSmallAllocatorFor(size) = XSmallAllocator(size);
+
+ for ( ; size <= smallMax; size += alignment)
+ smallAllocatorFor(size) = SmallAllocator(size);
</ins><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> Allocator::~Allocator()
</span><span class="lines">@@ -56,6 +57,10 @@
</span><span class="cx">
</span><span class="cx"> void Allocator::scavenge()
</span><span class="cx"> {
</span><ins>+ for (auto& allocator : m_xSmallAllocators)
+ log(allocator);
+ processXSmallAllocatorLog();
+
</ins><span class="cx"> for (auto& allocator : m_smallAllocators)
</span><span class="cx"> log(allocator);
</span><span class="cx"> processSmallAllocatorLog();
</span><span class="lines">@@ -64,14 +69,37 @@
</span><span class="cx"> processMediumAllocatorLog();
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+void Allocator::log(XSmallAllocator& allocator)
+{
+ if (allocator.isNull())
+ return;
+
+ if (m_xSmallAllocatorLog.size() == m_xSmallAllocatorLog.capacity())
+ processXSmallAllocatorLog();
+
+ m_xSmallAllocatorLog.push(std::make_pair(allocator.line(), allocator.derefCount()));
+}
+
+void Allocator::processXSmallAllocatorLog()
+{
+ std::lock_guard<Mutex> lock(PerProcess<Heap>::mutex());
+
+ for (auto& logEntry : m_xSmallAllocatorLog) {
+ if (!logEntry.first->deref(lock, logEntry.second))
+ continue;
+ m_deallocator.deallocateXSmallLine(lock, logEntry.first);
+ }
+ m_xSmallAllocatorLog.clear();
+}
+
</ins><span class="cx"> void Allocator::log(SmallAllocator& allocator)
</span><span class="cx"> {
</span><ins>+ if (allocator.isNull())
+ return;
+
</ins><span class="cx"> if (m_smallAllocatorLog.size() == m_smallAllocatorLog.capacity())
</span><span class="cx"> processSmallAllocatorLog();
</span><span class="cx">
</span><del>- if (allocator.isNull())
- return;
-
</del><span class="cx"> m_smallAllocatorLog.push(std::make_pair(allocator.line(), allocator.derefCount()));
</span><span class="cx"> }
</span><span class="cx">
</span><span class="lines">@@ -89,12 +117,12 @@
</span><span class="cx">
</span><span class="cx"> void Allocator::log(MediumAllocator& allocator)
</span><span class="cx"> {
</span><ins>+ if (allocator.isNull())
+ return;
+
</ins><span class="cx"> if (m_mediumAllocatorLog.size() == m_mediumAllocatorLog.capacity())
</span><span class="cx"> processMediumAllocatorLog();
</span><span class="cx">
</span><del>- if (allocator.isNull())
- return;
-
</del><span class="cx"> m_mediumAllocatorLog.push(std::make_pair(allocator.line(), allocator.derefCount()));
</span><span class="cx"> }
</span><span class="cx">
</span><span class="lines">@@ -144,6 +172,13 @@
</span><span class="cx"> void* dummy;
</span><span class="cx"> BASSERT(!allocateFastCase(size, dummy));
</span><span class="cx"> )
</span><ins>+ if (size <= xSmallMax) {
+ XSmallAllocator& allocator = xSmallAllocatorFor(size);
+ log(allocator);
+ allocator.refill(m_deallocator.allocateXSmallLine());
+ return allocator.allocate();
+ }
+
</ins><span class="cx"> if (size <= smallMax) {
</span><span class="cx"> SmallAllocator& allocator = smallAllocatorFor(size);
</span><span class="cx"> log(allocator);
</span></span></pre></div>
<a id="trunkSourcebmallocbmallocAllocatorh"></a>
<div class="modfile"><h4>Modified: trunk/Source/bmalloc/bmalloc/Allocator.h (167501 => 167502)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/bmalloc/bmalloc/Allocator.h 2014-04-18 20:07:50 UTC (rev 167501)
+++ trunk/Source/bmalloc/bmalloc/Allocator.h 2014-04-18 20:17:59 UTC (rev 167502)
</span><span class="lines">@@ -30,6 +30,7 @@
</span><span class="cx"> #include "MediumAllocator.h"
</span><span class="cx"> #include "Sizes.h"
</span><span class="cx"> #include "SmallAllocator.h"
</span><ins>+#include "XSmallAllocator.h"
</ins><span class="cx"> #include <array>
</span><span class="cx">
</span><span class="cx"> namespace bmalloc {
</span><span class="lines">@@ -50,28 +51,40 @@
</span><span class="cx"> void scavenge();
</span><span class="cx">
</span><span class="cx"> private:
</span><ins>+ XSmallAllocator& xSmallAllocatorFor(size_t);
</ins><span class="cx"> SmallAllocator& smallAllocatorFor(size_t);
</span><ins>+
</ins><span class="cx"> void* allocateFastCase(SmallAllocator&);
</span><span class="cx">
</span><span class="cx"> void* allocateMedium(size_t);
</span><span class="cx"> void* allocateLarge(size_t);
</span><span class="cx"> void* allocateXLarge(size_t);
</span><span class="cx">
</span><ins>+ void log(XSmallAllocator&);
</ins><span class="cx"> void log(SmallAllocator&);
</span><span class="cx"> void log(MediumAllocator&);
</span><span class="cx">
</span><ins>+ void processXSmallAllocatorLog();
</ins><span class="cx"> void processSmallAllocatorLog();
</span><span class="cx"> void processMediumAllocatorLog();
</span><span class="cx">
</span><span class="cx"> Deallocator& m_deallocator;
</span><span class="cx">
</span><ins>+ std::array<XSmallAllocator, xSmallMax / alignment> m_xSmallAllocators;
</ins><span class="cx"> std::array<SmallAllocator, smallMax / alignment> m_smallAllocators;
</span><span class="cx"> MediumAllocator m_mediumAllocator;
</span><span class="cx">
</span><ins>+ FixedVector<std::pair<XSmallLine*, unsigned char>, xSmallAllocatorLogCapacity> m_xSmallAllocatorLog;
</ins><span class="cx"> FixedVector<std::pair<SmallLine*, unsigned char>, smallAllocatorLogCapacity> m_smallAllocatorLog;
</span><span class="cx"> FixedVector<std::pair<MediumLine*, unsigned char>, mediumAllocatorLogCapacity> m_mediumAllocatorLog;
</span><span class="cx"> };
</span><span class="cx">
</span><ins>+inline XSmallAllocator& Allocator::xSmallAllocatorFor(size_t size)
+{
+ size_t index = mask((size - 1ul) / alignment, m_xSmallAllocators.size() - 1);
+ return m_xSmallAllocators[index];
+}
+
</ins><span class="cx"> inline SmallAllocator& Allocator::smallAllocatorFor(size_t size)
</span><span class="cx"> {
</span><span class="cx"> size_t index = mask((size - 1ul) / alignment, m_smallAllocators.size() - 1);
</span><span class="lines">@@ -80,15 +93,25 @@
</span><span class="cx">
</span><span class="cx"> inline bool Allocator::allocateFastCase(size_t size, void*& object)
</span><span class="cx"> {
</span><del>- if (size > smallMax)
- return false;
</del><ins>+ if (size <= xSmallMax) {
+ XSmallAllocator& allocator = xSmallAllocatorFor(size);
+ if (!allocator.canAllocate())
+ return false;
+
+ object = allocator.allocate();
+ return true;
+ }
+
+ if (size <= smallMax) {
+ SmallAllocator& allocator = smallAllocatorFor(size);
+ if (!allocator.canAllocate())
+ return false;
</ins><span class="cx">
</span><del>- SmallAllocator& allocator = smallAllocatorFor(size);
- if (!allocator.canAllocate())
- return false;
-
- object = allocator.allocate();
- return true;
</del><ins>+ object = allocator.allocate();
+ return true;
+ }
+
+ return false;
</ins><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> inline void* Allocator::allocate(size_t size)
</span></span></pre></div>
<a id="trunkSourcebmallocbmallocChunkh"></a>
<div class="modfile"><h4>Modified: trunk/Source/bmalloc/bmalloc/Chunk.h (167501 => 167502)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/bmalloc/bmalloc/Chunk.h 2014-04-18 20:07:50 UTC (rev 167501)
+++ trunk/Source/bmalloc/bmalloc/Chunk.h 2014-04-18 20:17:59 UTC (rev 167502)
</span><span class="lines">@@ -77,7 +77,7 @@
</span><span class="cx"> template<class Traits>
</span><span class="cx"> inline auto Chunk<Traits>::get(void* object) -> Chunk*
</span><span class="cx"> {
</span><del>- BASSERT(isSmallOrMedium(object));
</del><ins>+ BASSERT(!isLarge(object));
</ins><span class="cx"> return static_cast<Chunk*>(mask(object, chunkMask));
</span><span class="cx"> }
</span><span class="cx">
</span></span></pre></div>
<a id="trunkSourcebmallocbmallocDeallocatorcpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/bmalloc/bmalloc/Deallocator.cpp (167501 => 167502)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/bmalloc/bmalloc/Deallocator.cpp 2014-04-18 20:07:50 UTC (rev 167501)
+++ trunk/Source/bmalloc/bmalloc/Deallocator.cpp 2014-04-18 20:17:59 UTC (rev 167502)
</span><span class="lines">@@ -31,6 +31,7 @@
</span><span class="cx"> #include "Inline.h"
</span><span class="cx"> #include "PerProcess.h"
</span><span class="cx"> #include "SmallChunk.h"
</span><ins>+#include "XSmallChunk.h"
</ins><span class="cx"> #include <algorithm>
</span><span class="cx"> #include <sys/mman.h>
</span><span class="cx">
</span><span class="lines">@@ -57,6 +58,8 @@
</span><span class="cx"> std::lock_guard<Mutex> lock(PerProcess<Heap>::mutex());
</span><span class="cx"> Heap* heap = PerProcess<Heap>::getFastCase();
</span><span class="cx">
</span><ins>+ while (m_xSmallLineCache.size())
+ heap->deallocateXSmallLine(lock, m_xSmallLineCache.pop());
</ins><span class="cx"> while (m_smallLineCache.size())
</span><span class="cx"> heap->deallocateSmallLine(lock, m_smallLineCache.pop());
</span><span class="cx"> while (m_mediumLineCache.size())
</span><span class="lines">@@ -80,13 +83,18 @@
</span><span class="cx"> std::lock_guard<Mutex> lock(PerProcess<Heap>::mutex());
</span><span class="cx">
</span><span class="cx"> for (auto object : m_objectLog) {
</span><del>- if (isSmall(object)) {
</del><ins>+ if (isXSmall(object)) {
+ XSmallLine* line = XSmallLine::get(object);
+ if (!line->deref(lock))
+ continue;
+ deallocateXSmallLine(lock, line);
+ } else if (isSmall(object)) {
</ins><span class="cx"> SmallLine* line = SmallLine::get(object);
</span><span class="cx"> if (!line->deref(lock))
</span><span class="cx"> continue;
</span><span class="cx"> deallocateSmallLine(lock, line);
</span><span class="cx"> } else {
</span><del>- BASSERT(isSmallOrMedium(object));
</del><ins>+ BASSERT(isMedium(object));
</ins><span class="cx"> MediumLine* line = MediumLine::get(object);
</span><span class="cx"> if (!line->deref(lock))
</span><span class="cx"> continue;
</span><span class="lines">@@ -104,7 +112,7 @@
</span><span class="cx"> if (!object)
</span><span class="cx"> return;
</span><span class="cx">
</span><del>- if (isSmallOrMedium(object)) {
</del><ins>+ if (!isLarge(object)) {
</ins><span class="cx"> processObjectLog();
</span><span class="cx"> m_objectLog.push(object);
</span><span class="cx"> return;
</span><span class="lines">@@ -125,6 +133,14 @@
</span><span class="cx"> m_smallLineCache.push(line);
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+void Deallocator::deallocateXSmallLine(std::lock_guard<Mutex>& lock, XSmallLine* line)
+{
+ if (m_xSmallLineCache.size() == m_xSmallLineCache.capacity())
+ return PerProcess<Heap>::getFastCase()->deallocateXSmallLine(lock, line);
+
+ m_xSmallLineCache.push(line);
+}
+
</ins><span class="cx"> SmallLine* Deallocator::allocateSmallLine()
</span><span class="cx"> {
</span><span class="cx"> if (!m_smallLineCache.size()) {
</span><span class="lines">@@ -138,6 +154,19 @@
</span><span class="cx"> return m_smallLineCache.pop();
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+XSmallLine* Deallocator::allocateXSmallLine()
+{
+ if (!m_xSmallLineCache.size()) {
+ std::lock_guard<Mutex> lock(PerProcess<Heap>::mutex());
+ Heap* heap = PerProcess<Heap>::getFastCase();
+
+ while (m_xSmallLineCache.size() != m_xSmallLineCache.capacity())
+ m_xSmallLineCache.push(heap->allocateXSmallLine(lock));
+ }
+
+ return m_xSmallLineCache.pop();
+}
+
</ins><span class="cx"> void Deallocator::deallocateMediumLine(std::lock_guard<Mutex>& lock, MediumLine* line)
</span><span class="cx"> {
</span><span class="cx"> if (m_mediumLineCache.size() == m_mediumLineCache.capacity())
</span></span></pre></div>
<a id="trunkSourcebmallocbmallocDeallocatorh"></a>
<div class="modfile"><h4>Modified: trunk/Source/bmalloc/bmalloc/Deallocator.h (167501 => 167502)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/bmalloc/bmalloc/Deallocator.h 2014-04-18 20:07:50 UTC (rev 167501)
+++ trunk/Source/bmalloc/bmalloc/Deallocator.h 2014-04-18 20:17:59 UTC (rev 167502)
</span><span class="lines">@@ -30,6 +30,7 @@
</span><span class="cx"> #include "MediumLine.h"
</span><span class="cx"> #include "Sizes.h"
</span><span class="cx"> #include "SmallLine.h"
</span><ins>+#include "XSmallLine.h"
</ins><span class="cx">
</span><span class="cx"> namespace bmalloc {
</span><span class="cx">
</span><span class="lines">@@ -44,6 +45,9 @@
</span><span class="cx"> bool deallocateFastCase(void*);
</span><span class="cx"> void deallocateSlowCase(void*);
</span><span class="cx">
</span><ins>+ void deallocateXSmallLine(std::lock_guard<Mutex>&, XSmallLine*);
+ XSmallLine* allocateXSmallLine();
+
</ins><span class="cx"> void deallocateSmallLine(std::lock_guard<Mutex>&, SmallLine*);
</span><span class="cx"> SmallLine* allocateSmallLine();
</span><span class="cx">
</span><span class="lines">@@ -58,13 +62,14 @@
</span><span class="cx"> void processObjectLog();
</span><span class="cx">
</span><span class="cx"> FixedVector<void*, deallocatorLogCapacity> m_objectLog;
</span><ins>+ FixedVector<XSmallLine*, xSmallLineCacheCapacity> m_xSmallLineCache;
</ins><span class="cx"> FixedVector<SmallLine*, smallLineCacheCapacity> m_smallLineCache;
</span><span class="cx"> FixedVector<MediumLine*, mediumLineCacheCapacity> m_mediumLineCache;
</span><span class="cx"> };
</span><span class="cx">
</span><span class="cx"> inline bool Deallocator::deallocateFastCase(void* object)
</span><span class="cx"> {
</span><del>- if (!isSmallOrMedium(object))
</del><ins>+ if (isLarge(object))
</ins><span class="cx"> return false;
</span><span class="cx">
</span><span class="cx"> BASSERT(object);
</span></span></pre></div>
<a id="trunkSourcebmallocbmallocHeapcpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/bmalloc/bmalloc/Heap.cpp (167501 => 167502)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/bmalloc/bmalloc/Heap.cpp 2014-04-18 20:07:50 UTC (rev 167501)
+++ trunk/Source/bmalloc/bmalloc/Heap.cpp 2014-04-18 20:17:59 UTC (rev 167502)
</span><span class="lines">@@ -32,6 +32,7 @@
</span><span class="cx"> #include "PerProcess.h"
</span><span class="cx"> #include "SmallChunk.h"
</span><span class="cx"> #include "XLargeChunk.h"
</span><ins>+#include "XSmallChunk.h"
</ins><span class="cx"> #include <thread>
</span><span class="cx">
</span><span class="cx"> namespace bmalloc {
</span><span class="lines">@@ -60,6 +61,7 @@
</span><span class="cx">
</span><span class="cx"> void Heap::scavenge(std::unique_lock<Mutex>& lock, std::chrono::milliseconds sleepDuration)
</span><span class="cx"> {
</span><ins>+ scavengeXSmallPages(lock, sleepDuration);
</ins><span class="cx"> scavengeSmallPages(lock, sleepDuration);
</span><span class="cx"> scavengeMediumPages(lock, sleepDuration);
</span><span class="cx"> scavengeLargeRanges(lock, sleepDuration);
</span><span class="lines">@@ -83,6 +85,22 @@
</span><span class="cx"> }
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+void Heap::scavengeXSmallPages(std::unique_lock<Mutex>& lock, std::chrono::milliseconds sleepDuration)
+{
+ while (1) {
+ if (m_isAllocatingPages) {
+ m_isAllocatingPages = false;
+
+ sleep(lock, sleepDuration);
+ continue;
+ }
+
+ if (!m_xSmallPages.size())
+ return;
+ m_vmHeap.deallocateXSmallPage(lock, m_xSmallPages.pop());
+ }
+}
+
</ins><span class="cx"> void Heap::scavengeMediumPages(std::unique_lock<Mutex>& lock, std::chrono::milliseconds sleepDuration)
</span><span class="cx"> {
</span><span class="cx"> while (1) {
</span><span class="lines">@@ -116,6 +134,27 @@
</span><span class="cx"> }
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+XSmallLine* Heap::allocateXSmallLineSlowCase(std::lock_guard<Mutex>& lock)
+{
+ m_isAllocatingPages = true;
+
+ XSmallPage* page = [this]() {
+ if (m_xSmallPages.size())
+ return m_xSmallPages.pop();
+
+ XSmallPage* page = m_vmHeap.allocateXSmallPage();
+ vmAllocatePhysicalPages(page->begin()->begin(), vmPageSize);
+ return page;
+ }();
+
+ XSmallLine* line = page->begin();
+ for (auto it = line + 1; it != page->end(); ++it)
+ m_xSmallLines.push(it);
+
+ page->ref(lock);
+ return line;
+}
+
</ins><span class="cx"> SmallLine* Heap::allocateSmallLineSlowCase(std::lock_guard<Mutex>& lock)
</span><span class="cx"> {
</span><span class="cx"> m_isAllocatingPages = true;
</span></span></pre></div>
<a id="trunkSourcebmallocbmallocHeaph"></a>
<div class="modfile"><h4>Modified: trunk/Source/bmalloc/bmalloc/Heap.h (167501 => 167502)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/bmalloc/bmalloc/Heap.h 2014-04-18 20:07:50 UTC (rev 167501)
+++ trunk/Source/bmalloc/bmalloc/Heap.h 2014-04-18 20:17:59 UTC (rev 167502)
</span><span class="lines">@@ -28,15 +28,12 @@
</span><span class="cx">
</span><span class="cx"> #include "FixedVector.h"
</span><span class="cx"> #include "VMHeap.h"
</span><del>-#include "MediumLine.h"
</del><span class="cx"> #include "Mutex.h"
</span><del>-#include "SmallPage.h"
</del><span class="cx"> #include "MediumChunk.h"
</span><del>-#include "MediumPage.h"
</del><span class="cx"> #include "SegregatedFreeList.h"
</span><span class="cx"> #include "SmallChunk.h"
</span><del>-#include "SmallLine.h"
</del><span class="cx"> #include "Vector.h"
</span><ins>+#include "XSmallChunk.h"
</ins><span class="cx"> #include <array>
</span><span class="cx"> #include <mutex>
</span><span class="cx">
</span><span class="lines">@@ -49,6 +46,9 @@
</span><span class="cx"> public:
</span><span class="cx"> Heap(std::lock_guard<Mutex>&);
</span><span class="cx">
</span><ins>+ XSmallLine* allocateXSmallLine(std::lock_guard<Mutex>&);
+ void deallocateXSmallLine(std::lock_guard<Mutex>&, XSmallLine*);
+
</ins><span class="cx"> SmallLine* allocateSmallLine(std::lock_guard<Mutex>&);
</span><span class="cx"> void deallocateSmallLine(std::lock_guard<Mutex>&, SmallLine*);
</span><span class="cx">
</span><span class="lines">@@ -66,6 +66,7 @@
</span><span class="cx"> private:
</span><span class="cx"> ~Heap() = delete;
</span><span class="cx">
</span><ins>+ XSmallLine* allocateXSmallLineSlowCase(std::lock_guard<Mutex>&);
</ins><span class="cx"> SmallLine* allocateSmallLineSlowCase(std::lock_guard<Mutex>&);
</span><span class="cx"> MediumLine* allocateMediumLineSlowCase(std::lock_guard<Mutex>&);
</span><span class="cx">
</span><span class="lines">@@ -78,13 +79,16 @@
</span><span class="cx"> void mergeLargeRight(EndTag*&, BeginTag*&, Range&, bool& hasPhysicalPages);
</span><span class="cx">
</span><span class="cx"> void concurrentScavenge();
</span><ins>+ void scavengeXSmallPages(std::unique_lock<Mutex>&, std::chrono::milliseconds);
</ins><span class="cx"> void scavengeSmallPages(std::unique_lock<Mutex>&, std::chrono::milliseconds);
</span><span class="cx"> void scavengeMediumPages(std::unique_lock<Mutex>&, std::chrono::milliseconds);
</span><span class="cx"> void scavengeLargeRanges(std::unique_lock<Mutex>&, std::chrono::milliseconds);
</span><span class="cx">
</span><ins>+ Vector<XSmallLine*> m_xSmallLines;
</ins><span class="cx"> Vector<SmallLine*> m_smallLines;
</span><span class="cx"> Vector<MediumLine*> m_mediumLines;
</span><span class="cx">
</span><ins>+ Vector<XSmallPage*> m_xSmallPages;
</ins><span class="cx"> Vector<SmallPage*> m_smallPages;
</span><span class="cx"> Vector<MediumPage*> m_mediumPages;
</span><span class="cx">
</span><span class="lines">@@ -96,6 +100,31 @@
</span><span class="cx"> AsyncTask<Heap, decltype(&Heap::concurrentScavenge)> m_scavenger;
</span><span class="cx"> };
</span><span class="cx">
</span><ins>+inline void Heap::deallocateXSmallLine(std::lock_guard<Mutex>& lock, XSmallLine* line)
+{
+ XSmallPage* page = XSmallPage::get(line);
+ if (page->deref(lock)) {
+ m_xSmallPages.push(page);
+ m_scavenger.run();
+ return;
+ }
+ m_xSmallLines.push(line);
+}
+
+inline XSmallLine* Heap::allocateXSmallLine(std::lock_guard<Mutex>& lock)
+{
+ while (m_xSmallLines.size()) {
+ XSmallLine* line = m_xSmallLines.pop();
+ XSmallPage* page = XSmallPage::get(line);
+ if (!page->refCount(lock)) // The line was promoted to the small pages list.
+ continue;
+ page->ref(lock);
+ return line;
+ }
+
+ return allocateXSmallLineSlowCase(lock);
+}
+
</ins><span class="cx"> inline void Heap::deallocateSmallLine(std::lock_guard<Mutex>& lock, SmallLine* line)
</span><span class="cx"> {
</span><span class="cx"> SmallPage* page = SmallPage::get(line);
</span></span></pre></div>
<a id="trunkSourcebmallocbmallocLargeChunkh"></a>
<div class="modfile"><h4>Modified: trunk/Source/bmalloc/bmalloc/LargeChunk.h (167501 => 167502)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/bmalloc/bmalloc/LargeChunk.h 2014-04-18 20:07:50 UTC (rev 167501)
+++ trunk/Source/bmalloc/bmalloc/LargeChunk.h 2014-04-18 20:17:59 UTC (rev 167502)
</span><span class="lines">@@ -76,7 +76,7 @@
</span><span class="cx">
</span><span class="cx"> inline LargeChunk* LargeChunk::get(void* object)
</span><span class="cx"> {
</span><del>- BASSERT(!isSmallOrMedium(object));
</del><ins>+ BASSERT(isLarge(object));
</ins><span class="cx"> return static_cast<LargeChunk*>(mask(object, largeChunkMask));
</span><span class="cx"> }
</span><span class="cx">
</span><span class="lines">@@ -89,7 +89,7 @@
</span><span class="cx">
</span><span class="cx"> inline EndTag* LargeChunk::endTag(void* object, size_t size)
</span><span class="cx"> {
</span><del>- BASSERT(!isSmallOrMedium(object));
</del><ins>+ BASSERT(isLarge(object));
</ins><span class="cx">
</span><span class="cx"> LargeChunk* chunk = get(object);
</span><span class="cx"> char* end = static_cast<char*>(object) + size;
</span></span></pre></div>
<a id="trunkSourcebmallocbmallocLineh"></a>
<div class="modfile"><h4>Modified: trunk/Source/bmalloc/bmalloc/Line.h (167501 => 167502)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/bmalloc/bmalloc/Line.h 2014-04-18 20:07:50 UTC (rev 167501)
+++ trunk/Source/bmalloc/bmalloc/Line.h 2014-04-18 20:17:59 UTC (rev 167502)
</span><span class="lines">@@ -58,7 +58,7 @@
</span><span class="cx"> template<class Traits>
</span><span class="cx"> inline auto Line<Traits>::get(void* object) -> Line*
</span><span class="cx"> {
</span><del>- BASSERT(isSmallOrMedium(object));
</del><ins>+ BASSERT(!isLarge(object));
</ins><span class="cx"> Chunk* chunk = Chunk::get(object);
</span><span class="cx"> size_t lineNumber = (reinterpret_cast<char*>(object) - reinterpret_cast<char*>(chunk)) / lineSize;
</span><span class="cx"> return &chunk->lines()[lineNumber];
</span></span></pre></div>
<a id="trunkSourcebmallocbmallocMediumAllocatorh"></a>
<div class="modfile"><h4>Modified: trunk/Source/bmalloc/bmalloc/MediumAllocator.h (167501 => 167502)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/bmalloc/bmalloc/MediumAllocator.h 2014-04-18 20:07:50 UTC (rev 167501)
+++ trunk/Source/bmalloc/bmalloc/MediumAllocator.h 2014-04-18 20:17:59 UTC (rev 167502)
</span><span class="lines">@@ -73,7 +73,7 @@
</span><span class="cx">
</span><span class="cx"> m_remaining -= size;
</span><span class="cx"> void* object = m_end - m_remaining - size;
</span><del>- BASSERT(isSmallOrMedium(object) && !isSmall(object));
</del><ins>+ BASSERT(objectType(object) == Medium);
</ins><span class="cx">
</span><span class="cx"> ++m_objectCount;
</span><span class="cx"> return object;
</span><span class="lines">@@ -99,6 +99,7 @@
</span><span class="cx"> m_end = line->end();
</span><span class="cx"> m_remaining = mediumLineSize;
</span><span class="cx"> m_objectCount = 0;
</span><ins>+ BASSERT(objectType(m_end - 1) == Medium);
</ins><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> } // namespace bmalloc
</span></span></pre></div>
<a id="trunkSourcebmallocbmallocObjectTypecpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/bmalloc/bmalloc/ObjectType.cpp (167501 => 167502)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/bmalloc/bmalloc/ObjectType.cpp 2014-04-18 20:07:50 UTC (rev 167501)
+++ trunk/Source/bmalloc/bmalloc/ObjectType.cpp 2014-04-18 20:17:59 UTC (rev 167502)
</span><span class="lines">@@ -30,16 +30,27 @@
</span><span class="cx">
</span><span class="cx"> ObjectType objectType(void* object)
</span><span class="cx"> {
</span><del>- if (isSmallOrMedium(object)) {
- if (isSmall(object))
- return Small;
</del><ins>+ switch (mask(reinterpret_cast<uintptr_t>(object), typeMask)) {
+ case xSmallType: {
+ return XSmall;
+ }
+ case smallType: {
+ return Small;
+ }
+ case mediumType: {
</ins><span class="cx"> return Medium;
</span><span class="cx"> }
</span><del>-
- BeginTag* beginTag = LargeChunk::beginTag(object);
- if (!beginTag->isXLarge())
- return Large;
- return XLarge;
</del><ins>+ case largeType: {
+ BeginTag* beginTag = LargeChunk::beginTag(object);
+ if (!beginTag->isXLarge())
+ return Large;
+ return XLarge;
+ }
+ default: {
+ RELEASE_BASSERT(false);
+ return XLarge;
+ }
+ }
</ins><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> } // namespace bmalloc
</span></span></pre></div>
<a id="trunkSourcebmallocbmallocObjectTypeh"></a>
<div class="modfile"><h4>Modified: trunk/Source/bmalloc/bmalloc/ObjectType.h (167501 => 167502)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/bmalloc/bmalloc/ObjectType.h 2014-04-18 20:07:50 UTC (rev 167501)
+++ trunk/Source/bmalloc/bmalloc/ObjectType.h 2014-04-18 20:17:59 UTC (rev 167502)
</span><span class="lines">@@ -31,21 +31,30 @@
</span><span class="cx">
</span><span class="cx"> namespace bmalloc {
</span><span class="cx">
</span><del>-enum ObjectType { Small, Medium, Large, XLarge };
</del><ins>+enum ObjectType { XSmall, Small, Medium, Large, XLarge };
</ins><span class="cx">
</span><span class="cx"> ObjectType objectType(void*);
</span><span class="cx">
</span><del>-inline bool isSmallOrMedium(void* object)
</del><ins>+inline bool isXSmall(void* object)
</ins><span class="cx"> {
</span><del>- return test(object, smallOrMediumTypeMask);
</del><ins>+ return mask(reinterpret_cast<uintptr_t>(object), typeMask) == xSmallType;
</ins><span class="cx"> }
</span><span class="cx">
</span><del>-inline bool isSmall(void* smallOrMedium)
</del><ins>+inline bool isSmall(void* object)
</ins><span class="cx"> {
</span><del>- BASSERT(isSmallOrMedium(smallOrMedium));
- return test(smallOrMedium, smallOrMediumSmallTypeMask);
</del><ins>+ return mask(reinterpret_cast<uintptr_t>(object), typeMask) == smallType;
</ins><span class="cx"> }
</span><span class="cx">
</span><ins>+inline bool isMedium(void* object)
+{
+ return mask(reinterpret_cast<uintptr_t>(object), typeMask) == mediumType;
+}
+
+inline bool isLarge(void* object)
+{
+ return mask(reinterpret_cast<uintptr_t>(object), typeMask) == largeType;
+}
+
</ins><span class="cx"> } // namespace bmalloc
</span><span class="cx">
</span><span class="cx"> #endif // ObjectType_h
</span></span></pre></div>
<a id="trunkSourcebmallocbmallocSegregatedFreeListh"></a>
<div class="modfile"><h4>Modified: trunk/Source/bmalloc/bmalloc/SegregatedFreeList.h (167501 => 167502)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/bmalloc/bmalloc/SegregatedFreeList.h 2014-04-18 20:07:50 UTC (rev 167501)
+++ trunk/Source/bmalloc/bmalloc/SegregatedFreeList.h 2014-04-18 20:17:59 UTC (rev 167502)
</span><span class="lines">@@ -58,7 +58,7 @@
</span><span class="cx"> Range take(List&, size_t);
</span><span class="cx"> Range takeGreedy(List&, size_t);
</span><span class="cx">
</span><del>- std::array<List, 19> m_lists;
</del><ins>+ std::array<List, 18> m_lists;
</ins><span class="cx"> };
</span><span class="cx">
</span><span class="cx"> } // namespace bmalloc
</span></span></pre></div>
<a id="trunkSourcebmallocbmallocSizesh"></a>
<div class="modfile"><h4>Modified: trunk/Source/bmalloc/bmalloc/Sizes.h (167501 => 167502)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/bmalloc/bmalloc/Sizes.h 2014-04-18 20:07:50 UTC (rev 167501)
+++ trunk/Source/bmalloc/bmalloc/Sizes.h 2014-04-18 20:17:59 UTC (rev 167502)
</span><span class="lines">@@ -47,44 +47,53 @@
</span><span class="cx">
</span><span class="cx"> static const size_t superChunkSize = 32 * MB;
</span><span class="cx">
</span><del>- static const size_t smallMax = 256;
- static const size_t smallLineSize = 512;
</del><ins>+ static const size_t xSmallMax = 64;
+ static const size_t xSmallLineSize = 256;
+ static const size_t xSmallLineMask = ~(xSmallLineSize - 1ul);
+
+ static const size_t xSmallChunkSize = superChunkSize / 4;
+ static const size_t xSmallChunkOffset = superChunkSize * 1 / 4;
+ static const size_t xSmallChunkMask = ~(xSmallChunkSize - 1ul);
+
+ static const size_t smallMax = 128;
+ static const size_t smallLineSize = 256;
</ins><span class="cx"> static const size_t smallLineMask = ~(smallLineSize - 1ul);
</span><span class="cx">
</span><span class="cx"> static const size_t smallChunkSize = superChunkSize / 4;
</span><del>- static const size_t smallChunkOffset = superChunkSize * 3 / 4;
</del><ins>+ static const size_t smallChunkOffset = superChunkSize * 2 / 4;
</ins><span class="cx"> static const size_t smallChunkMask = ~(smallChunkSize - 1ul);
</span><span class="cx">
</span><del>- static const size_t mediumMax = 1024;
- static const size_t mediumLineSize = 2048;
</del><ins>+ static const size_t mediumMax = 256;
+ static const size_t mediumLineSize = 512;
</ins><span class="cx"> static const size_t mediumLineMask = ~(mediumLineSize - 1ul);
</span><span class="cx">
</span><span class="cx"> static const size_t mediumChunkSize = superChunkSize / 4;
</span><del>- static const size_t mediumChunkOffset = superChunkSize * 2 / 4;
</del><ins>+ static const size_t mediumChunkOffset = superChunkSize * 3 / 4;
</ins><span class="cx"> static const size_t mediumChunkMask = ~(mediumChunkSize - 1ul);
</span><span class="cx">
</span><del>- static const size_t largeChunkSize = superChunkSize / 2;
- static const size_t largeChunkOffset = 0;
</del><ins>+ static const size_t largeChunkSize = superChunkSize / 4;
+ static const size_t largeChunkOffset = superChunkSize * 0 / 4;
</ins><span class="cx"> static const size_t largeChunkMask = ~(largeChunkSize - 1ul);
</span><span class="cx">
</span><span class="cx"> static const size_t largeAlignment = 64;
</span><span class="cx"> static const size_t largeMax = largeChunkSize * 99 / 100; // Plenty of room for metadata.
</span><del>- static const size_t largeMin = 1024;
</del><ins>+ static const size_t largeMin = mediumMax;
</ins><span class="cx">
</span><span class="cx"> static const size_t segregatedFreeListSearchDepth = 16;
</span><span class="cx">
</span><span class="cx"> static const uintptr_t typeMask = (superChunkSize - 1) & ~((superChunkSize / 4) - 1); // 4 taggable chunks
</span><ins>+ static const uintptr_t xSmallType = (superChunkSize + xSmallChunkOffset) & typeMask;
</ins><span class="cx"> static const uintptr_t smallType = (superChunkSize + smallChunkOffset) & typeMask;
</span><span class="cx"> static const uintptr_t mediumType = (superChunkSize + mediumChunkOffset) & typeMask;
</span><del>- static const uintptr_t largeTypeMask = ~(mediumType & smallType);
- static const uintptr_t smallOrMediumTypeMask = mediumType & smallType;
- static const uintptr_t smallOrMediumSmallTypeMask = smallType ^ mediumType; // Only valid if object is known to be small or medium.
-
</del><ins>+ static const uintptr_t largeType = (superChunkSize + largeChunkOffset) & typeMask;
+
</ins><span class="cx"> static const size_t deallocatorLogCapacity = 256;
</span><span class="cx">
</span><ins>+ static const size_t xSmallLineCacheCapacity = 32;
</ins><span class="cx"> static const size_t smallLineCacheCapacity = 16;
</span><span class="cx"> static const size_t mediumLineCacheCapacity = 8;
</span><span class="cx">
</span><ins>+ static const size_t xSmallAllocatorLogCapacity = 32;
</ins><span class="cx"> static const size_t smallAllocatorLogCapacity = 16;
</span><span class="cx"> static const size_t mediumAllocatorLogCapacity = 8;
</span><span class="cx">
</span></span></pre></div>
<a id="trunkSourcebmallocbmallocSmallAllocatorh"></a>
<div class="modfile"><h4>Modified: trunk/Source/bmalloc/bmalloc/SmallAllocator.h (167501 => 167502)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/bmalloc/bmalloc/SmallAllocator.h 2014-04-18 20:07:50 UTC (rev 167501)
+++ trunk/Source/bmalloc/bmalloc/SmallAllocator.h 2014-04-18 20:17:59 UTC (rev 167502)
</span><span class="lines">@@ -28,7 +28,6 @@
</span><span class="cx">
</span><span class="cx"> #include "BAssert.h"
</span><span class="cx"> #include "SmallChunk.h"
</span><del>-#include "SmallLine.h"
</del><span class="cx">
</span><span class="cx"> namespace bmalloc {
</span><span class="cx">
</span><span class="lines">@@ -85,7 +84,7 @@
</span><span class="cx"> --m_remaining;
</span><span class="cx"> char* result = m_ptr;
</span><span class="cx"> m_ptr += m_size;
</span><del>- BASSERT(isSmall(result));
</del><ins>+ BASSERT(objectType(result) == Small);
</ins><span class="cx"> return result;
</span><span class="cx"> }
</span><span class="cx">
</span></span></pre></div>
<a id="trunkSourcebmallocbmallocSmallTraitsh"></a>
<div class="modfile"><h4>Modified: trunk/Source/bmalloc/bmalloc/SmallTraits.h (167501 => 167502)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/bmalloc/bmalloc/SmallTraits.h 2014-04-18 20:07:50 UTC (rev 167501)
+++ trunk/Source/bmalloc/bmalloc/SmallTraits.h 2014-04-18 20:17:59 UTC (rev 167502)
</span><span class="lines">@@ -40,7 +40,7 @@
</span><span class="cx"> typedef Page<SmallTraits> Page;
</span><span class="cx">
</span><span class="cx"> static const size_t lineSize = smallLineSize;
</span><del>- static const size_t minimumObjectSize = alignment;
</del><ins>+ static const size_t minimumObjectSize = xSmallMax + alignment;
</ins><span class="cx"> static const size_t chunkSize = smallChunkSize;
</span><span class="cx"> static const size_t chunkOffset = smallChunkOffset;
</span><span class="cx"> static const uintptr_t chunkMask = smallChunkMask;
</span></span></pre></div>
<a id="trunkSourcebmallocbmallocVMHeapcpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/bmalloc/bmalloc/VMHeap.cpp (167501 => 167502)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/bmalloc/bmalloc/VMHeap.cpp 2014-04-18 20:07:50 UTC (rev 167501)
+++ trunk/Source/bmalloc/bmalloc/VMHeap.cpp 2014-04-18 20:17:59 UTC (rev 167502)
</span><span class="lines">@@ -36,6 +36,13 @@
</span><span class="cx"> {
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+void VMHeap::allocateXSmallChunk()
+{
+ XSmallChunk* chunk = XSmallChunk::create();
+ for (auto* it = chunk->begin(); it != chunk->end(); ++it)
+ m_xSmallPages.push(it);
+}
+
</ins><span class="cx"> void VMHeap::allocateSmallChunk()
</span><span class="cx"> {
</span><span class="cx"> SmallChunk* chunk = SmallChunk::create();
</span></span></pre></div>
<a id="trunkSourcebmallocbmallocVMHeaph"></a>
<div class="modfile"><h4>Modified: trunk/Source/bmalloc/bmalloc/VMHeap.h (167501 => 167502)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/bmalloc/bmalloc/VMHeap.h 2014-04-18 20:07:50 UTC (rev 167501)
+++ trunk/Source/bmalloc/bmalloc/VMHeap.h 2014-04-18 20:17:59 UTC (rev 167502)
</span><span class="lines">@@ -34,6 +34,7 @@
</span><span class="cx"> #include "SegregatedFreeList.h"
</span><span class="cx"> #include "SmallChunk.h"
</span><span class="cx"> #include "Vector.h"
</span><ins>+#include "XSmallChunk.h"
</ins><span class="cx">
</span><span class="cx"> namespace bmalloc {
</span><span class="cx">
</span><span class="lines">@@ -45,24 +46,36 @@
</span><span class="cx"> public:
</span><span class="cx"> VMHeap();
</span><span class="cx">
</span><ins>+ XSmallPage* allocateXSmallPage();
</ins><span class="cx"> SmallPage* allocateSmallPage();
</span><span class="cx"> MediumPage* allocateMediumPage();
</span><span class="cx"> Range allocateLargeRange(size_t);
</span><span class="cx">
</span><ins>+ void deallocateXSmallPage(std::unique_lock<Mutex>&, XSmallPage*);
</ins><span class="cx"> void deallocateSmallPage(std::unique_lock<Mutex>&, SmallPage*);
</span><span class="cx"> void deallocateMediumPage(std::unique_lock<Mutex>&, MediumPage*);
</span><span class="cx"> void deallocateLargeRange(std::unique_lock<Mutex>&, Range);
</span><span class="cx">
</span><span class="cx"> private:
</span><ins>+ void allocateXSmallChunk();
</ins><span class="cx"> void allocateSmallChunk();
</span><span class="cx"> void allocateMediumChunk();
</span><span class="cx"> Range allocateLargeChunk();
</span><span class="cx">
</span><ins>+ Vector<XSmallPage*> m_xSmallPages;
</ins><span class="cx"> Vector<SmallPage*> m_smallPages;
</span><span class="cx"> Vector<MediumPage*> m_mediumPages;
</span><span class="cx"> SegregatedFreeList m_largeRanges;
</span><span class="cx"> };
</span><span class="cx">
</span><ins>+inline XSmallPage* VMHeap::allocateXSmallPage()
+{
+ if (!m_xSmallPages.size())
+ allocateXSmallChunk();
+
+ return m_xSmallPages.pop();
+}
+
</ins><span class="cx"> inline SmallPage* VMHeap::allocateSmallPage()
</span><span class="cx"> {
</span><span class="cx"> if (!m_smallPages.size())
</span><span class="lines">@@ -87,6 +100,15 @@
</span><span class="cx"> return range;
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+inline void VMHeap::deallocateXSmallPage(std::unique_lock<Mutex>& lock, XSmallPage* page)
+{
+ lock.unlock();
+ vmDeallocatePhysicalPages(page->begin()->begin(), vmPageSize);
+ lock.lock();
+
+ m_xSmallPages.push(page);
+}
+
</ins><span class="cx"> inline void VMHeap::deallocateSmallPage(std::unique_lock<Mutex>& lock, SmallPage* page)
</span><span class="cx"> {
</span><span class="cx"> lock.unlock();
</span></span></pre></div>
<a id="trunkSourcebmallocbmallocXSmallAllocatorh"></a>
<div class="addfile"><h4>Added: trunk/Source/bmalloc/bmalloc/XSmallAllocator.h (0 => 167502)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/bmalloc/bmalloc/XSmallAllocator.h (rev 0)
+++ trunk/Source/bmalloc/bmalloc/XSmallAllocator.h 2014-04-18 20:17:59 UTC (rev 167502)
</span><span class="lines">@@ -0,0 +1,111 @@
</span><ins>+/*
+ * Copyright (C) 2014 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+ * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+ * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+ * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef XSmallAllocator_h
+#define XSmallAllocator_h
+
+#include "BAssert.h"
+#include "XSmallChunk.h"
+
+namespace bmalloc {
+
+// Helper object for allocating XSmall objects.
+
+class XSmallAllocator {
+public:
+ XSmallAllocator();
+ XSmallAllocator(size_t);
+
+ bool isNull() { return !m_ptr; }
+ XSmallLine* line();
+
+ bool canAllocate() { return !!m_remaining; }
+ void* allocate();
+
+ unsigned short objectCount();
+ unsigned char derefCount();
+ void refill(XSmallLine*);
+
+private:
+ char* m_ptr;
+ unsigned short m_size;
+ unsigned char m_remaining;
+ unsigned char m_maxObjectCount;
+};
+
+inline XSmallAllocator::XSmallAllocator()
+ : m_ptr()
+ , m_size()
+ , m_remaining()
+ , m_maxObjectCount()
+{
+}
+
+inline XSmallAllocator::XSmallAllocator(size_t size)
+ : m_ptr()
+ , m_size(size)
+ , m_remaining()
+ , m_maxObjectCount(xSmallLineSize / size)
+{
+}
+
+inline XSmallLine* XSmallAllocator::line()
+{
+ return XSmallLine::get(canAllocate() ? m_ptr : m_ptr - 1);
+}
+
+inline void* XSmallAllocator::allocate()
+{
+ BASSERT(m_remaining);
+ BASSERT(m_size >= XSmallLine::minimumObjectSize);
+
+ --m_remaining;
+ char* result = m_ptr;
+ m_ptr += m_size;
+ BASSERT(objectType(result) == XSmall);
+ return result;
+}
+
+inline unsigned short XSmallAllocator::objectCount()
+{
+ return m_maxObjectCount - m_remaining;
+}
+
+inline unsigned char XSmallAllocator::derefCount()
+{
+ return XSmallLine::maxRefCount - objectCount();
+}
+
+inline void XSmallAllocator::refill(XSmallLine* line)
+{
+ BASSERT(!canAllocate());
+ line->concurrentRef(XSmallLine::maxRefCount);
+ m_ptr = line->begin();
+ m_remaining = m_maxObjectCount;
+}
+
+} // namespace bmalloc
+
+#endif // XSmallAllocator_h
</ins></span></pre></div>
<a id="trunkSourcebmallocbmallocXSmallChunkh"></a>
<div class="addfile"><h4>Added: trunk/Source/bmalloc/bmalloc/XSmallChunk.h (0 => 167502)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/bmalloc/bmalloc/XSmallChunk.h (rev 0)
+++ trunk/Source/bmalloc/bmalloc/XSmallChunk.h 2014-04-18 20:17:59 UTC (rev 167502)
</span><span class="lines">@@ -0,0 +1,40 @@
</span><ins>+/*
+ * Copyright (C) 2014 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+ * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+ * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+ * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef XSmallChunk_h
+#define XSmallChunk_h
+
+#include "Chunk.h"
+#include "XSmallLine.h"
+#include "XSmallPage.h"
+#include "XSmallTraits.h"
+
+namespace bmalloc {
+
+typedef Chunk<XSmallTraits> XSmallChunk;
+
+}; // namespace bmalloc
+
+#endif // XSmallChunk
</ins></span></pre></div>
<a id="trunkSourcebmallocbmallocXSmallLineh"></a>
<div class="addfile"><h4>Added: trunk/Source/bmalloc/bmalloc/XSmallLine.h (0 => 167502)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/bmalloc/bmalloc/XSmallLine.h (rev 0)
+++ trunk/Source/bmalloc/bmalloc/XSmallLine.h 2014-04-18 20:17:59 UTC (rev 167502)
</span><span class="lines">@@ -0,0 +1,38 @@
</span><ins>+/*
+ * Copyright (C) 2014 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+ * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+ * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+ * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef XSmallLine_h
+#define XSmallLine_h
+
+#include "Line.h"
+#include "XSmallTraits.h"
+
+namespace bmalloc {
+
+typedef Line<XSmallTraits> XSmallLine;
+
+} // namespace bmalloc
+
+#endif // XSmallLine_h
</ins></span></pre></div>
<a id="trunkSourcebmallocbmallocXSmallPageh"></a>
<div class="addfile"><h4>Added: trunk/Source/bmalloc/bmalloc/XSmallPage.h (0 => 167502)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/bmalloc/bmalloc/XSmallPage.h (rev 0)
+++ trunk/Source/bmalloc/bmalloc/XSmallPage.h 2014-04-18 20:17:59 UTC (rev 167502)
</span><span class="lines">@@ -0,0 +1,38 @@
</span><ins>+/*
+ * Copyright (C) 2014 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+ * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+ * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+ * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef XSmallPage_h
+#define XSmallPage_h
+
+#include "Page.h"
+#include "XSmallTraits.h"
+
+namespace bmalloc {
+
+typedef Page<XSmallTraits> XSmallPage;
+
+} // namespace bmalloc
+
+#endif // XSmallPage_h
</ins></span></pre></div>
<a id="trunkSourcebmallocbmallocXSmallTraitsh"></a>
<div class="addfile"><h4>Added: trunk/Source/bmalloc/bmalloc/XSmallTraits.h (0 => 167502)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/bmalloc/bmalloc/XSmallTraits.h (rev 0)
+++ trunk/Source/bmalloc/bmalloc/XSmallTraits.h 2014-04-18 20:17:59 UTC (rev 167502)
</span><span class="lines">@@ -0,0 +1,51 @@
</span><ins>+/*
+ * Copyright (C) 2014 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+ * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+ * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+ * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef XSmallTraits_h
+#define XSmallTraits_h
+
+#include "Sizes.h"
+
+namespace bmalloc {
+
+template<class Traits> class Chunk;
+template<class Traits> class Line;
+template<class Traits> class Page;
+
+struct XSmallTraits {
+ typedef Chunk<XSmallTraits> Chunk;
+ typedef Line<XSmallTraits> Line;
+ typedef Page<XSmallTraits> Page;
+
+ static const size_t lineSize = xSmallLineSize;
+ static const size_t minimumObjectSize = alignment;
+ static const size_t chunkSize = xSmallChunkSize;
+ static const size_t chunkOffset = xSmallChunkOffset;
+ static const uintptr_t chunkMask = xSmallChunkMask;
+};
+
+} // namespace bmalloc
+
+#endif // XSmallTraits_h
</ins></span></pre></div>
<a id="trunkSourcebmallocbmallocbmalloch"></a>
<div class="modfile"><h4>Modified: trunk/Source/bmalloc/bmalloc/bmalloc.h (167501 => 167502)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/bmalloc/bmalloc/bmalloc.h 2014-04-18 20:07:50 UTC (rev 167501)
+++ trunk/Source/bmalloc/bmalloc/bmalloc.h 2014-04-18 20:17:59 UTC (rev 167502)
</span><span class="lines">@@ -50,7 +50,13 @@
</span><span class="cx"> return result;
</span><span class="cx">
</span><span class="cx"> size_t oldSize = 0;
</span><del>- switch(objectType(object)) {
</del><ins>+ switch (objectType(object)) {
+ case XSmall: {
+ // We don't have an exact size, but we can calculate a maximum.
+ void* end = roundUpToMultipleOf<xSmallLineSize>(static_cast<char*>(object) + 1);
+ oldSize = static_cast<char*>(end) - static_cast<char*>(object);
+ break;
+ }
</ins><span class="cx"> case Small: {
</span><span class="cx"> // We don't have an exact size, but we can calculate a maximum.
</span><span class="cx"> void* end = roundUpToMultipleOf<smallLineSize>(static_cast<char*>(object) + 1);
</span></span></pre></div>
<a id="trunkSourcebmallocbmallocxcodeprojprojectpbxproj"></a>
<div class="modfile"><h4>Modified: trunk/Source/bmalloc/bmalloc.xcodeproj/project.pbxproj (167501 => 167502)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/bmalloc/bmalloc.xcodeproj/project.pbxproj 2014-04-18 20:07:50 UTC (rev 167501)
+++ trunk/Source/bmalloc/bmalloc.xcodeproj/project.pbxproj 2014-04-18 20:17:59 UTC (rev 167502)
</span><span class="lines">@@ -11,6 +11,11 @@
</span><span class="cx"> 1400274A18F89C2300115C97 /* VMHeap.h in Headers */ = {isa = PBXBuildFile; fileRef = 144F7BFC18BFC517003537F3 /* VMHeap.h */; settings = {ATTRIBUTES = (Private, ); }; };
</span><span class="cx"> 1400274B18F89C3D00115C97 /* BoundaryTagInlines.h in Headers */ = {isa = PBXBuildFile; fileRef = 14105E7B18DBD7AF003A106E /* BoundaryTagInlines.h */; settings = {ATTRIBUTES = (Private, ); }; };
</span><span class="cx"> 1400274C18F89C3D00115C97 /* SegregatedFreeList.h in Headers */ = {isa = PBXBuildFile; fileRef = 146BEE1E18C841C50002D5A2 /* SegregatedFreeList.h */; settings = {ATTRIBUTES = (Private, ); }; };
</span><ins>+ 142FCC78190080B8009032D4 /* XSmallChunk.h in Headers */ = {isa = PBXBuildFile; fileRef = 142FCC74190080B8009032D4 /* XSmallChunk.h */; };
+ 142FCC79190080B8009032D4 /* XSmallLine.h in Headers */ = {isa = PBXBuildFile; fileRef = 142FCC75190080B8009032D4 /* XSmallLine.h */; };
+ 142FCC7A190080B8009032D4 /* XSmallPage.h in Headers */ = {isa = PBXBuildFile; fileRef = 142FCC76190080B8009032D4 /* XSmallPage.h */; };
+ 142FCC7B190080B8009032D4 /* XSmallTraits.h in Headers */ = {isa = PBXBuildFile; fileRef = 142FCC77190080B8009032D4 /* XSmallTraits.h */; };
+ 142FCC7D1900815E009032D4 /* XSmallAllocator.h in Headers */ = {isa = PBXBuildFile; fileRef = 142FCC7C1900815E009032D4 /* XSmallAllocator.h */; };
</ins><span class="cx"> 1448C30018F3754600502839 /* mbmalloc.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 1448C2FF18F3754300502839 /* mbmalloc.cpp */; };
</span><span class="cx"> 1448C30118F3754C00502839 /* bmalloc.h in Headers */ = {isa = PBXBuildFile; fileRef = 1448C2FE18F3754300502839 /* bmalloc.h */; settings = {ATTRIBUTES = (Private, ); }; };
</span><span class="cx"> 14C919C918FCC59F0028DB43 /* BPlatform.h in Headers */ = {isa = PBXBuildFile; fileRef = 14C919C818FCC59F0028DB43 /* BPlatform.h */; };
</span><span class="lines">@@ -82,6 +87,11 @@
</span><span class="cx"> 1417F64F18B7280C0076FA3F /* Syscall.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = Syscall.h; path = bmalloc/Syscall.h; sourceTree = "<group>"; };
</span><span class="cx"> 1417F65218BA88A00076FA3F /* AsyncTask.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = AsyncTask.h; path = bmalloc/AsyncTask.h; sourceTree = "<group>"; };
</span><span class="cx"> 1421A87718EE462A00B4DD68 /* Algorithm.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; name = Algorithm.h; path = bmalloc/Algorithm.h; sourceTree = "<group>"; };
</span><ins>+ 142FCC74190080B8009032D4 /* XSmallChunk.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = XSmallChunk.h; path = bmalloc/XSmallChunk.h; sourceTree = "<group>"; };
+ 142FCC75190080B8009032D4 /* XSmallLine.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = XSmallLine.h; path = bmalloc/XSmallLine.h; sourceTree = "<group>"; };
+ 142FCC76190080B8009032D4 /* XSmallPage.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = XSmallPage.h; path = bmalloc/XSmallPage.h; sourceTree = "<group>"; };
+ 142FCC77190080B8009032D4 /* XSmallTraits.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = XSmallTraits.h; path = bmalloc/XSmallTraits.h; sourceTree = "<group>"; };
+ 142FCC7C1900815E009032D4 /* XSmallAllocator.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = XSmallAllocator.h; path = bmalloc/XSmallAllocator.h; sourceTree = "<group>"; };
</ins><span class="cx"> 143E29E918CAE8BE00FE8A0F /* MediumPage.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = MediumPage.h; path = bmalloc/MediumPage.h; sourceTree = "<group>"; };
</span><span class="cx"> 143E29ED18CAE90500FE8A0F /* SmallPage.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = SmallPage.h; path = bmalloc/SmallPage.h; sourceTree = "<group>"; };
</span><span class="cx"> 144469E417A46BFE00F9EA1D /* Cache.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; lineEnding = 0; name = Cache.cpp; path = bmalloc/Cache.cpp; sourceTree = "<group>"; xcLanguageSpecificationIdentifier = xcode.lang.cpp; };
</span><span class="lines">@@ -195,6 +205,10 @@
</span><span class="cx"> 1452478618BC757C00F80098 /* SmallLine.h */,
</span><span class="cx"> 143E29ED18CAE90500FE8A0F /* SmallPage.h */,
</span><span class="cx"> 147AAA9718CE5FB6002201E4 /* SmallTraits.h */,
</span><ins>+ 142FCC74190080B8009032D4 /* XSmallChunk.h */,
+ 142FCC75190080B8009032D4 /* XSmallLine.h */,
+ 142FCC76190080B8009032D4 /* XSmallPage.h */,
+ 142FCC77190080B8009032D4 /* XSmallTraits.h */,
</ins><span class="cx"> );
</span><span class="cx"> name = "heap: small | medium";
</span><span class="cx"> sourceTree = "<group>";
</span><span class="lines">@@ -237,6 +251,7 @@
</span><span class="cx"> 145F685A179DC90200D65598 /* Deallocator.h */,
</span><span class="cx"> 1413E47018A0661700546D68 /* MediumAllocator.h */,
</span><span class="cx"> 1413E462189DE1CD00546D68 /* SmallAllocator.h */,
</span><ins>+ 142FCC7C1900815E009032D4 /* XSmallAllocator.h */,
</ins><span class="cx"> );
</span><span class="cx"> name = cache;
</span><span class="cx"> sourceTree = "<group>";
</span><span class="lines">@@ -290,6 +305,7 @@
</span><span class="cx"> isa = PBXHeadersBuildPhase;
</span><span class="cx"> buildActionMask = 2147483647;
</span><span class="cx"> files = (
</span><ins>+ 142FCC7B190080B8009032D4 /* XSmallTraits.h in Headers */,
</ins><span class="cx"> 14DD78B518F48D6B00950702 /* Line.h in Headers */,
</span><span class="cx"> 14DD78CF18F48D7500950702 /* Vector.h in Headers */,
</span><span class="cx"> 14C919C918FCC59F0028DB43 /* BPlatform.h in Headers */,
</span><span class="lines">@@ -298,6 +314,7 @@
</span><span class="cx"> 14DD78CC18F48D7500950702 /* PerThread.h in Headers */,
</span><span class="cx"> 14DD78B418F48D6B00950702 /* Chunk.h in Headers */,
</span><span class="cx"> 14DD78CA18F48D7500950702 /* Mutex.h in Headers */,
</span><ins>+ 142FCC7D1900815E009032D4 /* XSmallAllocator.h in Headers */,
</ins><span class="cx"> 14DD78D118F48EC600950702 /* XLargeChunk.h in Headers */,
</span><span class="cx"> 14DD78B918F48D6B00950702 /* MediumTraits.h in Headers */,
</span><span class="cx"> 1448C30118F3754C00502839 /* bmalloc.h in Headers */,
</span><span class="lines">@@ -318,11 +335,14 @@
</span><span class="cx"> 14DD78BB18F48D6B00950702 /* SmallChunk.h in Headers */,
</span><span class="cx"> 14DD78C918F48D7500950702 /* Inline.h in Headers */,
</span><span class="cx"> 1400274A18F89C2300115C97 /* VMHeap.h in Headers */,
</span><ins>+ 142FCC7A190080B8009032D4 /* XSmallPage.h in Headers */,
</ins><span class="cx"> 1400274918F89C1300115C97 /* Heap.h in Headers */,
</span><span class="cx"> 14DD78B818F48D6B00950702 /* MediumPage.h in Headers */,
</span><span class="cx"> 14DD78C518F48D7500950702 /* Algorithm.h in Headers */,
</span><span class="cx"> 14DD78BD18F48D6B00950702 /* SmallPage.h in Headers */,
</span><span class="cx"> 1400274B18F89C3D00115C97 /* BoundaryTagInlines.h in Headers */,
</span><ins>+ 142FCC79190080B8009032D4 /* XSmallLine.h in Headers */,
+ 142FCC78190080B8009032D4 /* XSmallChunk.h in Headers */,
</ins><span class="cx"> 14DD788E18F48CCD00950702 /* BoundaryTag.h in Headers */,
</span><span class="cx"> 14DD78C818F48D7500950702 /* FixedVector.h in Headers */,
</span><span class="cx"> 14DD78B718F48D6B00950702 /* MediumLine.h in Headers */,
</span></span></pre>
</div>
</div>
</body>
</html>