<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
"http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head><meta http-equiv="content-type" content="text/html; charset=utf-8" />
<title>[198571] trunk/Source/bmalloc</title>
</head>
<body>
<style type="text/css"><!--
#msg dl.meta { border: 1px #006 solid; background: #369; padding: 6px; color: #fff; }
#msg dl.meta dt { float: left; width: 6em; font-weight: bold; }
#msg dt:after { content:':';}
#msg dl, #msg dt, #msg ul, #msg li, #header, #footer, #logmsg { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt; }
#msg dl a { font-weight: bold}
#msg dl a:link { color:#fc3; }
#msg dl a:active { color:#ff0; }
#msg dl a:visited { color:#cc6; }
h3 { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt; font-weight: bold; }
#msg pre { overflow: auto; background: #ffc; border: 1px #fa0 solid; padding: 6px; }
#logmsg { background: #ffc; border: 1px #fa0 solid; padding: 1em 1em 0 1em; }
#logmsg p, #logmsg pre, #logmsg blockquote { margin: 0 0 1em 0; }
#logmsg p, #logmsg li, #logmsg dt, #logmsg dd { line-height: 14pt; }
#logmsg h1, #logmsg h2, #logmsg h3, #logmsg h4, #logmsg h5, #logmsg h6 { margin: .5em 0; }
#logmsg h1:first-child, #logmsg h2:first-child, #logmsg h3:first-child, #logmsg h4:first-child, #logmsg h5:first-child, #logmsg h6:first-child { margin-top: 0; }
#logmsg ul, #logmsg ol { padding: 0; list-style-position: inside; margin: 0 0 0 1em; }
#logmsg ul { text-indent: -1em; padding-left: 1em; }#logmsg ol { text-indent: -1.5em; padding-left: 1.5em; }
#logmsg > ul, #logmsg > ol { margin: 0 0 1em 0; }
#logmsg pre { background: #eee; padding: 1em; }
#logmsg blockquote { border: 1px solid #fa0; border-left-width: 10px; padding: 1em 1em 0 1em; background: white;}
#logmsg dl { margin: 0; }
#logmsg dt { font-weight: bold; }
#logmsg dd { margin: 0; padding: 0 0 0.5em 0; }
#logmsg dd:before { content:'\00bb';}
#logmsg table { border-spacing: 0px; border-collapse: collapse; border-top: 4px solid #fa0; border-bottom: 1px solid #fa0; background: #fff; }
#logmsg table th { text-align: left; font-weight: normal; padding: 0.2em 0.5em; border-top: 1px dotted #fa0; }
#logmsg table td { text-align: right; border-top: 1px dotted #fa0; padding: 0.2em 0.5em; }
#logmsg table thead th { text-align: center; border-bottom: 1px solid #fa0; }
#logmsg table th.Corner { text-align: left; }
#logmsg hr { border: none 0; border-top: 2px dashed #fa0; height: 1px; }
#header, #footer { color: #fff; background: #636; border: 1px #300 solid; padding: 6px; }
#patch { width: 100%; }
#patch h4 {font-family: verdana,arial,helvetica,sans-serif;font-size:10pt;padding:8px;background:#369;color:#fff;margin:0;}
#patch .propset h4, #patch .binary h4 {margin:0;}
#patch pre {padding:0;line-height:1.2em;margin:0;}
#patch .diff {width:100%;background:#eee;padding: 0 0 10px 0;overflow:auto;}
#patch .propset .diff, #patch .binary .diff {padding:10px 0;}
#patch span {display:block;padding:0 10px;}
#patch .modfile, #patch .addfile, #patch .delfile, #patch .propset, #patch .binary, #patch .copfile {border:1px solid #ccc;margin:10px 0;}
#patch ins {background:#dfd;text-decoration:none;display:block;padding:0 10px;}
#patch del {background:#fdd;text-decoration:none;display:block;padding:0 10px;}
#patch .lines, .info {color:#888;background:#fff;}
--></style>
<div id="msg">
<dl class="meta">
<dt>Revision</dt> <dd><a href="http://trac.webkit.org/projects/webkit/changeset/198571">198571</a></dd>
<dt>Author</dt> <dd>ggaren@apple.com</dd>
<dt>Date</dt> <dd>2016-03-22 18:39:36 -0700 (Tue, 22 Mar 2016)</dd>
</dl>
<h3>Log Message</h3>
<pre>bmalloc: use a log scale for large-ish size classes
https://bugs.webkit.org/show_bug.cgi?id=155770
Reviewed by Michael Saboff.
At larger sizes, precise allocation sizes don't save much memory -- and
they can cost memory when objects of distinct size classes can't
allocate together.
This is a small savings up to our current allocation limits, and it may
enable changing those limits in the long term.
* bmalloc/Algorithm.h:
(bmalloc::log2): We use this to compute large-ish size classes.
* bmalloc/Allocator.cpp:
(bmalloc::Allocator::Allocator): Iterate by size class instead of by
object size so we can change object size limits without breaking stuff.
(bmalloc::Allocator::scavenge): Ditto.
(bmalloc::Allocator::allocateLogSizeClass): New helper function for
allocating based on log size classes.
(bmalloc::Allocator::allocateSlowCase): Account for extra size class
possibilities.
* bmalloc/Allocator.h:
(bmalloc::Allocator::allocateFastCase): We only handle up to 512b on
the fastest fast path now.
* bmalloc/BumpAllocator.h:
(bmalloc::BumpAllocator::validate): Deleted. I noticed that this function
had been refactored not to do anything anymore.
* bmalloc/Heap.cpp:
(bmalloc::Heap::initializeLineMetadata): Iterate by size class. (See
Allocator::Allocator.)
* bmalloc/Heap.h: Use the sizeClassCount constant instead of hard coding
things.
* bmalloc/Sizes.h:
(bmalloc::Sizes::maskSizeClass):
(bmalloc::Sizes::maskObjectSize):
(bmalloc::Sizes::logSizeClass):
(bmalloc::Sizes::logObjectSize):
(bmalloc::Sizes::sizeClass):
(bmalloc::Sizes::objectSize): Separate size class calculation between
simple size classes that can be computed with a mask and are 8-byte-precise
and complex size classes that require more math and are less precise.
* bmalloc/SmallLine.h:
(bmalloc::SmallLine::ref):
* bmalloc/SmallPage.h:
(bmalloc::SmallPage::SmallPage):
(bmalloc::SmallPage::ref):
(bmalloc::SmallPage::deref): Cleaned up some ASSERTs that triggered
while working on this patch.
* bmalloc/Zone.cpp:
(bmalloc::statistics):
(bmalloc::zoneSize):
(bmalloc::Zone::Zone):
(bmalloc::size): Deleted. Renamed these symbols to work around an lldb
bug that makes it impossible to print out variables named 'size' -- which
can be a problem when working on malloc.</pre>
<h3>Modified Paths</h3>
<ul>
<li><a href="#trunkSourcebmallocChangeLog">trunk/Source/bmalloc/ChangeLog</a></li>
<li><a href="#trunkSourcebmallocbmallocAlgorithmh">trunk/Source/bmalloc/bmalloc/Algorithm.h</a></li>
<li><a href="#trunkSourcebmallocbmallocAllocatorcpp">trunk/Source/bmalloc/bmalloc/Allocator.cpp</a></li>
<li><a href="#trunkSourcebmallocbmallocAllocatorh">trunk/Source/bmalloc/bmalloc/Allocator.h</a></li>
<li><a href="#trunkSourcebmallocbmallocBumpAllocatorh">trunk/Source/bmalloc/bmalloc/BumpAllocator.h</a></li>
<li><a href="#trunkSourcebmallocbmallocHeapcpp">trunk/Source/bmalloc/bmalloc/Heap.cpp</a></li>
<li><a href="#trunkSourcebmallocbmallocHeaph">trunk/Source/bmalloc/bmalloc/Heap.h</a></li>
<li><a href="#trunkSourcebmallocbmallocSizesh">trunk/Source/bmalloc/bmalloc/Sizes.h</a></li>
<li><a href="#trunkSourcebmallocbmallocSmallLineh">trunk/Source/bmalloc/bmalloc/SmallLine.h</a></li>
<li><a href="#trunkSourcebmallocbmallocSmallPageh">trunk/Source/bmalloc/bmalloc/SmallPage.h</a></li>
<li><a href="#trunkSourcebmallocbmallocZonecpp">trunk/Source/bmalloc/bmalloc/Zone.cpp</a></li>
</ul>
</div>
<div id="patch">
<h3>Diff</h3>
<a id="trunkSourcebmallocChangeLog"></a>
<div class="modfile"><h4>Modified: trunk/Source/bmalloc/ChangeLog (198570 => 198571)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/bmalloc/ChangeLog 2016-03-23 01:38:49 UTC (rev 198570)
+++ trunk/Source/bmalloc/ChangeLog 2016-03-23 01:39:36 UTC (rev 198571)
</span><span class="lines">@@ -1,5 +1,75 @@
</span><span class="cx"> 2016-03-22 Geoffrey Garen <ggaren@apple.com>
</span><span class="cx">
</span><ins>+ bmalloc: use a log scale for large-ish size classes
+ https://bugs.webkit.org/show_bug.cgi?id=155770
+
+ Reviewed by Michael Saboff.
+
+ At larger sizes, precise allocation sizes don't save much memory -- and
+ they can cost memory when objects of distinct size classes can't
+ allocate together.
+
+ This is a small savings up to our current allocation limits, and it may
+ enable changing those limits in the long term.
+
+ * bmalloc/Algorithm.h:
+ (bmalloc::log2): We use this to compute large-ish size classes.
+
+ * bmalloc/Allocator.cpp:
+ (bmalloc::Allocator::Allocator): Iterate by size class instead of by
+ object size so we can change object size limits without breaking stuff.
+
+ (bmalloc::Allocator::scavenge): Ditto.
+
+ (bmalloc::Allocator::allocateLogSizeClass): New helper function for
+ allocating based on log size classes.
+
+ (bmalloc::Allocator::allocateSlowCase): Account for extra size class
+ possibilities.
+
+ * bmalloc/Allocator.h:
+ (bmalloc::Allocator::allocateFastCase): We only handle up to 512b on
+ the fastest fast path now.
+
+ * bmalloc/BumpAllocator.h:
+ (bmalloc::BumpAllocator::validate): Deleted. I noticed that this function
+ had been refactored not to do anything anymore.
+
+ * bmalloc/Heap.cpp:
+ (bmalloc::Heap::initializeLineMetadata): Iterate by size class. (See
+ Allocator::Allocator.)
+
+ * bmalloc/Heap.h: Use the sizeClassCount constant instead of hard coding
+ things.
+
+ * bmalloc/Sizes.h:
+ (bmalloc::Sizes::maskSizeClass):
+ (bmalloc::Sizes::maskObjectSize):
+ (bmalloc::Sizes::logSizeClass):
+ (bmalloc::Sizes::logObjectSize):
+ (bmalloc::Sizes::sizeClass):
+ (bmalloc::Sizes::objectSize): Separate size class calculation between
+ simple size classes that can be computed with a mask and are 8-byte-precise
+ and complex size classes that require more math and are less precise.
+
+ * bmalloc/SmallLine.h:
+ (bmalloc::SmallLine::ref):
+ * bmalloc/SmallPage.h:
+ (bmalloc::SmallPage::SmallPage):
+ (bmalloc::SmallPage::ref):
+ (bmalloc::SmallPage::deref): Cleaned up some ASSERTs that triggered
+ while working on this patch.
+
+ * bmalloc/Zone.cpp:
+ (bmalloc::statistics):
+ (bmalloc::zoneSize):
+ (bmalloc::Zone::Zone):
+ (bmalloc::size): Deleted. Renamed these symbols to work around an lldb
+ bug that makes it impossible to print out variables named 'size' -- which
+ can be a problem when working on malloc.
+
+2016-03-22 Geoffrey Garen <ggaren@apple.com>
+
</ins><span class="cx"> bmalloc: shrink largeMax
</span><span class="cx"> https://bugs.webkit.org/show_bug.cgi?id=155759
</span><span class="cx">
</span></span></pre></div>
<a id="trunkSourcebmallocbmallocAlgorithmh"></a>
<div class="modfile"><h4>Modified: trunk/Source/bmalloc/bmalloc/Algorithm.h (198570 => 198571)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/bmalloc/bmalloc/Algorithm.h 2016-03-23 01:38:49 UTC (rev 198570)
+++ trunk/Source/bmalloc/bmalloc/Algorithm.h 2016-03-23 01:39:36 UTC (rev 198571)
</span><span class="lines">@@ -108,6 +108,11 @@
</span><span class="cx"> return sizeof(T) * 8;
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+inline constexpr unsigned long log2(unsigned long value)
+{
+ return bitCount<unsigned long>() - 1 - __builtin_clzl(value);
+}
+
</ins><span class="cx"> } // namespace bmalloc
</span><span class="cx">
</span><span class="cx"> #endif // Algorithm_h
</span></span></pre></div>
<a id="trunkSourcebmallocbmallocAllocatorcpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/bmalloc/bmalloc/Allocator.cpp (198570 => 198571)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/bmalloc/bmalloc/Allocator.cpp 2016-03-23 01:38:49 UTC (rev 198570)
+++ trunk/Source/bmalloc/bmalloc/Allocator.cpp 2016-03-23 01:39:36 UTC (rev 198571)
</span><span class="lines">@@ -42,8 +42,8 @@
</span><span class="cx"> : m_isBmallocEnabled(heap->environment().isBmallocEnabled())
</span><span class="cx"> , m_deallocator(deallocator)
</span><span class="cx"> {
</span><del>- for (unsigned short size = alignment; size <= smallMax; size += alignment)
- m_bumpAllocators[sizeClass(size)].init(size);
</del><ins>+ for (size_t sizeClass = 0; sizeClass < sizeClassCount; ++sizeClass)
+ m_bumpAllocators[sizeClass].init(objectSize(sizeClass));
</ins><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> Allocator::~Allocator()
</span><span class="lines">@@ -164,9 +164,9 @@
</span><span class="cx">
</span><span class="cx"> void Allocator::scavenge()
</span><span class="cx"> {
</span><del>- for (unsigned short i = alignment; i <= smallMax; i += alignment) {
- BumpAllocator& allocator = m_bumpAllocators[sizeClass(i)];
- BumpRangeCache& bumpRangeCache = m_bumpRangeCaches[sizeClass(i)];
</del><ins>+ for (size_t sizeClass = 0; sizeClass < sizeClassCount; ++sizeClass) {
+ BumpAllocator& allocator = m_bumpAllocators[sizeClass];
+ BumpRangeCache& bumpRangeCache = m_bumpRangeCaches[sizeClass];
</ins><span class="cx">
</span><span class="cx"> while (allocator.canAllocate())
</span><span class="cx"> m_deallocator.deallocate(allocator.allocate());
</span><span class="lines">@@ -210,18 +210,30 @@
</span><span class="cx"> return PerProcess<Heap>::getFastCase()->allocateXLarge(lock, size);
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+NO_INLINE void* Allocator::allocateLogSizeClass(size_t size)
+{
+ size_t sizeClass = bmalloc::sizeClass(size);
+ BumpAllocator& allocator = m_bumpAllocators[sizeClass];
+ if (!allocator.canAllocate())
+ refillAllocator(allocator, sizeClass);
+ return allocator.allocate();
+}
+
</ins><span class="cx"> void* Allocator::allocateSlowCase(size_t size)
</span><span class="cx"> {
</span><span class="cx"> if (!m_isBmallocEnabled)
</span><span class="cx"> return malloc(size);
</span><span class="cx">
</span><del>- if (size <= smallMax) {
- size_t sizeClass = bmalloc::sizeClass(size);
</del><ins>+ if (size <= maskSizeClassMax) {
+ size_t sizeClass = bmalloc::maskSizeClass(size);
</ins><span class="cx"> BumpAllocator& allocator = m_bumpAllocators[sizeClass];
</span><span class="cx"> refillAllocator(allocator, sizeClass);
</span><span class="cx"> return allocator.allocate();
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+ if (size <= smallMax)
+ return allocateLogSizeClass(size);
+
</ins><span class="cx"> if (size <= largeMax)
</span><span class="cx"> return allocateLarge(size);
</span><span class="cx">
</span></span></pre></div>
<a id="trunkSourcebmallocbmallocAllocatorh"></a>
<div class="modfile"><h4>Modified: trunk/Source/bmalloc/bmalloc/Allocator.h (198570 => 198571)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/bmalloc/bmalloc/Allocator.h 2016-03-23 01:38:49 UTC (rev 198570)
+++ trunk/Source/bmalloc/bmalloc/Allocator.h 2016-03-23 01:39:36 UTC (rev 198571)
</span><span class="lines">@@ -52,14 +52,15 @@
</span><span class="cx"> bool allocateFastCase(size_t, void*&);
</span><span class="cx"> void* allocateSlowCase(size_t);
</span><span class="cx">
</span><ins>+ void* allocateLogSizeClass(size_t);
</ins><span class="cx"> void* allocateLarge(size_t);
</span><span class="cx"> void* allocateXLarge(size_t);
</span><span class="cx">
</span><span class="cx"> void refillAllocator(BumpAllocator&, size_t sizeClass);
</span><span class="cx"> void refillAllocatorSlowCase(BumpAllocator&, size_t sizeClass);
</span><span class="cx">
</span><del>- std::array<BumpAllocator, smallMax / alignment> m_bumpAllocators;
- std::array<BumpRangeCache, smallMax / alignment> m_bumpRangeCaches;
</del><ins>+ std::array<BumpAllocator, sizeClassCount> m_bumpAllocators;
+ std::array<BumpRangeCache, sizeClassCount> m_bumpRangeCaches;
</ins><span class="cx">
</span><span class="cx"> bool m_isBmallocEnabled;
</span><span class="cx"> Deallocator& m_deallocator;
</span><span class="lines">@@ -67,10 +68,10 @@
</span><span class="cx">
</span><span class="cx"> inline bool Allocator::allocateFastCase(size_t size, void*& object)
</span><span class="cx"> {
</span><del>- if (size > smallMax)
</del><ins>+ if (size > maskSizeClassMax)
</ins><span class="cx"> return false;
</span><span class="cx">
</span><del>- BumpAllocator& allocator = m_bumpAllocators[sizeClass(size)];
</del><ins>+ BumpAllocator& allocator = m_bumpAllocators[maskSizeClass(size)];
</ins><span class="cx"> if (!allocator.canAllocate())
</span><span class="cx"> return false;
</span><span class="cx">
</span></span></pre></div>
<a id="trunkSourcebmallocbmallocBumpAllocatorh"></a>
<div class="modfile"><h4>Modified: trunk/Source/bmalloc/bmalloc/BumpAllocator.h (198570 => 198571)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/bmalloc/bmalloc/BumpAllocator.h 2016-03-23 01:38:49 UTC (rev 198570)
+++ trunk/Source/bmalloc/bmalloc/BumpAllocator.h 2016-03-23 01:39:36 UTC (rev 198571)
</span><span class="lines">@@ -50,8 +50,6 @@
</span><span class="cx"> void refill(const BumpRange&);
</span><span class="cx">
</span><span class="cx"> private:
</span><del>- void validate(void*);
-
</del><span class="cx"> char* m_ptr;
</span><span class="cx"> unsigned short m_size;
</span><span class="cx"> unsigned short m_remaining;
</span><span class="lines">@@ -71,18 +69,6 @@
</span><span class="cx"> m_remaining = 0;
</span><span class="cx"> }
</span><span class="cx">
</span><del>-inline void BumpAllocator::validate(void* ptr)
-{
- UNUSED(ptr);
- if (m_size <= smallMax) {
- BASSERT(isSmall(ptr));
- return;
- }
-
- BASSERT(m_size <= smallMax);
- BASSERT(isSmall(ptr));
-}
-
</del><span class="cx"> inline void* BumpAllocator::allocate()
</span><span class="cx"> {
</span><span class="cx"> BASSERT(m_remaining);
</span><span class="lines">@@ -90,7 +76,7 @@
</span><span class="cx"> --m_remaining;
</span><span class="cx"> char* result = m_ptr;
</span><span class="cx"> m_ptr += m_size;
</span><del>- validate(result);
</del><ins>+ BASSERT(isSmall(result));
</ins><span class="cx"> return result;
</span><span class="cx"> }
</span><span class="cx">
</span></span></pre></div>
<a id="trunkSourcebmallocbmallocHeapcpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/bmalloc/bmalloc/Heap.cpp (198570 => 198571)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/bmalloc/bmalloc/Heap.cpp 2016-03-23 01:38:49 UTC (rev 198570)
+++ trunk/Source/bmalloc/bmalloc/Heap.cpp 2016-03-23 01:39:36 UTC (rev 198571)
</span><span class="lines">@@ -47,8 +47,8 @@
</span><span class="cx"> {
</span><span class="cx"> // We assume that m_smallLineMetadata is zero-filled.
</span><span class="cx">
</span><del>- for (size_t size = alignment; size <= smallMax; size += alignment) {
- size_t sizeClass = bmalloc::sizeClass(size);
</del><ins>+ for (size_t sizeClass = 0; sizeClass < sizeClassCount; ++sizeClass) {
+ size_t size = objectSize(sizeClass);
</ins><span class="cx"> auto& metadata = m_smallLineMetadata[sizeClass];
</span><span class="cx">
</span><span class="cx"> size_t object = 0;
</span></span></pre></div>
<a id="trunkSourcebmallocbmallocHeaph"></a>
<div class="modfile"><h4>Modified: trunk/Source/bmalloc/bmalloc/Heap.h (198570 => 198571)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/bmalloc/bmalloc/Heap.h 2016-03-23 01:38:49 UTC (rev 198570)
+++ trunk/Source/bmalloc/bmalloc/Heap.h 2016-03-23 01:39:36 UTC (rev 198571)
</span><span class="lines">@@ -92,9 +92,9 @@
</span><span class="cx"> void scavengeLargeObjects(std::unique_lock<StaticMutex>&, std::chrono::milliseconds);
</span><span class="cx"> void scavengeXLargeObjects(std::unique_lock<StaticMutex>&, std::chrono::milliseconds);
</span><span class="cx">
</span><del>- std::array<std::array<LineMetadata, smallLineCount>, smallMax / alignment> m_smallLineMetadata;
</del><ins>+ std::array<std::array<LineMetadata, smallLineCount>, sizeClassCount> m_smallLineMetadata;
</ins><span class="cx">
</span><del>- std::array<List<SmallPage>, smallMax / alignment> m_smallPagesWithFreeLines;
</del><ins>+ std::array<List<SmallPage>, sizeClassCount> m_smallPagesWithFreeLines;
</ins><span class="cx">
</span><span class="cx"> List<SmallPage> m_smallPages;
</span><span class="cx">
</span></span></pre></div>
<a id="trunkSourcebmallocbmallocSizesh"></a>
<div class="modfile"><h4>Modified: trunk/Source/bmalloc/bmalloc/Sizes.h (198570 => 198571)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/bmalloc/bmalloc/Sizes.h 2016-03-23 01:38:49 UTC (rev 198570)
+++ trunk/Source/bmalloc/bmalloc/Sizes.h 2016-03-23 01:39:36 UTC (rev 198571)
</span><span class="lines">@@ -60,10 +60,12 @@
</span><span class="cx"> static const size_t smallChunkOffset = superChunkSize / 2;
</span><span class="cx"> static const size_t smallChunkMask = ~(smallChunkSize - 1ul);
</span><span class="cx">
</span><del>- static const size_t smallMax = 1024;
</del><span class="cx"> static const size_t smallLineSize = 256;
</span><span class="cx"> static const size_t smallLineCount = vmPageSize / smallLineSize;
</span><span class="cx">
</span><ins>+ static const size_t smallMax = 1 * kB;
+ static const size_t maskSizeClassMax = 512;
+
</ins><span class="cx"> static const size_t largeChunkSize = superChunkSize / 2;
</span><span class="cx"> static const size_t largeChunkOffset = 0;
</span><span class="cx"> static const size_t largeChunkMask = ~(largeChunkSize - 1ul);
</span><span class="lines">@@ -90,17 +92,54 @@
</span><span class="cx">
</span><span class="cx"> static const std::chrono::milliseconds scavengeSleepDuration = std::chrono::milliseconds(512);
</span><span class="cx">
</span><ins>+ static const size_t maskSizeClassCount = maskSizeClassMax / alignment;
+
+ inline constexpr size_t maskSizeClass(size_t size)
+ {
+ // We mask to accommodate zero.
+ return mask((size - 1) / alignment, maskSizeClassCount - 1);
+ }
+
+ inline size_t maskObjectSize(size_t maskSizeClass)
+ {
+ return (maskSizeClass + 1) * alignment;
+ }
+
+ static const size_t logWasteFactor = 8;
+ static const size_t logAlignmentMin = maskSizeClassMax / logWasteFactor;
+
+ static const size_t logSizeClassCount = (log2(smallMax) - log2(maskSizeClassMax)) * logWasteFactor;
+
+ inline size_t logSizeClass(size_t size)
+ {
+ size_t base = log2(size - 1) - log2(maskSizeClassMax);
+ size_t offset = (size - 1 - (maskSizeClassMax << base));
+ return base * logWasteFactor + offset / (logAlignmentMin << base);
+ }
+
+ inline size_t logObjectSize(size_t logSizeClass)
+ {
+ size_t base = logSizeClass / logWasteFactor;
+ size_t offset = logSizeClass % logWasteFactor;
+ return (maskSizeClassMax << base) + (offset + 1) * (logAlignmentMin << base);
+ }
+
+ static const size_t sizeClassCount = maskSizeClassCount + logSizeClassCount;
+
</ins><span class="cx"> inline size_t sizeClass(size_t size)
</span><span class="cx"> {
</span><del>- static const size_t sizeClassMask = (smallMax / alignment) - 1;
- return mask((size - 1) / alignment, sizeClassMask);
</del><ins>+ if (size <= maskSizeClassMax)
+ return maskSizeClass(size);
+ return maskSizeClassCount + logSizeClass(size);
</ins><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> inline size_t objectSize(size_t sizeClass)
</span><span class="cx"> {
</span><del>- return (sizeClass + 1) * alignment;
</del><ins>+ if (sizeClass < maskSizeClassCount)
+ return maskObjectSize(sizeClass);
+ return logObjectSize(sizeClass - maskSizeClassCount);
</ins><span class="cx"> }
</span><del>-};
</del><ins>+}
</ins><span class="cx">
</span><span class="cx"> using namespace Sizes;
</span><span class="cx">
</span></span></pre></div>
<a id="trunkSourcebmallocbmallocSmallLineh"></a>
<div class="modfile"><h4>Modified: trunk/Source/bmalloc/bmalloc/SmallLine.h (198570 => 198571)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/bmalloc/bmalloc/SmallLine.h 2016-03-23 01:38:49 UTC (rev 198570)
+++ trunk/Source/bmalloc/bmalloc/SmallLine.h 2016-03-23 01:39:36 UTC (rev 198571)
</span><span class="lines">@@ -35,9 +35,6 @@
</span><span class="cx">
</span><span class="cx"> class SmallLine {
</span><span class="cx"> public:
</span><del>- static const unsigned char maxRefCount = std::numeric_limits<unsigned char>::max();
- static_assert(smallLineSize / alignment < maxRefCount, "maximum object count must fit in Line");
-
</del><span class="cx"> static SmallLine* get(void*);
</span><span class="cx">
</span><span class="cx"> void ref(std::lock_guard<StaticMutex>&, unsigned char);
</span><span class="lines">@@ -49,6 +46,11 @@
</span><span class="cx">
</span><span class="cx"> private:
</span><span class="cx"> unsigned char m_refCount;
</span><ins>+
+static_assert(
+ smallLineSize / alignment <= std::numeric_limits<decltype(m_refCount)>::max(),
+ "maximum object count must fit in SmallLine::m_refCount");
+
</ins><span class="cx"> };
</span><span class="cx">
</span><span class="cx"> inline void SmallLine::ref(std::lock_guard<StaticMutex>&, unsigned char refCount)
</span></span></pre></div>
<a id="trunkSourcebmallocbmallocSmallPageh"></a>
<div class="modfile"><h4>Modified: trunk/Source/bmalloc/bmalloc/SmallPage.h (198570 => 198571)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/bmalloc/bmalloc/SmallPage.h 2016-03-23 01:38:49 UTC (rev 198570)
+++ trunk/Source/bmalloc/bmalloc/SmallPage.h 2016-03-23 01:39:36 UTC (rev 198571)
</span><span class="lines">@@ -36,9 +36,6 @@
</span><span class="cx">
</span><span class="cx"> class SmallPage : public ListNode<SmallPage> {
</span><span class="cx"> public:
</span><del>- static const unsigned char maxRefCount = std::numeric_limits<unsigned char>::max();
- static_assert(smallLineCount < maxRefCount, "maximum line count must fit in SmallPage");
-
</del><span class="cx"> static SmallPage* get(SmallLine*);
</span><span class="cx">
</span><span class="cx"> SmallPage()
</span><span class="lines">@@ -63,12 +60,16 @@
</span><span class="cx"> unsigned char m_hasFreeLines: 1;
</span><span class="cx"> unsigned char m_refCount: 7;
</span><span class="cx"> unsigned char m_sizeClass;
</span><ins>+
+static_assert(
+ sizeClassCount <= std::numeric_limits<decltype(m_sizeClass)>::max(),
+ "Largest size class must fit in SmallPage metadata");
</ins><span class="cx"> };
</span><span class="cx">
</span><span class="cx"> inline void SmallPage::ref(std::lock_guard<StaticMutex>&)
</span><span class="cx"> {
</span><del>- BASSERT(m_refCount < maxRefCount);
</del><span class="cx"> ++m_refCount;
</span><ins>+ BASSERT(m_refCount);
</ins><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> inline bool SmallPage::deref(std::lock_guard<StaticMutex>&)
</span></span></pre></div>
<a id="trunkSourcebmallocbmallocZonecpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/bmalloc/bmalloc/Zone.cpp (198570 => 198571)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/bmalloc/bmalloc/Zone.cpp 2016-03-23 01:38:49 UTC (rev 198570)
+++ trunk/Source/bmalloc/bmalloc/Zone.cpp 2016-03-23 01:39:36 UTC (rev 198571)
</span><span class="lines">@@ -78,7 +78,7 @@
</span><span class="cx"> memset(statistics, 0, sizeof(malloc_statistics_t));
</span><span class="cx"> }
</span><span class="cx">
</span><del>-static size_t size(malloc_zone_t*, const void*)
</del><ins>+static size_t zoneSize(malloc_zone_t*, const void*)
</ins><span class="cx"> {
</span><span class="cx"> // Our zone is not public API, so no pointer can belong to us.
</span><span class="cx"> return 0;
</span><span class="lines">@@ -104,7 +104,7 @@
</span><span class="cx"> // The memory analysis API requires the contents of this struct to be a static
</span><span class="cx"> // constant in the program binary. The leaks process will load this struct
</span><span class="cx"> // out of the program binary (and not out of the running process).
</span><del>-static malloc_introspection_t introspect = {
</del><ins>+static malloc_introspection_t zoneIntrospect = {
</ins><span class="cx"> .enumerator = bmalloc::enumerator,
</span><span class="cx"> .good_size = bmalloc::good_size,
</span><span class="cx"> .check = bmalloc::check,
</span><span class="lines">@@ -117,9 +117,9 @@
</span><span class="cx">
</span><span class="cx"> Zone::Zone()
</span><span class="cx"> {
</span><del>- malloc_zone_t::size = &bmalloc::size;
</del><ins>+ malloc_zone_t::size = &bmalloc::zoneSize;
</ins><span class="cx"> malloc_zone_t::zone_name = "WebKit Malloc";
</span><del>- malloc_zone_t::introspect = &bmalloc::introspect;
</del><ins>+ malloc_zone_t::introspect = &bmalloc::zoneIntrospect;
</ins><span class="cx"> malloc_zone_t::version = 4;
</span><span class="cx"> malloc_zone_register(this);
</span><span class="cx"> }
</span></span></pre>
</div>
</div>
</body>
</html>