<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
"http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head><meta http-equiv="content-type" content="text/html; charset=utf-8" />
<title>[196922] releases/WebKitGTK/webkit-2.10/Source/bmalloc</title>
</head>
<body>
<style type="text/css"><!--
#msg dl.meta { border: 1px #006 solid; background: #369; padding: 6px; color: #fff; }
#msg dl.meta dt { float: left; width: 6em; font-weight: bold; }
#msg dt:after { content:':';}
#msg dl, #msg dt, #msg ul, #msg li, #header, #footer, #logmsg { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt; }
#msg dl a { font-weight: bold}
#msg dl a:link { color:#fc3; }
#msg dl a:active { color:#ff0; }
#msg dl a:visited { color:#cc6; }
h3 { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt; font-weight: bold; }
#msg pre { overflow: auto; background: #ffc; border: 1px #fa0 solid; padding: 6px; }
#logmsg { background: #ffc; border: 1px #fa0 solid; padding: 1em 1em 0 1em; }
#logmsg p, #logmsg pre, #logmsg blockquote { margin: 0 0 1em 0; }
#logmsg p, #logmsg li, #logmsg dt, #logmsg dd { line-height: 14pt; }
#logmsg h1, #logmsg h2, #logmsg h3, #logmsg h4, #logmsg h5, #logmsg h6 { margin: .5em 0; }
#logmsg h1:first-child, #logmsg h2:first-child, #logmsg h3:first-child, #logmsg h4:first-child, #logmsg h5:first-child, #logmsg h6:first-child { margin-top: 0; }
#logmsg ul, #logmsg ol { padding: 0; list-style-position: inside; margin: 0 0 0 1em; }
#logmsg ul { text-indent: -1em; padding-left: 1em; }#logmsg ol { text-indent: -1.5em; padding-left: 1.5em; }
#logmsg > ul, #logmsg > ol { margin: 0 0 1em 0; }
#logmsg pre { background: #eee; padding: 1em; }
#logmsg blockquote { border: 1px solid #fa0; border-left-width: 10px; padding: 1em 1em 0 1em; background: white;}
#logmsg dl { margin: 0; }
#logmsg dt { font-weight: bold; }
#logmsg dd { margin: 0; padding: 0 0 0.5em 0; }
#logmsg dd:before { content:'\00bb';}
#logmsg table { border-spacing: 0px; border-collapse: collapse; border-top: 4px solid #fa0; border-bottom: 1px solid #fa0; background: #fff; }
#logmsg table th { text-align: left; font-weight: normal; padding: 0.2em 0.5em; border-top: 1px dotted #fa0; }
#logmsg table td { text-align: right; border-top: 1px dotted #fa0; padding: 0.2em 0.5em; }
#logmsg table thead th { text-align: center; border-bottom: 1px solid #fa0; }
#logmsg table th.Corner { text-align: left; }
#logmsg hr { border: none 0; border-top: 2px dashed #fa0; height: 1px; }
#header, #footer { color: #fff; background: #636; border: 1px #300 solid; padding: 6px; }
#patch { width: 100%; }
#patch h4 {font-family: verdana,arial,helvetica,sans-serif;font-size:10pt;padding:8px;background:#369;color:#fff;margin:0;}
#patch .propset h4, #patch .binary h4 {margin:0;}
#patch pre {padding:0;line-height:1.2em;margin:0;}
#patch .diff {width:100%;background:#eee;padding: 0 0 10px 0;overflow:auto;}
#patch .propset .diff, #patch .binary .diff {padding:10px 0;}
#patch span {display:block;padding:0 10px;}
#patch .modfile, #patch .addfile, #patch .delfile, #patch .propset, #patch .binary, #patch .copfile {border:1px solid #ccc;margin:10px 0;}
#patch ins {background:#dfd;text-decoration:none;display:block;padding:0 10px;}
#patch del {background:#fdd;text-decoration:none;display:block;padding:0 10px;}
#patch .lines, .info {color:#888;background:#fff;}
--></style>
<div id="msg">
<dl class="meta">
<dt>Revision</dt> <dd><a href="http://trac.webkit.org/projects/webkit/changeset/196922">196922</a></dd>
<dt>Author</dt> <dd>carlosgc@webkit.org</dd>
<dt>Date</dt> <dd>2016-02-22 02:19:05 -0800 (Mon, 22 Feb 2016)</dd>
</dl>
<h3>Log Message</h3>
<pre>Merge <a href="http://trac.webkit.org/projects/webkit/changeset/196421">r196421</a>, <a href="http://trac.webkit.org/projects/webkit/changeset/196424">r196424</a>, <a href="http://trac.webkit.org/projects/webkit/changeset/196536">r196536</a>: bmalloc: large aligned allocations will put 1 or 2 free object on free list without merging with free neighbors
https://bugs.webkit.org/show_bug.cgi?id=154091
Reviewed by Geoffrey Garen.
If we split off any unused free object in the aligned version of Heap::allocateLarge(), we merge them with
free neighbors before putting them back on the free list. Added helpers to verify that when we
add LargeObjects to the free list their neighbors are allocated.
* bmalloc/Heap.cpp:
(bmalloc::Heap::allocateLarge): Deleted private helper version and rolled it into the two the
two public versions of allocateLarge().
* bmalloc/Heap.h:
* bmalloc/LargeObject.h:
(bmalloc::LargeObject::prevIsAllocated): New helper.
(bmalloc::LargeObject::nextIsAllocated): New helper.
(bmalloc::LargeObject::merge): Check that the merge object has allocated neighbors.
Unreviewed build fix after <a href="http://trac.webkit.org/projects/webkit/changeset/196421">r196421</a>.
Removed BASSERTs that are firing to eliminate Debug build crashes. I'll debug locally and
enable or alter after the issue is understood.
* bmalloc/LargeObject.h:
(bmalloc::LargeObject::merge): Removed BASSERTs that are firing.
BASSERTs added in <a href="http://trac.webkit.org/projects/webkit/changeset/196421">r196421</a> are causing debug test failures
https://bugs.webkit.org/show_bug.cgi?id=154113
Reviewed by Geoffrey Garen.
In VMHeap::deallocateLargeObject(), we drop the lock to deallocate the physical pages.
If the scavenger thread is running at the same time a synchronous call to scavenge()
comes in, we could call VMHeap::deallocateLargeObject() for an adjacent object while the
lock in the other thread is dropped. We fix this by checking for adjacent objects we
can merge with and loop if we have one.
* bmalloc/FreeList.h:
(bmalloc::FreeList::push): Added BASSERT to catch adding unmerged free objects
* bmalloc/Heap.cpp:
(bmalloc::Heap::allocateLarge): Changed to use nextCanMerge().
* bmalloc/LargeObject.h:
(bmalloc::LargeObject::prevCanMerge): Repurposed prevIsAllocated.
(bmalloc::LargeObject::nextCanMerge): Repurposed nextIsAllocated.
(bmalloc::LargeObject::prevIsAllocated): Deleted.
(bmalloc::LargeObject::nextIsAllocated): Deleted.
* bmalloc/VMHeap.h:
(bmalloc::VMHeap::allocateLargeObject): Moved adding the extra object back to the free list
to after we set the object we'll return as being allocated.
(bmalloc::VMHeap::deallocateLargeObject):</pre>
<h3>Modified Paths</h3>
<ul>
<li><a href="#releasesWebKitGTKwebkit210SourcebmallocChangeLog">releases/WebKitGTK/webkit-2.10/Source/bmalloc/ChangeLog</a></li>
<li><a href="#releasesWebKitGTKwebkit210SourcebmallocbmallocFreeListh">releases/WebKitGTK/webkit-2.10/Source/bmalloc/bmalloc/FreeList.h</a></li>
<li><a href="#releasesWebKitGTKwebkit210SourcebmallocbmallocHeapcpp">releases/WebKitGTK/webkit-2.10/Source/bmalloc/bmalloc/Heap.cpp</a></li>
<li><a href="#releasesWebKitGTKwebkit210SourcebmallocbmallocHeaph">releases/WebKitGTK/webkit-2.10/Source/bmalloc/bmalloc/Heap.h</a></li>
<li><a href="#releasesWebKitGTKwebkit210SourcebmallocbmallocLargeObjecth">releases/WebKitGTK/webkit-2.10/Source/bmalloc/bmalloc/LargeObject.h</a></li>
<li><a href="#releasesWebKitGTKwebkit210SourcebmallocbmallocVMHeaph">releases/WebKitGTK/webkit-2.10/Source/bmalloc/bmalloc/VMHeap.h</a></li>
</ul>
</div>
<div id="patch">
<h3>Diff</h3>
<a id="releasesWebKitGTKwebkit210SourcebmallocChangeLog"></a>
<div class="modfile"><h4>Modified: releases/WebKitGTK/webkit-2.10/Source/bmalloc/ChangeLog (196921 => 196922)</h4>
<pre class="diff"><span>
<span class="info">--- releases/WebKitGTK/webkit-2.10/Source/bmalloc/ChangeLog 2016-02-22 10:12:04 UTC (rev 196921)
+++ releases/WebKitGTK/webkit-2.10/Source/bmalloc/ChangeLog 2016-02-22 10:19:05 UTC (rev 196922)
</span><span class="lines">@@ -1,3 +1,60 @@
</span><ins>+2016-02-12 Michael Saboff <msaboff@apple.com>
+
+ BASSERTs added in r196421 are causing debug test failures
+ https://bugs.webkit.org/show_bug.cgi?id=154113
+
+ Reviewed by Geoffrey Garen.
+
+ In VMHeap::deallocateLargeObject(), we drop the lock to deallocate the physical pages.
+ If the scavenger thread is running at the same time a synchronous call to scavenge()
+ comes in, we could call VMHeap::deallocateLargeObject() for an adjacent object while the
+ lock in the other thread is dropped. We fix this by checking for adjacent objects we
+ can merge with and loop if we have one.
+
+ * bmalloc/FreeList.h:
+ (bmalloc::FreeList::push): Added BASSERT to catch adding unmerged free objects
+ * bmalloc/Heap.cpp:
+ (bmalloc::Heap::allocateLarge): Changed to use nextCanMerge().
+ * bmalloc/LargeObject.h:
+ (bmalloc::LargeObject::prevCanMerge): Repurposed prevIsAllocated.
+ (bmalloc::LargeObject::nextCanMerge): Repurposed nextIsAllocated.
+ (bmalloc::LargeObject::prevIsAllocated): Deleted.
+ (bmalloc::LargeObject::nextIsAllocated): Deleted.
+ * bmalloc/VMHeap.h:
+ (bmalloc::VMHeap::allocateLargeObject): Moved adding the extra object back to the free list
+ to after we set the object we'll return as being allocated.
+ (bmalloc::VMHeap::deallocateLargeObject):
+
+2016-02-11 Michael Saboff <msaboff@apple.com>
+
+ Unreviewed build fix after r196421.
+
+ Removed BASSERTs that are firing to eliminate Debug build crashes. I'll debug locally and
+ enable or alter after the issue is understood.
+
+ * bmalloc/LargeObject.h:
+ (bmalloc::LargeObject::merge): Removed BASSERTs that are firing.
+
+2016-02-11 Michael Saboff <msaboff@apple.com>
+
+ bmalloc: large aligned allocations will put 1 or 2 free object on free list without merging with free neighbors
+ https://bugs.webkit.org/show_bug.cgi?id=154091
+
+ Reviewed by Geoffrey Garen.
+
+ If we split off any unused free object in the aligned version of Heap::allocateLarge(), we merge them with
+ free neighbors before putting them back on the free list. Added helpers to verify that when we
+ add LargeObjects to the free list their neighbors are allocated.
+
+ * bmalloc/Heap.cpp:
+ (bmalloc::Heap::allocateLarge): Deleted private helper version and rolled it into the two the
+ two public versions of allocateLarge().
+ * bmalloc/Heap.h:
+ * bmalloc/LargeObject.h:
+ (bmalloc::LargeObject::prevIsAllocated): New helper.
+ (bmalloc::LargeObject::nextIsAllocated): New helper.
+ (bmalloc::LargeObject::merge): Check that the merge object has allocated neighbors.
+
</ins><span class="cx"> 2015-12-03 Michael Saboff <msaboff@apple.com>
</span><span class="cx">
</span><span class="cx"> bmalloc: extra large allocations could be more efficient
</span></span></pre></div>
<a id="releasesWebKitGTKwebkit210SourcebmallocbmallocFreeListh"></a>
<div class="modfile"><h4>Modified: releases/WebKitGTK/webkit-2.10/Source/bmalloc/bmalloc/FreeList.h (196921 => 196922)</h4>
<pre class="diff"><span>
<span class="info">--- releases/WebKitGTK/webkit-2.10/Source/bmalloc/bmalloc/FreeList.h 2016-02-22 10:12:04 UTC (rev 196921)
+++ releases/WebKitGTK/webkit-2.10/Source/bmalloc/bmalloc/FreeList.h 2016-02-22 10:19:05 UTC (rev 196922)
</span><span class="lines">@@ -1,5 +1,5 @@
</span><span class="cx"> /*
</span><del>- * Copyright (C) 2014, 2015 Apple Inc. All rights reserved.
</del><ins>+ * Copyright (C) 2014-2016 Apple Inc. All rights reserved.
</ins><span class="cx"> *
</span><span class="cx"> * Redistribution and use in source and binary forms, with or without
</span><span class="cx"> * modification, are permitted provided that the following conditions
</span><span class="lines">@@ -60,6 +60,8 @@
</span><span class="cx"> inline void FreeList::push(Owner owner, const LargeObject& largeObject)
</span><span class="cx"> {
</span><span class="cx"> BASSERT(largeObject.isFree());
</span><ins>+ BASSERT(!largeObject.prevCanMerge());
+ BASSERT(!largeObject.nextCanMerge());
</ins><span class="cx"> if (m_vector.size() == m_limit) {
</span><span class="cx"> removeInvalidAndDuplicateEntries(owner);
</span><span class="cx"> m_limit = std::max(m_vector.size() * freeListGrowFactor, freeListSearchDepth);
</span></span></pre></div>
<a id="releasesWebKitGTKwebkit210SourcebmallocbmallocHeapcpp"></a>
<div class="modfile"><h4>Modified: releases/WebKitGTK/webkit-2.10/Source/bmalloc/bmalloc/Heap.cpp (196921 => 196922)</h4>
<pre class="diff"><span>
<span class="info">--- releases/WebKitGTK/webkit-2.10/Source/bmalloc/bmalloc/Heap.cpp 2016-02-22 10:12:04 UTC (rev 196921)
+++ releases/WebKitGTK/webkit-2.10/Source/bmalloc/bmalloc/Heap.cpp 2016-02-22 10:19:05 UTC (rev 196922)
</span><span class="lines">@@ -346,22 +346,8 @@
</span><span class="cx"> lock.lock();
</span><span class="cx"> }
</span><span class="cx">
</span><del>-void* Heap::allocateLarge(std::lock_guard<StaticMutex>&, LargeObject& largeObject, size_t size)
</del><ins>+void* Heap::allocateLarge(std::lock_guard<StaticMutex>&, size_t size)
</ins><span class="cx"> {
</span><del>- BASSERT(largeObject.isFree());
-
- if (largeObject.size() - size > largeMin) {
- std::pair<LargeObject, LargeObject> split = largeObject.split(size);
- largeObject = split.first;
- m_largeObjects.insert(split.second);
- }
-
- largeObject.setFree(false);
- return largeObject.begin();
-}
-
-void* Heap::allocateLarge(std::lock_guard<StaticMutex>& lock, size_t size)
-{
</del><span class="cx"> BASSERT(size <= largeMax);
</span><span class="cx"> BASSERT(size >= largeMin);
</span><span class="cx"> BASSERT(size == roundUpToMultipleOf<largeAlignment>(size));
</span><span class="lines">@@ -372,10 +358,27 @@
</span><span class="cx"> largeObject = m_vmHeap.allocateLargeObject(size);
</span><span class="cx"> }
</span><span class="cx">
</span><del>- return allocateLarge(lock, largeObject, size);
</del><ins>+ BASSERT(largeObject.isFree());
+
+ LargeObject nextLargeObject;
+
+ if (largeObject.size() - size > largeMin) {
+ std::pair<LargeObject, LargeObject> split = largeObject.split(size);
+ largeObject = split.first;
+ nextLargeObject = split.second;
+ }
+
+ largeObject.setFree(false);
+
+ if (nextLargeObject) {
+ BASSERT(!nextLargeObject.nextCanMerge());
+ m_largeObjects.insert(nextLargeObject);
+ }
+
+ return largeObject.begin();
</ins><span class="cx"> }
</span><span class="cx">
</span><del>-void* Heap::allocateLarge(std::lock_guard<StaticMutex>& lock, size_t alignment, size_t size, size_t unalignedSize)
</del><ins>+void* Heap::allocateLarge(std::lock_guard<StaticMutex>&, size_t alignment, size_t size, size_t unalignedSize)
</ins><span class="cx"> {
</span><span class="cx"> BASSERT(size <= largeMax);
</span><span class="cx"> BASSERT(size >= largeMin);
</span><span class="lines">@@ -393,15 +396,38 @@
</span><span class="cx"> largeObject = m_vmHeap.allocateLargeObject(alignment, size, unalignedSize);
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+ LargeObject prevLargeObject;
+ LargeObject nextLargeObject;
+
</ins><span class="cx"> size_t alignmentMask = alignment - 1;
</span><span class="cx"> if (test(largeObject.begin(), alignmentMask)) {
</span><span class="cx"> size_t prefixSize = roundUpToMultipleOf(alignment, largeObject.begin() + largeMin) - largeObject.begin();
</span><span class="cx"> std::pair<LargeObject, LargeObject> pair = largeObject.split(prefixSize);
</span><del>- m_largeObjects.insert(pair.first);
</del><ins>+ prevLargeObject = pair.first;
</ins><span class="cx"> largeObject = pair.second;
</span><span class="cx"> }
</span><span class="cx">
</span><del>- return allocateLarge(lock, largeObject, size);
</del><ins>+ BASSERT(largeObject.isFree());
+
+ if (largeObject.size() - size > largeMin) {
+ std::pair<LargeObject, LargeObject> split = largeObject.split(size);
+ largeObject = split.first;
+ nextLargeObject = split.second;
+ }
+
+ largeObject.setFree(false);
+
+ if (prevLargeObject) {
+ LargeObject merged = prevLargeObject.merge();
+ m_largeObjects.insert(merged);
+ }
+
+ if (nextLargeObject) {
+ LargeObject merged = nextLargeObject.merge();
+ m_largeObjects.insert(merged);
+ }
+
+ return largeObject.begin();
</ins><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> void Heap::deallocateLarge(std::lock_guard<StaticMutex>&, const LargeObject& largeObject)
</span></span></pre></div>
<a id="releasesWebKitGTKwebkit210SourcebmallocbmallocHeaph"></a>
<div class="modfile"><h4>Modified: releases/WebKitGTK/webkit-2.10/Source/bmalloc/bmalloc/Heap.h (196921 => 196922)</h4>
<pre class="diff"><span>
<span class="info">--- releases/WebKitGTK/webkit-2.10/Source/bmalloc/bmalloc/Heap.h 2016-02-22 10:12:04 UTC (rev 196921)
+++ releases/WebKitGTK/webkit-2.10/Source/bmalloc/bmalloc/Heap.h 2016-02-22 10:19:05 UTC (rev 196922)
</span><span class="lines">@@ -83,7 +83,6 @@
</span><span class="cx"> void deallocateSmallLine(std::lock_guard<StaticMutex>&, SmallLine*);
</span><span class="cx"> void deallocateMediumLine(std::lock_guard<StaticMutex>&, MediumLine*);
</span><span class="cx">
</span><del>- void* allocateLarge(std::lock_guard<StaticMutex>&, LargeObject&, size_t);
</del><span class="cx"> void deallocateLarge(std::lock_guard<StaticMutex>&, const LargeObject&);
</span><span class="cx">
</span><span class="cx"> void splitLarge(BeginTag*, size_t, EndTag*&, Range&);
</span></span></pre></div>
<a id="releasesWebKitGTKwebkit210SourcebmallocbmallocLargeObjecth"></a>
<div class="modfile"><h4>Modified: releases/WebKitGTK/webkit-2.10/Source/bmalloc/bmalloc/LargeObject.h (196921 => 196922)</h4>
<pre class="diff"><span>
<span class="info">--- releases/WebKitGTK/webkit-2.10/Source/bmalloc/bmalloc/LargeObject.h 2016-02-22 10:12:04 UTC (rev 196921)
+++ releases/WebKitGTK/webkit-2.10/Source/bmalloc/bmalloc/LargeObject.h 2016-02-22 10:19:05 UTC (rev 196922)
</span><span class="lines">@@ -1,5 +1,5 @@
</span><span class="cx"> /*
</span><del>- * Copyright (C) 2015 Apple Inc. All rights reserved.
</del><ins>+ * Copyright (C) 2015-2016 Apple Inc. All rights reserved.
</ins><span class="cx"> *
</span><span class="cx"> * Redistribution and use in source and binary forms, with or without
</span><span class="cx"> * modification, are permitted provided that the following conditions
</span><span class="lines">@@ -53,7 +53,10 @@
</span><span class="cx">
</span><span class="cx"> void setFree(bool) const;
</span><span class="cx"> bool isFree() const;
</span><del>-
</del><ins>+
+ bool prevCanMerge() const;
+ bool nextCanMerge() const;
+
</ins><span class="cx"> Owner owner() const;
</span><span class="cx"> void setOwner(Owner) const;
</span><span class="cx">
</span><span class="lines">@@ -118,6 +121,20 @@
</span><span class="cx"> return m_beginTag->isFree();
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+inline bool LargeObject::prevCanMerge() const
+{
+ EndTag* prev = m_beginTag->prev();
+
+ return prev->isFree() && prev->owner() == this->owner();
+}
+
+inline bool LargeObject::nextCanMerge() const
+{
+ BeginTag* next = m_endTag->next();
+
+ return next->isFree() && next->owner() == this->owner();
+}
+
</ins><span class="cx"> inline Owner LargeObject::owner() const
</span><span class="cx"> {
</span><span class="cx"> validate();
</span></span></pre></div>
<a id="releasesWebKitGTKwebkit210SourcebmallocbmallocVMHeaph"></a>
<div class="modfile"><h4>Modified: releases/WebKitGTK/webkit-2.10/Source/bmalloc/bmalloc/VMHeap.h (196921 => 196922)</h4>
<pre class="diff"><span>
<span class="info">--- releases/WebKitGTK/webkit-2.10/Source/bmalloc/bmalloc/VMHeap.h 2016-02-22 10:12:04 UTC (rev 196921)
+++ releases/WebKitGTK/webkit-2.10/Source/bmalloc/bmalloc/VMHeap.h 2016-02-22 10:19:05 UTC (rev 196922)
</span><span class="lines">@@ -1,5 +1,5 @@
</span><span class="cx"> /*
</span><del>- * Copyright (C) 2014, 2015 Apple Inc. All rights reserved.
</del><ins>+ * Copyright (C) 2014-2016 Apple Inc. All rights reserved.
</ins><span class="cx"> *
</span><span class="cx"> * Redistribution and use in source and binary forms, with or without
</span><span class="cx"> * modification, are permitted provided that the following conditions
</span><span class="lines">@@ -57,7 +57,7 @@
</span><span class="cx">
</span><span class="cx"> void deallocateSmallPage(std::unique_lock<StaticMutex>&, SmallPage*);
</span><span class="cx"> void deallocateMediumPage(std::unique_lock<StaticMutex>&, MediumPage*);
</span><del>- void deallocateLargeObject(std::unique_lock<StaticMutex>&, LargeObject&);
</del><ins>+ void deallocateLargeObject(std::unique_lock<StaticMutex>&, LargeObject);
</ins><span class="cx">
</span><span class="cx"> private:
</span><span class="cx"> LargeObject allocateLargeObject(LargeObject&, size_t);
</span><span class="lines">@@ -95,14 +95,23 @@
</span><span class="cx"> {
</span><span class="cx"> BASSERT(largeObject.isFree());
</span><span class="cx">
</span><ins>+ LargeObject nextLargeObject;
+
</ins><span class="cx"> if (largeObject.size() - size > largeMin) {
</span><span class="cx"> std::pair<LargeObject, LargeObject> split = largeObject.split(size);
</span><span class="cx"> largeObject = split.first;
</span><del>- m_largeObjects.insert(split.second);
</del><ins>+ nextLargeObject = split.second;
</ins><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> vmAllocatePhysicalPagesSloppy(largeObject.begin(), largeObject.size());
</span><span class="cx"> largeObject.setOwner(Owner::Heap);
</span><ins>+
+ // Be sure to set the owner for the object we return before inserting the leftover back
+ // into the free list. The free list asserts that we never insert an object that could
+ // have merged with its neighbor.
+ if (nextLargeObject)
+ m_largeObjects.insert(nextLargeObject);
+
</ins><span class="cx"> return largeObject.begin();
</span><span class="cx"> }
</span><span class="cx">
</span><span class="lines">@@ -151,25 +160,29 @@
</span><span class="cx"> m_mediumPages.push(page);
</span><span class="cx"> }
</span><span class="cx">
</span><del>-inline void VMHeap::deallocateLargeObject(std::unique_lock<StaticMutex>& lock, LargeObject& largeObject)
</del><ins>+inline void VMHeap::deallocateLargeObject(std::unique_lock<StaticMutex>& lock, LargeObject largeObject)
</ins><span class="cx"> {
</span><span class="cx"> largeObject.setOwner(Owner::VMHeap);
</span><del>-
- // If we couldn't merge with our neighbors before because they were in the
- // VM heap, we can merge with them now.
- LargeObject merged = largeObject.merge();
</del><span class="cx">
</span><del>- // Temporarily mark this object as allocated to prevent clients from merging
- // with it or allocating it while we're messing with its physical pages.
- merged.setFree(false);
</del><ins>+ // Multiple threads might scavenge concurrently, meaning that new merging opportunities
+ // become visible after we reacquire the lock. Therefore we loop.
+ do {
+ // If we couldn't merge with our neighbors before because they were in the
+ // VM heap, we can merge with them now.
+ largeObject = largeObject.merge();
</ins><span class="cx">
</span><del>- lock.unlock();
- vmDeallocatePhysicalPagesSloppy(merged.begin(), merged.size());
- lock.lock();
</del><ins>+ // Temporarily mark this object as allocated to prevent clients from merging
+ // with it or allocating it while we're messing with its physical pages.
+ largeObject.setFree(false);
</ins><span class="cx">
</span><del>- merged.setFree(true);
</del><ins>+ lock.unlock();
+ vmDeallocatePhysicalPagesSloppy(largeObject.begin(), largeObject.size());
+ lock.lock();
</ins><span class="cx">
</span><del>- m_largeObjects.insert(merged);
</del><ins>+ largeObject.setFree(true);
+ } while (largeObject.prevCanMerge() || largeObject.nextCanMerge());
+
+ m_largeObjects.insert(largeObject);
</ins><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> } // namespace bmalloc
</span></span></pre>
</div>
</div>
</body>
</html>