<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
"http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head><meta http-equiv="content-type" content="text/html; charset=utf-8" />
<title>[181086] trunk/Source</title>
</head>
<body>
<style type="text/css"><!--
#msg dl.meta { border: 1px #006 solid; background: #369; padding: 6px; color: #fff; }
#msg dl.meta dt { float: left; width: 6em; font-weight: bold; }
#msg dt:after { content:':';}
#msg dl, #msg dt, #msg ul, #msg li, #header, #footer, #logmsg { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt; }
#msg dl a { font-weight: bold}
#msg dl a:link { color:#fc3; }
#msg dl a:active { color:#ff0; }
#msg dl a:visited { color:#cc6; }
h3 { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt; font-weight: bold; }
#msg pre { overflow: auto; background: #ffc; border: 1px #fa0 solid; padding: 6px; }
#logmsg { background: #ffc; border: 1px #fa0 solid; padding: 1em 1em 0 1em; }
#logmsg p, #logmsg pre, #logmsg blockquote { margin: 0 0 1em 0; }
#logmsg p, #logmsg li, #logmsg dt, #logmsg dd { line-height: 14pt; }
#logmsg h1, #logmsg h2, #logmsg h3, #logmsg h4, #logmsg h5, #logmsg h6 { margin: .5em 0; }
#logmsg h1:first-child, #logmsg h2:first-child, #logmsg h3:first-child, #logmsg h4:first-child, #logmsg h5:first-child, #logmsg h6:first-child { margin-top: 0; }
#logmsg ul, #logmsg ol { padding: 0; list-style-position: inside; margin: 0 0 0 1em; }
#logmsg ul { text-indent: -1em; padding-left: 1em; }#logmsg ol { text-indent: -1.5em; padding-left: 1.5em; }
#logmsg > ul, #logmsg > ol { margin: 0 0 1em 0; }
#logmsg pre { background: #eee; padding: 1em; }
#logmsg blockquote { border: 1px solid #fa0; border-left-width: 10px; padding: 1em 1em 0 1em; background: white;}
#logmsg dl { margin: 0; }
#logmsg dt { font-weight: bold; }
#logmsg dd { margin: 0; padding: 0 0 0.5em 0; }
#logmsg dd:before { content:'\00bb';}
#logmsg table { border-spacing: 0px; border-collapse: collapse; border-top: 4px solid #fa0; border-bottom: 1px solid #fa0; background: #fff; }
#logmsg table th { text-align: left; font-weight: normal; padding: 0.2em 0.5em; border-top: 1px dotted #fa0; }
#logmsg table td { text-align: right; border-top: 1px dotted #fa0; padding: 0.2em 0.5em; }
#logmsg table thead th { text-align: center; border-bottom: 1px solid #fa0; }
#logmsg table th.Corner { text-align: left; }
#logmsg hr { border: none 0; border-top: 2px dashed #fa0; height: 1px; }
#header, #footer { color: #fff; background: #636; border: 1px #300 solid; padding: 6px; }
#patch { width: 100%; }
#patch h4 {font-family: verdana,arial,helvetica,sans-serif;font-size:10pt;padding:8px;background:#369;color:#fff;margin:0;}
#patch .propset h4, #patch .binary h4 {margin:0;}
#patch pre {padding:0;line-height:1.2em;margin:0;}
#patch .diff {width:100%;background:#eee;padding: 0 0 10px 0;overflow:auto;}
#patch .propset .diff, #patch .binary .diff {padding:10px 0;}
#patch span {display:block;padding:0 10px;}
#patch .modfile, #patch .addfile, #patch .delfile, #patch .propset, #patch .binary, #patch .copfile {border:1px solid #ccc;margin:10px 0;}
#patch ins {background:#dfd;text-decoration:none;display:block;padding:0 10px;}
#patch del {background:#fdd;text-decoration:none;display:block;padding:0 10px;}
#patch .lines, .info {color:#888;background:#fff;}
--></style>
<div id="msg">
<dl class="meta">
<dt>Revision</dt> <dd><a href="http://trac.webkit.org/projects/webkit/changeset/181086">181086</a></dd>
<dt>Author</dt> <dd>cdumez@apple.com</dd>
<dt>Date</dt> <dd>2015-03-05 10:59:57 -0800 (Thu, 05 Mar 2015)</dd>
</dl>
<h3>Log Message</h3>
<pre>NetworkCache efficacy logging is using too much CPU
https://bugs.webkit.org/show_bug.cgi?id=142186
<rdar://problem/19632080>
Reviewed by Antti Koivisto.
Source/WebKit2:
NetworkCache efficacy logging was using too much CPU mostly due to
database writes. Logging was using ~11.3% of the NetworkProcess'
CPU usage (~9.2% for writes / ~1.3% for reads).
This patch buffers writes requests in memory and only writes them
to disk every 10 seconds. We are thus writing to this less frequently
and writing more at once, in a single SQL transaction. After this
change, efficacy logging only accounts for ~3.5% of the NetworkProcess'
CPU activity (1.5% for writes / 1.2% for reads).
Now that CPU usage is more acceptable, this patch re-enables the
network cache efficicy logging.
* NetworkProcess/cache/NetworkCacheStatistics.h:
* NetworkProcess/cache/NetworkCacheStatisticsCocoa.mm:
(WebKit::executeSQLCommand):
(WebKit::executeSQLStatement):
Call step() instead of executeCommand() because:
- The input statement is already prepared.
- step() does not finalize() the statement and thus we caller can
reuse it after calling reset().
(WebKit::NetworkCacheStatistics::NetworkCacheStatistics):
(WebKit::NetworkCacheStatistics::initialize):
(WebKit::NetworkCacheStatistics::bootstrapFromNetworkCache):
(WebKit::NetworkCacheStatistics::recordNotCachingResponse):
(WebKit::NetworkCacheStatistics::recordNotUsingCacheForRequest):
(WebKit::NetworkCacheStatistics::recordRetrievalFailure):
(WebKit::NetworkCacheStatistics::markAsRequested):
(WebKit::NetworkCacheStatistics::writeTimerFired):
(WebKit::NetworkCacheStatistics::queryWasEverRequested):
(WebKit::NetworkCacheStatistics::addHashesToDatabase):
(WebKit::NetworkCacheStatistics::addStoreDecisionsToDatabase):
(WebKit::NetworkCacheStatistics::addHashToDatabase): Deleted.
* UIProcess/Cocoa/WebProcessPoolCocoa.mm:
(WebKit::registerUserDefaultsIfNeeded):
Re-enable the network cache efficacy logging.
Source/WTF:
* wtf/HashMap.h:
(WTF::copyToVector):
Add copyToVector() utility function for HashMap that uses an std::pair<key, value>
as element type.
* wtf/text/WTFString.h:
(WTF::StringCapture::StringCapture):
(WTF::StringCapture::operator=):
Add default constructor and assignment operator to StringCapture so that
it can be used in a Vector.</pre>
<h3>Modified Paths</h3>
<ul>
<li><a href="#trunkSourceWTFChangeLog">trunk/Source/WTF/ChangeLog</a></li>
<li><a href="#trunkSourceWTFwtfHashMaph">trunk/Source/WTF/wtf/HashMap.h</a></li>
<li><a href="#trunkSourceWTFwtftextWTFStringh">trunk/Source/WTF/wtf/text/WTFString.h</a></li>
<li><a href="#trunkSourceWebKit2ChangeLog">trunk/Source/WebKit2/ChangeLog</a></li>
<li><a href="#trunkSourceWebKit2NetworkProcesscacheNetworkCacheStatisticsh">trunk/Source/WebKit2/NetworkProcess/cache/NetworkCacheStatistics.h</a></li>
<li><a href="#trunkSourceWebKit2NetworkProcesscacheNetworkCacheStatisticsCocoamm">trunk/Source/WebKit2/NetworkProcess/cache/NetworkCacheStatisticsCocoa.mm</a></li>
<li><a href="#trunkSourceWebKit2UIProcessCocoaWebProcessPoolCocoamm">trunk/Source/WebKit2/UIProcess/Cocoa/WebProcessPoolCocoa.mm</a></li>
</ul>
</div>
<div id="patch">
<h3>Diff</h3>
<a id="trunkSourceWTFChangeLog"></a>
<div class="modfile"><h4>Modified: trunk/Source/WTF/ChangeLog (181085 => 181086)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/WTF/ChangeLog 2015-03-05 18:57:46 UTC (rev 181085)
+++ trunk/Source/WTF/ChangeLog 2015-03-05 18:59:57 UTC (rev 181086)
</span><span class="lines">@@ -1,3 +1,22 @@
</span><ins>+2015-03-05 Chris Dumez <cdumez@apple.com>
+
+ NetworkCache efficacy logging is using too much CPU
+ https://bugs.webkit.org/show_bug.cgi?id=142186
+ <rdar://problem/19632080>
+
+ Reviewed by Antti Koivisto.
+
+ * wtf/HashMap.h:
+ (WTF::copyToVector):
+ Add copyToVector() utility function for HashMap that uses an std::pair<key, value>
+ as element type.
+
+ * wtf/text/WTFString.h:
+ (WTF::StringCapture::StringCapture):
+ (WTF::StringCapture::operator=):
+ Add default constructor and assignment operator to StringCapture so that
+ it can be used in a Vector.
+
</ins><span class="cx"> 2015-03-05 Antti Koivisto <antti@apple.com>
</span><span class="cx">
</span><span class="cx"> Support WorkQueue QOS classes on Mavericks
</span></span></pre></div>
<a id="trunkSourceWTFwtfHashMaph"></a>
<div class="modfile"><h4>Modified: trunk/Source/WTF/wtf/HashMap.h (181085 => 181086)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/WTF/wtf/HashMap.h 2015-03-05 18:57:46 UTC (rev 181085)
+++ trunk/Source/WTF/wtf/HashMap.h 2015-03-05 18:59:57 UTC (rev 181086)
</span><span class="lines">@@ -500,6 +500,19 @@
</span><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> template<typename T, typename U, typename V, typename W, typename X, typename Y>
</span><ins>+inline void copyToVector(const HashMap<T, U, V, W, X>& collection, Y& vector)
+{
+ typedef typename HashMap<T, U, V, W, X>::const_iterator iterator;
+
+ vector.resize(collection.size());
+
+ iterator it = collection.begin();
+ iterator end = collection.end();
+ for (unsigned i = 0; it != end; ++it, ++i)
+ vector[i] = { (*it).key, (*it).value };
+}
+
+template<typename T, typename U, typename V, typename W, typename X, typename Y>
</ins><span class="cx"> inline void copyKeysToVector(const HashMap<T, U, V, W, X>& collection, Y& vector)
</span><span class="cx"> {
</span><span class="cx"> typedef typename HashMap<T, U, V, W, X>::const_iterator::Keys iterator;
</span></span></pre></div>
<a id="trunkSourceWTFwtftextWTFStringh"></a>
<div class="modfile"><h4>Modified: trunk/Source/WTF/wtf/text/WTFString.h (181085 => 181086)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/WTF/wtf/text/WTFString.h 2015-03-05 18:57:46 UTC (rev 181085)
+++ trunk/Source/WTF/wtf/text/WTFString.h 2015-03-05 18:59:57 UTC (rev 181086)
</span><span class="lines">@@ -638,14 +638,16 @@
</span><span class="cx"> // FIXME: Remove when we can use C++14 initialized lambda capture: [string = string.isolatedCopy()].
</span><span class="cx"> class StringCapture {
</span><span class="cx"> public:
</span><del>- explicit StringCapture(const String& string) : m_string(string) { }
</del><ins>+ StringCapture() { }
+ StringCapture(const String& string) : m_string(string) { }
</ins><span class="cx"> explicit StringCapture(String&& string) : m_string(string) { }
</span><span class="cx"> StringCapture(const StringCapture& other) : m_string(other.m_string.isolatedCopy()) { }
</span><span class="cx"> const String& string() const { return m_string; }
</span><span class="cx"> String releaseString() { return WTF::move(m_string); }
</span><span class="cx">
</span><ins>+ void operator=(const StringCapture& other) { m_string = other.m_string.isolatedCopy(); }
+
</ins><span class="cx"> private:
</span><del>- void operator=(const StringCapture&) = delete;
</del><span class="cx"> String m_string;
</span><span class="cx"> };
</span><span class="cx">
</span></span></pre></div>
<a id="trunkSourceWebKit2ChangeLog"></a>
<div class="modfile"><h4>Modified: trunk/Source/WebKit2/ChangeLog (181085 => 181086)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/WebKit2/ChangeLog 2015-03-05 18:57:46 UTC (rev 181085)
+++ trunk/Source/WebKit2/ChangeLog 2015-03-05 18:59:57 UTC (rev 181086)
</span><span class="lines">@@ -1,3 +1,51 @@
</span><ins>+2015-03-05 Chris Dumez <cdumez@apple.com>
+
+ NetworkCache efficacy logging is using too much CPU
+ https://bugs.webkit.org/show_bug.cgi?id=142186
+ <rdar://problem/19632080>
+
+ Reviewed by Antti Koivisto.
+
+ NetworkCache efficacy logging was using too much CPU mostly due to
+ database writes. Logging was using ~11.3% of the NetworkProcess'
+ CPU usage (~9.2% for writes / ~1.3% for reads).
+
+ This patch buffers writes requests in memory and only writes them
+ to disk every 10 seconds. We are thus writing to this less frequently
+ and writing more at once, in a single SQL transaction. After this
+ change, efficacy logging only accounts for ~3.5% of the NetworkProcess'
+ CPU activity (1.5% for writes / 1.2% for reads).
+
+ Now that CPU usage is more acceptable, this patch re-enables the
+ network cache efficicy logging.
+
+ * NetworkProcess/cache/NetworkCacheStatistics.h:
+
+ * NetworkProcess/cache/NetworkCacheStatisticsCocoa.mm:
+ (WebKit::executeSQLCommand):
+ (WebKit::executeSQLStatement):
+ Call step() instead of executeCommand() because:
+ - The input statement is already prepared.
+ - step() does not finalize() the statement and thus we caller can
+ reuse it after calling reset().
+
+ (WebKit::NetworkCacheStatistics::NetworkCacheStatistics):
+ (WebKit::NetworkCacheStatistics::initialize):
+ (WebKit::NetworkCacheStatistics::bootstrapFromNetworkCache):
+ (WebKit::NetworkCacheStatistics::recordNotCachingResponse):
+ (WebKit::NetworkCacheStatistics::recordNotUsingCacheForRequest):
+ (WebKit::NetworkCacheStatistics::recordRetrievalFailure):
+ (WebKit::NetworkCacheStatistics::markAsRequested):
+ (WebKit::NetworkCacheStatistics::writeTimerFired):
+ (WebKit::NetworkCacheStatistics::queryWasEverRequested):
+ (WebKit::NetworkCacheStatistics::addHashesToDatabase):
+ (WebKit::NetworkCacheStatistics::addStoreDecisionsToDatabase):
+ (WebKit::NetworkCacheStatistics::addHashToDatabase): Deleted.
+
+ * UIProcess/Cocoa/WebProcessPoolCocoa.mm:
+ (WebKit::registerUserDefaultsIfNeeded):
+ Re-enable the network cache efficacy logging.
+
</ins><span class="cx"> 2015-03-05 Antti Koivisto <antti@apple.com>
</span><span class="cx">
</span><span class="cx"> Switch NetworkCacheStorage to WorkQueue abstraction
</span></span></pre></div>
<a id="trunkSourceWebKit2NetworkProcesscacheNetworkCacheStatisticsh"></a>
<div class="modfile"><h4>Modified: trunk/Source/WebKit2/NetworkProcess/cache/NetworkCacheStatistics.h (181085 => 181086)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/WebKit2/NetworkProcess/cache/NetworkCacheStatistics.h 2015-03-05 18:57:46 UTC (rev 181085)
+++ trunk/Source/WebKit2/NetworkProcess/cache/NetworkCacheStatistics.h 2015-03-05 18:59:57 UTC (rev 181086)
</span><span class="lines">@@ -32,6 +32,7 @@
</span><span class="cx"> #include "NetworkCacheKey.h"
</span><span class="cx"> #include "NetworkCacheStorage.h"
</span><span class="cx"> #include <WebCore/SQLiteDatabase.h>
</span><ins>+#include <WebCore/Timer.h>
</ins><span class="cx">
</span><span class="cx"> namespace WebCore {
</span><span class="cx"> class ResourceRequest;
</span><span class="lines">@@ -57,14 +58,18 @@
</span><span class="cx"> void bootstrapFromNetworkCache(const String& networkCachePath);
</span><span class="cx"> void shrinkIfNeeded();
</span><span class="cx">
</span><del>- void addHashToDatabase(const String& hash);
</del><ins>+ void addHashesToDatabase(const Vector<StringCapture>& hashes);
+ void addStoreDecisionsToDatabase(const Vector<std::pair<StringCapture, NetworkCache::StoreDecision>>&);
+ void writeTimerFired();
</ins><span class="cx">
</span><span class="cx"> typedef std::function<void (bool wasEverRequested, const Optional<NetworkCache::StoreDecision>&)> RequestedCompletionHandler;
</span><del>- void queryWasEverRequested(const String&, const RequestedCompletionHandler&);
</del><ins>+ enum class NeedUncachedReason { No, Yes };
+ void queryWasEverRequested(const String&, NeedUncachedReason, const RequestedCompletionHandler&);
</ins><span class="cx"> void markAsRequested(const String& hash);
</span><span class="cx">
</span><span class="cx"> struct EverRequestedQuery {
</span><span class="cx"> String hash;
</span><ins>+ bool needUncachedReason;
</ins><span class="cx"> RequestedCompletionHandler completionHandler;
</span><span class="cx"> };
</span><span class="cx">
</span><span class="lines">@@ -75,6 +80,9 @@
</span><span class="cx"> #endif
</span><span class="cx"> mutable HashSet<std::unique_ptr<const EverRequestedQuery>> m_activeQueries;
</span><span class="cx"> WebCore::SQLiteDatabase m_database;
</span><ins>+ HashSet<String> m_hashesToAdd;
+ HashMap<String, NetworkCache::StoreDecision> m_storeDecisionsToAdd;
+ WebCore::Timer m_writeTimer;
</ins><span class="cx"> };
</span><span class="cx">
</span><span class="cx"> } // namespace WebKit
</span></span></pre></div>
<a id="trunkSourceWebKit2NetworkProcesscacheNetworkCacheStatisticsCocoamm"></a>
<div class="modfile"><h4>Modified: trunk/Source/WebKit2/NetworkProcess/cache/NetworkCacheStatisticsCocoa.mm (181085 => 181086)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/WebKit2/NetworkProcess/cache/NetworkCacheStatisticsCocoa.mm 2015-03-05 18:57:46 UTC (rev 181085)
+++ trunk/Source/WebKit2/NetworkProcess/cache/NetworkCacheStatisticsCocoa.mm 2015-03-05 18:59:57 UTC (rev 181086)
</span><span class="lines">@@ -43,6 +43,7 @@
</span><span class="cx"> namespace WebKit {
</span><span class="cx">
</span><span class="cx"> static const char* networkCacheStatisticsDatabaseName = "WebKitCacheStatistics.db";
</span><ins>+static const std::chrono::milliseconds mininumWriteInterval = std::chrono::milliseconds(10000);
</ins><span class="cx">
</span><span class="cx"> static bool executeSQLCommand(WebCore::SQLiteDatabase& database, const String& sql)
</span><span class="cx"> {
</span><span class="lines">@@ -63,11 +64,12 @@
</span><span class="cx"> ASSERT(WebCore::SQLiteDatabaseTracker::hasTransactionInProgress());
</span><span class="cx"> ASSERT(statement.database().isOpen());
</span><span class="cx">
</span><del>- bool result = statement.executeCommand();
- if (!result)
</del><ins>+ if (statement.step() != WebCore::SQLResultDone) {
</ins><span class="cx"> LOG_ERROR("Network cache statistics: failed to execute statement \"%s\" error \"%s\"", statement.query().utf8().data(), statement.database().lastErrorMsg());
</span><ins>+ return false;
+ }
</ins><span class="cx">
</span><del>- return result;
</del><ins>+ return true;
</ins><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> std::unique_ptr<NetworkCacheStatistics> NetworkCacheStatistics::open(const String& cachePath)
</span><span class="lines">@@ -80,6 +82,7 @@
</span><span class="cx">
</span><span class="cx"> NetworkCacheStatistics::NetworkCacheStatistics(const String& databasePath)
</span><span class="cx"> : m_backgroundIOQueue(adoptDispatch(dispatch_queue_create("com.apple.WebKit.Cache.Statistics.Background", DISPATCH_QUEUE_SERIAL)))
</span><ins>+ , m_writeTimer(*this, &NetworkCacheStatistics::writeTimerFired)
</ins><span class="cx"> {
</span><span class="cx"> dispatch_set_target_queue(m_backgroundIOQueue.get(), dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_BACKGROUND, 0));
</span><span class="cx">
</span><span class="lines">@@ -141,14 +144,18 @@
</span><span class="cx">
</span><span class="cx"> LOG(NetworkCache, "(NetworkProcess) Bootstrapping the network cache statistics database from the network cache...");
</span><span class="cx">
</span><ins>+ Vector<StringCapture> hashes;
+ traverseCacheFiles(networkCachePath, [&hashes](const String& hash, const String&) {
+ hashes.append(hash);
+ });
+
</ins><span class="cx"> WebCore::SQLiteTransactionInProgressAutoCounter transactionCounter;
</span><del>- WebCore::SQLiteTransaction bootstrapTransaction(m_database);
- bootstrapTransaction.begin();
</del><ins>+ WebCore::SQLiteTransaction writeTransaction(m_database);
+ writeTransaction.begin();
</ins><span class="cx">
</span><del>- traverseCacheFiles(networkCachePath, [this](const String& hash, const String&) {
- addHashToDatabase(hash);
- });
- bootstrapTransaction.commit();
</del><ins>+ addHashesToDatabase(hashes);
+
+ writeTransaction.commit();
</ins><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> void NetworkCacheStatistics::shrinkIfNeeded()
</span><span class="lines">@@ -174,18 +181,9 @@
</span><span class="cx"> {
</span><span class="cx"> ASSERT(storeDecision != NetworkCache::StoreDecision::Yes);
</span><span class="cx">
</span><del>- StringCapture hashCapture(key.hashAsString());
- dispatch_async(m_backgroundIOQueue.get(), [this, hashCapture, storeDecision] {
- if (!m_database.isOpen())
- return;
- WebCore::SQLiteStatement statement(m_database, ASCIILiteral("INSERT OR REPLACE INTO UncachedReason (hash, reason) VALUES (?, ?)"));
- if (statement.prepare() != WebCore::SQLResultOk)
- return;
-
- statement.bindText(1, hashCapture.string());
- statement.bindInt(2, static_cast<int>(storeDecision));
- executeSQLStatement(statement);
- });
</del><ins>+ m_storeDecisionsToAdd.set(key.hashAsString(), storeDecision);
+ if (!m_writeTimer.isActive())
+ m_writeTimer.startOneShot(mininumWriteInterval);
</ins><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> static String retrieveDecisionToDiagnosticKey(NetworkCache::RetrieveDecision retrieveDecision)
</span><span class="lines">@@ -212,7 +210,7 @@
</span><span class="cx">
</span><span class="cx"> String hash = key.hashAsString();
</span><span class="cx"> WebCore::URL requestURL = request.url();
</span><del>- queryWasEverRequested(hash, [this, hash, requestURL, webPageID, retrieveDecision](bool wasEverRequested, const Optional<NetworkCache::StoreDecision>&) {
</del><ins>+ queryWasEverRequested(hash, NeedUncachedReason::No, [this, hash, requestURL, webPageID, retrieveDecision](bool wasEverRequested, const Optional<NetworkCache::StoreDecision>&) {
</ins><span class="cx"> if (wasEverRequested) {
</span><span class="cx"> String diagnosticKey = retrieveDecisionToDiagnosticKey(retrieveDecision);
</span><span class="cx"> LOG(NetworkCache, "(NetworkProcess) webPageID %llu: %s was previously requested but we are not using the cache, reason: %s", webPageID, requestURL.string().ascii().data(), diagnosticKey.utf8().data());
</span><span class="lines">@@ -246,7 +244,7 @@
</span><span class="cx"> {
</span><span class="cx"> String hash = key.hashAsString();
</span><span class="cx"> WebCore::URL requestURL = request.url();
</span><del>- queryWasEverRequested(hash, [this, hash, requestURL, webPageID](bool wasPreviouslyRequested, const Optional<NetworkCache::StoreDecision>& storeDecision) {
</del><ins>+ queryWasEverRequested(hash, NeedUncachedReason::Yes, [this, hash, requestURL, webPageID](bool wasPreviouslyRequested, const Optional<NetworkCache::StoreDecision>& storeDecision) {
</ins><span class="cx"> if (wasPreviouslyRequested) {
</span><span class="cx"> String diagnosticKey = storeDecisionToDiagnosticKey(storeDecision.value());
</span><span class="cx"> LOG(NetworkCache, "(NetworkProcess) webPageID %llu: %s was previously request but is not in the cache, reason: %s", webPageID, requestURL.string().ascii().data(), diagnosticKey.utf8().data());
</span><span class="lines">@@ -290,36 +288,79 @@
</span><span class="cx"> {
</span><span class="cx"> ASSERT(RunLoop::isMain());
</span><span class="cx">
</span><del>- StringCapture hashCapture(hash);
- dispatch_async(m_backgroundIOQueue.get(), [this, hashCapture] {
</del><ins>+ m_hashesToAdd.add(hash);
+ if (!m_writeTimer.isActive())
+ m_writeTimer.startOneShot(mininumWriteInterval);
+}
+
+void NetworkCacheStatistics::writeTimerFired()
+{
+ ASSERT(RunLoop::isMain());
+
+ Vector<StringCapture> hashesToAdd;
+ copyToVector(m_hashesToAdd, hashesToAdd);
+ m_hashesToAdd.clear();
+
+ Vector<std::pair<StringCapture, NetworkCache::StoreDecision>> storeDecisionsToAdd;
+ copyToVector(m_storeDecisionsToAdd, storeDecisionsToAdd);
+ m_storeDecisionsToAdd.clear();
+
+ shrinkIfNeeded();
+
+ dispatch_async(m_backgroundIOQueue.get(), [this, hashesToAdd, storeDecisionsToAdd] {
+ if (!m_database.isOpen())
+ return;
+
</ins><span class="cx"> WebCore::SQLiteTransactionInProgressAutoCounter transactionCounter;
</span><del>- if (m_database.isOpen())
- addHashToDatabase(hashCapture.string());
</del><ins>+ WebCore::SQLiteTransaction writeTransaction(m_database);
+ writeTransaction.begin();
+
+ addHashesToDatabase(hashesToAdd);
+ addStoreDecisionsToDatabase(storeDecisionsToAdd);
+
+ writeTransaction.commit();
</ins><span class="cx"> });
</span><del>-
- shrinkIfNeeded();
</del><span class="cx"> }
</span><span class="cx">
</span><del>-void NetworkCacheStatistics::queryWasEverRequested(const String& hash, const RequestedCompletionHandler& completionHandler)
</del><ins>+void NetworkCacheStatistics::queryWasEverRequested(const String& hash, NeedUncachedReason needUncachedReason, const RequestedCompletionHandler& completionHandler)
</ins><span class="cx"> {
</span><span class="cx"> ASSERT(RunLoop::isMain());
</span><span class="cx">
</span><del>- auto everRequestedQuery = std::make_unique<EverRequestedQuery>(EverRequestedQuery { hash, completionHandler });
</del><ins>+ // Query pending writes first.
+ bool wasAlreadyRequested = m_hashesToAdd.contains(hash);
+ if (wasAlreadyRequested && needUncachedReason == NeedUncachedReason::No) {
+ completionHandler(true, Nullopt);
+ return;
+ }
+ if (needUncachedReason == NeedUncachedReason::Yes && m_storeDecisionsToAdd.contains(hash)) {
+ completionHandler(true, m_storeDecisionsToAdd.get(hash));
+ return;
+ }
+
+ // Query the database.
+ auto everRequestedQuery = std::make_unique<EverRequestedQuery>(EverRequestedQuery { hash, needUncachedReason == NeedUncachedReason::Yes, completionHandler });
</ins><span class="cx"> auto& query = *everRequestedQuery;
</span><span class="cx"> m_activeQueries.add(WTF::move(everRequestedQuery));
</span><del>- dispatch_async(m_backgroundIOQueue.get(), [this, &query] {
</del><ins>+ dispatch_async(m_backgroundIOQueue.get(), [this, wasAlreadyRequested, &query] () mutable {
</ins><span class="cx"> WebCore::SQLiteTransactionInProgressAutoCounter transactionCounter;
</span><del>- bool wasAlreadyRequested = false;
</del><span class="cx"> Optional<NetworkCache::StoreDecision> storeDecision;
</span><span class="cx"> if (m_database.isOpen()) {
</span><del>- WebCore::SQLiteStatement statement(m_database, ASCIILiteral("SELECT AlreadyRequested.hash, reason FROM AlreadyRequested LEFT OUTER JOIN UncachedReason ON AlreadyRequested.hash=UncachedReason.hash WHERE AlreadyRequested.hash=?"));
- if (statement.prepare() == WebCore::SQLResultOk) {
- statement.bindText(1, query.hash);
- if (statement.step() == WebCore::SQLResultRow) {
- wasAlreadyRequested = true;
- storeDecision = statement.isColumnNull(1) ? NetworkCache::StoreDecision::Yes : static_cast<NetworkCache::StoreDecision>(statement.getColumnInt(1));
</del><ins>+ if (!wasAlreadyRequested) {
+ WebCore::SQLiteStatement statement(m_database, ASCIILiteral("SELECT hash FROM AlreadyRequested WHERE hash=?"));
+ if (statement.prepare() == WebCore::SQLResultOk) {
+ statement.bindText(1, query.hash);
+ wasAlreadyRequested = (statement.step() == WebCore::SQLResultRow);
</ins><span class="cx"> }
</span><span class="cx"> }
</span><ins>+ if (wasAlreadyRequested && query.needUncachedReason) {
+ WebCore::SQLiteStatement statement(m_database, ASCIILiteral("SELECT reason FROM UncachedReason WHERE hash=?"));
+ storeDecision = NetworkCache::StoreDecision::Yes;
+ if (statement.prepare() == WebCore::SQLResultOk) {
+ statement.bindText(1, query.hash);
+ if (statement.step() == WebCore::SQLResultRow)
+ storeDecision = static_cast<NetworkCache::StoreDecision>(statement.getColumnInt(0));
+ }
+ }
</ins><span class="cx"> }
</span><span class="cx"> dispatch_async(dispatch_get_main_queue(), [this, &query, wasAlreadyRequested, storeDecision] {
</span><span class="cx"> query.completionHandler(wasAlreadyRequested, storeDecision);
</span><span class="lines">@@ -345,7 +386,7 @@
</span><span class="cx"> });
</span><span class="cx"> }
</span><span class="cx">
</span><del>-void NetworkCacheStatistics::addHashToDatabase(const String& hash)
</del><ins>+void NetworkCacheStatistics::addHashesToDatabase(const Vector<StringCapture>& hashes)
</ins><span class="cx"> {
</span><span class="cx"> ASSERT(!RunLoop::isMain());
</span><span class="cx"> ASSERT(WebCore::SQLiteDatabaseTracker::hasTransactionInProgress());
</span><span class="lines">@@ -355,11 +396,30 @@
</span><span class="cx"> if (statement.prepare() != WebCore::SQLResultOk)
</span><span class="cx"> return;
</span><span class="cx">
</span><del>- statement.bindText(1, hash);
- if (!executeSQLStatement(statement))
</del><ins>+ for (auto& hash : hashes) {
+ statement.bindText(1, hash.string());
+ if (executeSQLStatement(statement))
+ ++m_approximateEntryCount;
+ statement.reset();
+ }
+}
+
+void NetworkCacheStatistics::addStoreDecisionsToDatabase(const Vector<std::pair<StringCapture, NetworkCache::StoreDecision>>& storeDecisions)
+{
+ ASSERT(!RunLoop::isMain());
+ ASSERT(WebCore::SQLiteDatabaseTracker::hasTransactionInProgress());
+ ASSERT(m_database.isOpen());
+
+ WebCore::SQLiteStatement statement(m_database, ASCIILiteral("INSERT OR REPLACE INTO UncachedReason (hash, reason) VALUES (?, ?)"));
+ if (statement.prepare() != WebCore::SQLResultOk)
</ins><span class="cx"> return;
</span><span class="cx">
</span><del>- ++m_approximateEntryCount;
</del><ins>+ for (auto& pair : storeDecisions) {
+ statement.bindText(1, pair.first.string());
+ statement.bindInt(2, static_cast<int>(pair.second));
+ executeSQLStatement(statement);
+ statement.reset();
+ }
</ins><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> } // namespace WebKit
</span></span></pre></div>
<a id="trunkSourceWebKit2UIProcessCocoaWebProcessPoolCocoamm"></a>
<div class="modfile"><h4>Modified: trunk/Source/WebKit2/UIProcess/Cocoa/WebProcessPoolCocoa.mm (181085 => 181086)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/WebKit2/UIProcess/Cocoa/WebProcessPoolCocoa.mm 2015-03-05 18:57:46 UTC (rev 181085)
+++ trunk/Source/WebKit2/UIProcess/Cocoa/WebProcessPoolCocoa.mm 2015-03-05 18:59:57 UTC (rev 181086)
</span><span class="lines">@@ -107,7 +107,7 @@
</span><span class="cx">
</span><span class="cx"> #if ENABLE(NETWORK_CACHE)
</span><span class="cx"> [registrationDictionary setObject:[NSNumber numberWithBool:YES] forKey:WebKitNetworkCacheEnabledDefaultsKey];
</span><del>- [registrationDictionary setObject:[NSNumber numberWithBool:NO] forKey:WebKitNetworkCacheEfficacyLoggingEnabledDefaultsKey];
</del><ins>+ [registrationDictionary setObject:[NSNumber numberWithBool:YES] forKey:WebKitNetworkCacheEfficacyLoggingEnabledDefaultsKey];
</ins><span class="cx"> #endif
</span><span class="cx">
</span><span class="cx"> [[NSUserDefaults standardUserDefaults] registerDefaults:registrationDictionary];
</span></span></pre>
</div>
</div>
</body>
</html>