<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
"http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head><meta http-equiv="content-type" content="text/html; charset=utf-8" />
<title>[183817] trunk</title>
</head>
<body>
<style type="text/css"><!--
#msg dl.meta { border: 1px #006 solid; background: #369; padding: 6px; color: #fff; }
#msg dl.meta dt { float: left; width: 6em; font-weight: bold; }
#msg dt:after { content:':';}
#msg dl, #msg dt, #msg ul, #msg li, #header, #footer, #logmsg { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt; }
#msg dl a { font-weight: bold}
#msg dl a:link { color:#fc3; }
#msg dl a:active { color:#ff0; }
#msg dl a:visited { color:#cc6; }
h3 { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt; font-weight: bold; }
#msg pre { overflow: auto; background: #ffc; border: 1px #fa0 solid; padding: 6px; }
#logmsg { background: #ffc; border: 1px #fa0 solid; padding: 1em 1em 0 1em; }
#logmsg p, #logmsg pre, #logmsg blockquote { margin: 0 0 1em 0; }
#logmsg p, #logmsg li, #logmsg dt, #logmsg dd { line-height: 14pt; }
#logmsg h1, #logmsg h2, #logmsg h3, #logmsg h4, #logmsg h5, #logmsg h6 { margin: .5em 0; }
#logmsg h1:first-child, #logmsg h2:first-child, #logmsg h3:first-child, #logmsg h4:first-child, #logmsg h5:first-child, #logmsg h6:first-child { margin-top: 0; }
#logmsg ul, #logmsg ol { padding: 0; list-style-position: inside; margin: 0 0 0 1em; }
#logmsg ul { text-indent: -1em; padding-left: 1em; }#logmsg ol { text-indent: -1.5em; padding-left: 1.5em; }
#logmsg > ul, #logmsg > ol { margin: 0 0 1em 0; }
#logmsg pre { background: #eee; padding: 1em; }
#logmsg blockquote { border: 1px solid #fa0; border-left-width: 10px; padding: 1em 1em 0 1em; background: white;}
#logmsg dl { margin: 0; }
#logmsg dt { font-weight: bold; }
#logmsg dd { margin: 0; padding: 0 0 0.5em 0; }
#logmsg dd:before { content:'\00bb';}
#logmsg table { border-spacing: 0px; border-collapse: collapse; border-top: 4px solid #fa0; border-bottom: 1px solid #fa0; background: #fff; }
#logmsg table th { text-align: left; font-weight: normal; padding: 0.2em 0.5em; border-top: 1px dotted #fa0; }
#logmsg table td { text-align: right; border-top: 1px dotted #fa0; padding: 0.2em 0.5em; }
#logmsg table thead th { text-align: center; border-bottom: 1px solid #fa0; }
#logmsg table th.Corner { text-align: left; }
#logmsg hr { border: none 0; border-top: 2px dashed #fa0; height: 1px; }
#header, #footer { color: #fff; background: #636; border: 1px #300 solid; padding: 6px; }
#patch { width: 100%; }
#patch h4 {font-family: verdana,arial,helvetica,sans-serif;font-size:10pt;padding:8px;background:#369;color:#fff;margin:0;}
#patch .propset h4, #patch .binary h4 {margin:0;}
#patch pre {padding:0;line-height:1.2em;margin:0;}
#patch .diff {width:100%;background:#eee;padding: 0 0 10px 0;overflow:auto;}
#patch .propset .diff, #patch .binary .diff {padding:10px 0;}
#patch span {display:block;padding:0 10px;}
#patch .modfile, #patch .addfile, #patch .delfile, #patch .propset, #patch .binary, #patch .copfile {border:1px solid #ccc;margin:10px 0;}
#patch ins {background:#dfd;text-decoration:none;display:block;padding:0 10px;}
#patch del {background:#fdd;text-decoration:none;display:block;padding:0 10px;}
#patch .lines, .info {color:#888;background:#fff;}
--></style>
<div id="msg">
<dl class="meta">
<dt>Revision</dt> <dd><a href="http://trac.webkit.org/projects/webkit/changeset/183817">183817</a></dd>
<dt>Author</dt> <dd>achristensen@apple.com</dd>
<dt>Date</dt> <dd>2015-05-05 10:12:36 -0700 (Tue, 05 May 2015)</dd>
</dl>
<h3>Log Message</h3>
<pre>[Content Extensions] Use less memory when writing byte code to file
https://bugs.webkit.org/show_bug.cgi?id=144602
Reviewed by Darin Adler.
Source/WebCore:
* contentextensions/ContentExtensionCompiler.cpp:
(WebCore::ContentExtensions::compileRuleList):
* contentextensions/ContentExtensionCompiler.h:
Compile one DFA at a time so we don't need to keep all the bytecode in memory at the same time.
* contentextensions/DFABytecodeInterpreter.cpp:
(WebCore::ContentExtensions::DFABytecodeInterpreter::interpret):
Jumps are now relative to the current DFA because we don't know about other DFAs that
have been compiling when linking the DFA bytecode. This will also make the DFA bytecode
easier to minimize because more of the values are small in the DFAs after the first DFA.
* platform/FileSystem.h:
Source/WebKit2:
* UIProcess/API/APIUserContentExtensionStore.cpp:
(API::decodeContentExtensionMetaData):
(API::writeDataToFile):
(API::compiledToFile):
(API::createExtension):
Compile and write each DFA to file, then come back and write the header when finalizing.
Also don't copy the DFA bytecode. This way, we don't need to keep more than one DFA's
bytecode in memory at a time.
Tools:
* TestWebKitAPI/Tests/WebCore/ContentExtensions.cpp:</pre>
<h3>Modified Paths</h3>
<ul>
<li><a href="#trunkSourceWebCoreChangeLog">trunk/Source/WebCore/ChangeLog</a></li>
<li><a href="#trunkSourceWebCorecontentextensionsContentExtensionCompilercpp">trunk/Source/WebCore/contentextensions/ContentExtensionCompiler.cpp</a></li>
<li><a href="#trunkSourceWebCorecontentextensionsContentExtensionCompilerh">trunk/Source/WebCore/contentextensions/ContentExtensionCompiler.h</a></li>
<li><a href="#trunkSourceWebCorecontentextensionsDFABytecodeInterpretercpp">trunk/Source/WebCore/contentextensions/DFABytecodeInterpreter.cpp</a></li>
<li><a href="#trunkSourceWebCoreplatformFileSystemh">trunk/Source/WebCore/platform/FileSystem.h</a></li>
<li><a href="#trunkSourceWebKit2ChangeLog">trunk/Source/WebKit2/ChangeLog</a></li>
<li><a href="#trunkSourceWebKit2UIProcessAPIAPIUserContentExtensionStorecpp">trunk/Source/WebKit2/UIProcess/API/APIUserContentExtensionStore.cpp</a></li>
<li><a href="#trunkToolsChangeLog">trunk/Tools/ChangeLog</a></li>
<li><a href="#trunkToolsTestWebKitAPITestsWebCoreContentExtensionscpp">trunk/Tools/TestWebKitAPI/Tests/WebCore/ContentExtensions.cpp</a></li>
</ul>
</div>
<div id="patch">
<h3>Diff</h3>
<a id="trunkSourceWebCoreChangeLog"></a>
<div class="modfile"><h4>Modified: trunk/Source/WebCore/ChangeLog (183816 => 183817)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/WebCore/ChangeLog 2015-05-05 16:59:31 UTC (rev 183816)
+++ trunk/Source/WebCore/ChangeLog 2015-05-05 17:12:36 UTC (rev 183817)
</span><span class="lines">@@ -1,3 +1,21 @@
</span><ins>+2015-05-04 Alex Christensen <achristensen@webkit.org>
+
+ [Content Extensions] Use less memory when writing byte code to file
+ https://bugs.webkit.org/show_bug.cgi?id=144602
+
+ Reviewed by Darin Adler.
+
+ * contentextensions/ContentExtensionCompiler.cpp:
+ (WebCore::ContentExtensions::compileRuleList):
+ * contentextensions/ContentExtensionCompiler.h:
+ Compile one DFA at a time so we don't need to keep all the bytecode in memory at the same time.
+ * contentextensions/DFABytecodeInterpreter.cpp:
+ (WebCore::ContentExtensions::DFABytecodeInterpreter::interpret):
+ Jumps are now relative to the current DFA because we don't know about other DFAs that
+ have been compiling when linking the DFA bytecode. This will also make the DFA bytecode
+ easier to minimize because more of the values are small in the DFAs after the first DFA.
+ * platform/FileSystem.h:
+
</ins><span class="cx"> 2015-05-05 Csaba Osztrogonác <ossy@webkit.org>
</span><span class="cx">
</span><span class="cx"> Unreviewed, speculative WinCairo buildfix after r183807.
</span></span></pre></div>
<a id="trunkSourceWebCorecontentextensionsContentExtensionCompilercpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/WebCore/contentextensions/ContentExtensionCompiler.cpp (183816 => 183817)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/WebCore/contentextensions/ContentExtensionCompiler.cpp 2015-05-05 16:59:31 UTC (rev 183816)
+++ trunk/Source/WebCore/contentextensions/ContentExtensionCompiler.cpp 2015-05-05 17:12:36 UTC (rev 183817)
</span><span class="lines">@@ -197,7 +197,6 @@
</span><span class="cx"> double totalNFAToByteCodeBuildTimeStart = monotonicallyIncreasingTime();
</span><span class="cx"> #endif
</span><span class="cx">
</span><del>- Vector<DFABytecode> bytecode;
</del><span class="cx"> bool firstNFASeen = false;
</span><span class="cx"> combinedURLFilters.processNFAs([&](NFA&& nfa) {
</span><span class="cx"> #if CONTENT_EXTENSIONS_STATE_MACHINE_DEBUGGING
</span><span class="lines">@@ -237,8 +236,11 @@
</span><span class="cx"> addUniversalActionsToDFA(dfa, universalActionLocations);
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+ Vector<DFABytecode> bytecode;
</ins><span class="cx"> DFABytecodeCompiler compiler(dfa, bytecode);
</span><span class="cx"> compiler.compile();
</span><ins>+ LOG_LARGE_STRUCTURES(bytecode, bytecode.capacity() * sizeof(uint8_t));
+ client.writeBytecode(WTF::move(bytecode));
</ins><span class="cx">
</span><span class="cx"> firstNFASeen = true;
</span><span class="cx"> });
</span><span class="lines">@@ -252,8 +254,11 @@
</span><span class="cx">
</span><span class="cx"> addUniversalActionsToDFA(dummyDFA, universalActionLocations);
</span><span class="cx">
</span><ins>+ Vector<DFABytecode> bytecode;
</ins><span class="cx"> DFABytecodeCompiler compiler(dummyDFA, bytecode);
</span><span class="cx"> compiler.compile();
</span><ins>+ LOG_LARGE_STRUCTURES(bytecode, bytecode.capacity() * sizeof(uint8_t));
+ client.writeBytecode(WTF::move(bytecode));
</ins><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> // FIXME: combinedURLFilters should be cleared incrementally as it is processing NFAs.
</span><span class="lines">@@ -268,9 +273,7 @@
</span><span class="cx"> dataLogF(" Bytecode size %zu\n", bytecode.size());
</span><span class="cx"> #endif
</span><span class="cx">
</span><del>- LOG_LARGE_STRUCTURES(bytecode, bytecode.capacity() * sizeof(uint8_t));
- client.writeBytecode(WTF::move(bytecode));
- bytecode.clear();
</del><ins>+ client.finalize();
</ins><span class="cx">
</span><span class="cx"> return { };
</span><span class="cx"> }
</span></span></pre></div>
<a id="trunkSourceWebCorecontentextensionsContentExtensionCompilerh"></a>
<div class="modfile"><h4>Modified: trunk/Source/WebCore/contentextensions/ContentExtensionCompiler.h (183816 => 183817)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/WebCore/contentextensions/ContentExtensionCompiler.h 2015-05-05 16:59:31 UTC (rev 183816)
+++ trunk/Source/WebCore/contentextensions/ContentExtensionCompiler.h 2015-05-05 17:12:36 UTC (rev 183817)
</span><span class="lines">@@ -47,6 +47,7 @@
</span><span class="cx">
</span><span class="cx"> virtual void writeBytecode(Vector<DFABytecode>&&) = 0;
</span><span class="cx"> virtual void writeActions(Vector<SerializedActionByte>&&) = 0;
</span><ins>+ virtual void finalize() = 0;
</ins><span class="cx"> };
</span><span class="cx">
</span><span class="cx"> WEBCORE_EXPORT std::error_code compileRuleList(ContentExtensionCompilationClient&, const String&);
</span></span></pre></div>
<a id="trunkSourceWebCorecontentextensionsDFABytecodeInterpretercpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/WebCore/contentextensions/DFABytecodeInterpreter.cpp (183816 => 183817)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/WebCore/contentextensions/DFABytecodeInterpreter.cpp 2015-05-05 16:59:31 UTC (rev 183816)
+++ trunk/Source/WebCore/contentextensions/DFABytecodeInterpreter.cpp 2015-05-05 17:12:36 UTC (rev 183817)
</span><span class="lines">@@ -114,7 +114,7 @@
</span><span class="cx"> if (programCounter >= m_bytecodeLength)
</span><span class="cx"> return actions;
</span><span class="cx"> } else {
</span><del>- ASSERT_WITH_MESSAGE(static_cast<DFABytecodeInstruction>(m_bytecode[programCounter]) != DFABytecodeInstruction::AppendAction
</del><ins>+ ASSERT_WITH_MESSAGE(static_cast<DFABytecodeInstruction>(m_bytecode[programCounter]) != DFABytecodeInstruction::AppendAction
</ins><span class="cx"> && static_cast<DFABytecodeInstruction>(m_bytecode[programCounter]) != DFABytecodeInstruction::TestFlagsAndAppendAction,
</span><span class="cx"> "Triggers that match everything should only be in the first DFA.");
</span><span class="cx"> }
</span><span class="lines">@@ -137,7 +137,7 @@
</span><span class="cx"> // Check to see if the next character in the url is the value stored with the bytecode.
</span><span class="cx"> char character = url[urlIndex];
</span><span class="cx"> if (character == getBits<uint8_t>(m_bytecode, m_bytecodeLength, programCounter + sizeof(DFABytecode), m_pagesUsed)) {
</span><del>- programCounter = getBits<unsigned>(m_bytecode, m_bytecodeLength, programCounter + sizeof(DFABytecode) + sizeof(uint8_t), m_pagesUsed);
</del><ins>+ programCounter = dfaStart + getBits<unsigned>(m_bytecode, m_bytecodeLength, programCounter + sizeof(DFABytecode) + sizeof(uint8_t), m_pagesUsed);
</ins><span class="cx"> if (!character)
</span><span class="cx"> urlIndexIsAfterEndOfString = true;
</span><span class="cx"> urlIndex++; // This represents an edge in the DFA.
</span><span class="lines">@@ -153,7 +153,7 @@
</span><span class="cx"> // Check to see if the next character in the url is the value stored with the bytecode.
</span><span class="cx"> char character = toASCIILower(url[urlIndex]);
</span><span class="cx"> if (character == getBits<uint8_t>(m_bytecode, m_bytecodeLength, programCounter + sizeof(DFABytecode), m_pagesUsed)) {
</span><del>- programCounter = getBits<unsigned>(m_bytecode, m_bytecodeLength, programCounter + sizeof(DFABytecode) + sizeof(uint8_t), m_pagesUsed);
</del><ins>+ programCounter = dfaStart + getBits<unsigned>(m_bytecode, m_bytecodeLength, programCounter + sizeof(DFABytecode) + sizeof(uint8_t), m_pagesUsed);
</ins><span class="cx"> if (!character)
</span><span class="cx"> urlIndexIsAfterEndOfString = true;
</span><span class="cx"> urlIndex++; // This represents an edge in the DFA.
</span><span class="lines">@@ -169,7 +169,7 @@
</span><span class="cx"> char character = url[urlIndex];
</span><span class="cx"> if (character >= getBits<uint8_t>(m_bytecode, m_bytecodeLength, programCounter + sizeof(DFABytecode), m_pagesUsed)
</span><span class="cx"> && character <= getBits<uint8_t>(m_bytecode, m_bytecodeLength, programCounter + sizeof(DFABytecode) + sizeof(uint8_t), m_pagesUsed)) {
</span><del>- programCounter = getBits<unsigned>(m_bytecode, m_bytecodeLength, programCounter + sizeof(DFABytecode) + sizeof(uint8_t) + sizeof(uint8_t), m_pagesUsed);
</del><ins>+ programCounter = dfaStart + getBits<unsigned>(m_bytecode, m_bytecodeLength, programCounter + sizeof(DFABytecode) + sizeof(uint8_t) + sizeof(uint8_t), m_pagesUsed);
</ins><span class="cx"> if (!character)
</span><span class="cx"> urlIndexIsAfterEndOfString = true;
</span><span class="cx"> urlIndex++; // This represents an edge in the DFA.
</span><span class="lines">@@ -185,7 +185,7 @@
</span><span class="cx"> char character = toASCIILower(url[urlIndex]);
</span><span class="cx"> if (character >= getBits<uint8_t>(m_bytecode, m_bytecodeLength, programCounter + sizeof(DFABytecode), m_pagesUsed)
</span><span class="cx"> && character <= getBits<uint8_t>(m_bytecode, m_bytecodeLength, programCounter + sizeof(DFABytecode) + sizeof(uint8_t), m_pagesUsed)) {
</span><del>- programCounter = getBits<unsigned>(m_bytecode, m_bytecodeLength, programCounter + sizeof(DFABytecode) + sizeof(uint8_t) + sizeof(uint8_t), m_pagesUsed);
</del><ins>+ programCounter = dfaStart + getBits<unsigned>(m_bytecode, m_bytecodeLength, programCounter + sizeof(DFABytecode) + sizeof(uint8_t) + sizeof(uint8_t), m_pagesUsed);
</ins><span class="cx"> if (!character)
</span><span class="cx"> urlIndexIsAfterEndOfString = true;
</span><span class="cx"> urlIndex++; // This represents an edge in the DFA.
</span><span class="lines">@@ -198,7 +198,7 @@
</span><span class="cx"> if (!url[urlIndex] || urlIndexIsAfterEndOfString)
</span><span class="cx"> goto nextDFA;
</span><span class="cx">
</span><del>- programCounter = getBits<unsigned>(m_bytecode, m_bytecodeLength, programCounter + sizeof(DFABytecode), m_pagesUsed);
</del><ins>+ programCounter = dfaStart + getBits<unsigned>(m_bytecode, m_bytecodeLength, programCounter + sizeof(DFABytecode), m_pagesUsed);
</ins><span class="cx"> urlIndex++; // This represents an edge in the DFA.
</span><span class="cx"> break;
</span><span class="cx">
</span></span></pre></div>
<a id="trunkSourceWebCoreplatformFileSystemh"></a>
<div class="modfile"><h4>Modified: trunk/Source/WebCore/platform/FileSystem.h (183816 => 183817)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/WebCore/platform/FileSystem.h 2015-05-05 16:59:31 UTC (rev 183816)
+++ trunk/Source/WebCore/platform/FileSystem.h 2015-05-05 17:12:36 UTC (rev 183817)
</span><span class="lines">@@ -168,7 +168,7 @@
</span><span class="cx"> WEBCORE_EXPORT PlatformFileHandle openFile(const String& path, FileOpenMode);
</span><span class="cx"> WEBCORE_EXPORT void closeFile(PlatformFileHandle&);
</span><span class="cx"> // Returns the resulting offset from the beginning of the file if successful, -1 otherwise.
</span><del>-long long seekFile(PlatformFileHandle, long long offset, FileSeekOrigin);
</del><ins>+WEBCORE_EXPORT long long seekFile(PlatformFileHandle, long long offset, FileSeekOrigin);
</ins><span class="cx"> bool truncateFile(PlatformFileHandle, long long offset);
</span><span class="cx"> // Returns number of bytes actually read if successful, -1 otherwise.
</span><span class="cx"> WEBCORE_EXPORT int writeToFile(PlatformFileHandle, const char* data, int length);
</span></span></pre></div>
<a id="trunkSourceWebKit2ChangeLog"></a>
<div class="modfile"><h4>Modified: trunk/Source/WebKit2/ChangeLog (183816 => 183817)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/WebKit2/ChangeLog 2015-05-05 16:59:31 UTC (rev 183816)
+++ trunk/Source/WebKit2/ChangeLog 2015-05-05 17:12:36 UTC (rev 183817)
</span><span class="lines">@@ -1,3 +1,19 @@
</span><ins>+2015-05-04 Alex Christensen <achristensen@webkit.org>
+
+ [Content Extensions] Use less memory when writing byte code to file
+ https://bugs.webkit.org/show_bug.cgi?id=144602
+
+ Reviewed by Darin Adler.
+
+ * UIProcess/API/APIUserContentExtensionStore.cpp:
+ (API::decodeContentExtensionMetaData):
+ (API::writeDataToFile):
+ (API::compiledToFile):
+ (API::createExtension):
+ Compile and write each DFA to file, then come back and write the header when finalizing.
+ Also don't copy the DFA bytecode. This way, we don't need to keep more than one DFA's
+ bytecode in memory at a time.
+
</ins><span class="cx"> 2015-05-05 Sungmann Cho <sungmann.cho@navercorp.com>
</span><span class="cx">
</span><span class="cx"> Make all FrameLoadState data members private.
</span></span></pre></div>
<a id="trunkSourceWebKit2UIProcessAPIAPIUserContentExtensionStorecpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/WebKit2/UIProcess/API/APIUserContentExtensionStore.cpp (183816 => 183817)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/WebKit2/UIProcess/API/APIUserContentExtensionStore.cpp 2015-05-05 16:59:31 UTC (rev 183816)
+++ trunk/Source/WebKit2/UIProcess/API/APIUserContentExtensionStore.cpp 2015-05-05 17:12:36 UTC (rev 183817)
</span><span class="lines">@@ -74,12 +74,11 @@
</span><span class="cx"> return WebCore::pathByAppendingComponent(base, "ContentExtension-" + WebCore::encodeForFileName(identifier));
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+const size_t ContentExtensionFileHeaderSize = sizeof(uint32_t) + 2 * sizeof(uint64_t);
</ins><span class="cx"> struct ContentExtensionMetaData {
</span><del>- uint32_t version;
- uint64_t actionsOffset;
- uint64_t actionsSize;
- uint64_t bytecodeOffset;
- uint64_t bytecodeSize;
</del><ins>+ uint32_t version { 1 };
+ uint64_t actionsSize { 0 };
+ uint64_t bytecodeSize { 0 };
</ins><span class="cx"> };
</span><span class="cx">
</span><span class="cx"> static Data encodeContentExtensionMetaData(const ContentExtensionMetaData& metaData)
</span><span class="lines">@@ -90,6 +89,7 @@
</span><span class="cx"> encoder << metaData.actionsSize;
</span><span class="cx"> encoder << metaData.bytecodeSize;
</span><span class="cx">
</span><ins>+ ASSERT(encoder.bufferSize() == ContentExtensionFileHeaderSize);
</ins><span class="cx"> return Data(encoder.buffer(), encoder.bufferSize());
</span><span class="cx"> }
</span><span class="cx">
</span><span class="lines">@@ -109,8 +109,6 @@
</span><span class="cx"> return false;
</span><span class="cx"> if (!decoder.decode(metaData.bytecodeSize))
</span><span class="cx"> return false;
</span><del>- metaData.actionsOffset = decoder.currentOffset();
- metaData.bytecodeOffset = metaData.actionsOffset + metaData.actionsSize;
</del><span class="cx"> success = true;
</span><span class="cx"> return false;
</span><span class="cx"> });
</span><span class="lines">@@ -141,7 +139,7 @@
</span><span class="cx"> return true;
</span><span class="cx"> }
</span><span class="cx">
</span><del>-static bool writeDataToFile(Data& fileData, WebCore::PlatformFileHandle fd)
</del><ins>+static bool writeDataToFile(const Data& fileData, WebCore::PlatformFileHandle fd)
</ins><span class="cx"> {
</span><span class="cx"> bool success = true;
</span><span class="cx"> fileData.apply([fd, &success](const uint8_t* data, size_t size) {
</span><span class="lines">@@ -161,65 +159,76 @@
</span><span class="cx">
</span><span class="cx"> class CompilationClient final : public ContentExtensionCompilationClient {
</span><span class="cx"> public:
</span><del>- CompilationClient(Data& bytecodeData, Data& actionsData)
- : m_bytecodeData(bytecodeData)
- , m_actionsData(actionsData)
</del><ins>+ CompilationClient(WebCore::PlatformFileHandle fileHandle, ContentExtensionMetaData& metaData)
+ : m_fileHandle(fileHandle)
+ , m_metaData(metaData)
</ins><span class="cx"> {
</span><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> virtual void writeBytecode(Vector<DFABytecode>&& bytecode) override
</span><span class="cx"> {
</span><del>- m_bytecodeData = Data(bytecode.data(), bytecode.size());
</del><ins>+ m_bytecodeWritten += bytecode.size();
+ write(Data(bytecode.data(), bytecode.size()));
</ins><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> virtual void writeActions(Vector<SerializedActionByte>&& actions) override
</span><span class="cx"> {
</span><del>- m_actionsData = Data(actions.data(), actions.size());
</del><ins>+ ASSERT(!m_bytecodeWritten);
+ ASSERT(!m_actionsWritten);
+ m_actionsWritten += actions.size();
+ write(Data(actions.data(), actions.size()));
</ins><span class="cx"> }
</span><ins>+
+ virtual void finalize() override
+ {
+ m_metaData.actionsSize = m_actionsWritten;
+ m_metaData.bytecodeSize = m_bytecodeWritten;
+
+ Data header = encodeContentExtensionMetaData(m_metaData);
+ if (!m_fileError && WebCore::seekFile(m_fileHandle, 0ll, WebCore::FileSeekOrigin::SeekFromBeginning) == -1) {
+ WebCore::closeFile(m_fileHandle);
+ m_fileError = true;
+ }
+ write(header);
+ }
+
+ bool hadErrorWhileWritingToFile() { return m_fileError; }
</ins><span class="cx">
</span><span class="cx"> private:
</span><del>- Data& m_bytecodeData;
- Data& m_actionsData;
</del><ins>+ void write(const Data& data)
+ {
+ if (!m_fileError && !writeDataToFile(data, m_fileHandle)) {
+ WebCore::closeFile(m_fileHandle);
+ m_fileError = true;
+ }
+ }
+
+ WebCore::PlatformFileHandle m_fileHandle;
+ ContentExtensionMetaData& m_metaData;
+ size_t m_bytecodeWritten { 0 };
+ size_t m_actionsWritten { 0 };
+ bool m_fileError { false };
</ins><span class="cx"> };
</span><span class="cx">
</span><del>- Data bytecode;
- Data actions;
- CompilationClient compilationClient(bytecode, actions);
-
- // FIXME: This copies the data. Instead, we should be passing an interface
- // to the compiler that can write directly to a file.
-
- auto compilerError = compileRuleList(compilationClient, json);
- if (compilerError)
- return compilerError;
-
- auto actionsAndBytecode = concatenate(actions, bytecode);
-
- metaData.version = 1;
- metaData.actionsSize = actions.size();
- metaData.bytecodeSize = bytecode.size();
-
- auto encodedMetaData = encodeContentExtensionMetaData(metaData);
-
- metaData.actionsOffset = encodedMetaData.size();
- metaData.bytecodeOffset = encodedMetaData.size() + metaData.actionsSize;
-
- auto data = concatenate(encodedMetaData, actionsAndBytecode);
- auto dataSize = data.size();
-
- ASSERT(metaData.actionsOffset + metaData.actionsSize + metaData.bytecodeSize == dataSize);
-
</del><span class="cx"> auto temporaryFileHandle = WebCore::invalidPlatformFileHandle;
</span><span class="cx"> String temporaryFilePath = WebCore::openTemporaryFile("ContentExtension", temporaryFileHandle);
</span><span class="cx"> if (temporaryFileHandle == WebCore::invalidPlatformFileHandle)
</span><span class="cx"> return UserContentExtensionStore::Error::CompileFailed;
</span><ins>+
+ char invalidHeader[ContentExtensionFileHeaderSize];
+ memset(invalidHeader, 0xFF, sizeof(invalidHeader));
+ // This header will be rewritten in CompilationClient::finalize.
+ if (WebCore::writeToFile(temporaryFileHandle, invalidHeader, sizeof(invalidHeader)) == -1)
+ return UserContentExtensionStore::Error::CompileFailed;
</ins><span class="cx">
</span><del>- if (!writeDataToFile(data, temporaryFileHandle)) {
- WebCore::closeFile(temporaryFileHandle);
</del><ins>+ CompilationClient compilationClient(temporaryFileHandle, metaData);
+
+ if (auto compilerError = compileRuleList(compilationClient, json))
+ return compilerError;
+ if (compilationClient.hadErrorWhileWritingToFile())
</ins><span class="cx"> return UserContentExtensionStore::Error::CompileFailed;
</span><del>- }
-
- mappedData = mapFile(temporaryFileHandle, 0, dataSize);
</del><ins>+
+ mappedData = mapFile(temporaryFileHandle, 0, ContentExtensionFileHeaderSize + metaData.actionsSize + metaData.bytecodeSize);
</ins><span class="cx"> WebCore::closeFile(temporaryFileHandle);
</span><span class="cx">
</span><span class="cx"> if (mappedData.isNull())
</span><span class="lines">@@ -237,9 +246,9 @@
</span><span class="cx"> auto compiledContentExtensionData = WebKit::WebCompiledContentExtensionData(
</span><span class="cx"> WTF::move(sharedMemory),
</span><span class="cx"> fileData,
</span><del>- metaData.actionsOffset,
</del><ins>+ ContentExtensionFileHeaderSize,
</ins><span class="cx"> metaData.actionsSize,
</span><del>- metaData.bytecodeOffset,
</del><ins>+ ContentExtensionFileHeaderSize + metaData.actionsSize,
</ins><span class="cx"> metaData.bytecodeSize
</span><span class="cx"> );
</span><span class="cx"> auto compiledContentExtension = WebKit::WebCompiledContentExtension::create(WTF::move(compiledContentExtensionData));
</span></span></pre></div>
<a id="trunkToolsChangeLog"></a>
<div class="modfile"><h4>Modified: trunk/Tools/ChangeLog (183816 => 183817)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Tools/ChangeLog 2015-05-05 16:59:31 UTC (rev 183816)
+++ trunk/Tools/ChangeLog 2015-05-05 17:12:36 UTC (rev 183817)
</span><span class="lines">@@ -1,3 +1,12 @@
</span><ins>+2015-05-04 Alex Christensen <achristensen@webkit.org>
+
+ [Content Extensions] Use less memory when writing byte code to file
+ https://bugs.webkit.org/show_bug.cgi?id=144602
+
+ Reviewed by Darin Adler.
+
+ * TestWebKitAPI/Tests/WebCore/ContentExtensions.cpp:
+
</ins><span class="cx"> 2015-05-05 Carlos Garcia Campos <cgarcia@igalia.com>
</span><span class="cx">
</span><span class="cx"> [GTK] Async operations running in the WorkQueue thread should schedule their sources to the WorkQueue main lopp
</span></span></pre></div>
<a id="trunkToolsTestWebKitAPITestsWebCoreContentExtensionscpp"></a>
<div class="modfile"><h4>Modified: trunk/Tools/TestWebKitAPI/Tests/WebCore/ContentExtensions.cpp (183816 => 183817)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Tools/TestWebKitAPI/Tests/WebCore/ContentExtensions.cpp 2015-05-05 16:59:31 UTC (rev 183816)
+++ trunk/Tools/TestWebKitAPI/Tests/WebCore/ContentExtensions.cpp 2015-05-05 17:12:36 UTC (rev 183817)
</span><span class="lines">@@ -81,20 +81,30 @@
</span><span class="cx"> InMemoryContentExtensionCompilationClient(WebCore::ContentExtensions::CompiledContentExtensionData& data)
</span><span class="cx"> : m_data(data)
</span><span class="cx"> {
</span><ins>+ EXPECT_EQ(data.bytecode.size(), 0ull);
+ EXPECT_EQ(data.actions.size(), 0ull);
</ins><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> virtual void writeBytecode(Vector<WebCore::ContentExtensions::DFABytecode>&& bytecode) override
</span><span class="cx"> {
</span><del>- m_data.bytecode = WTF::move(bytecode);
</del><ins>+ EXPECT_FALSE(finalized);
+ m_data.bytecode.appendVector(bytecode);
</ins><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> virtual void writeActions(Vector<WebCore::ContentExtensions::SerializedActionByte>&& actions) override
</span><span class="cx"> {
</span><del>- m_data.actions = WTF::move(actions);
</del><ins>+ EXPECT_FALSE(finalized);
+ m_data.actions.appendVector(actions);
</ins><span class="cx"> }
</span><ins>+
+ virtual void finalize() override
+ {
+ finalized = true;
+ }
</ins><span class="cx">
</span><span class="cx"> private:
</span><span class="cx"> WebCore::ContentExtensions::CompiledContentExtensionData& m_data;
</span><ins>+ bool finalized { false };
</ins><span class="cx"> };
</span><span class="cx">
</span><span class="cx"> class InMemoryCompiledContentExtension : public ContentExtensions::CompiledContentExtension {
</span></span></pre>
</div>
</div>
</body>
</html>