<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
"http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head><meta http-equiv="content-type" content="text/html; charset=utf-8" />
<title>[283194] trunk</title>
</head>
<body>

<style type="text/css"><!--
#msg dl.meta { border: 1px #006 solid; background: #369; padding: 6px; color: #fff; }
#msg dl.meta dt { float: left; width: 6em; font-weight: bold; }
#msg dt:after { content:':';}
#msg dl, #msg dt, #msg ul, #msg li, #header, #footer, #logmsg { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt;  }
#msg dl a { font-weight: bold}
#msg dl a:link    { color:#fc3; }
#msg dl a:active  { color:#ff0; }
#msg dl a:visited { color:#cc6; }
h3 { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt; font-weight: bold; }
#msg pre { overflow: auto; background: #ffc; border: 1px #fa0 solid; padding: 6px; }
#logmsg { background: #ffc; border: 1px #fa0 solid; padding: 1em 1em 0 1em; }
#logmsg p, #logmsg pre, #logmsg blockquote { margin: 0 0 1em 0; }
#logmsg p, #logmsg li, #logmsg dt, #logmsg dd { line-height: 14pt; }
#logmsg h1, #logmsg h2, #logmsg h3, #logmsg h4, #logmsg h5, #logmsg h6 { margin: .5em 0; }
#logmsg h1:first-child, #logmsg h2:first-child, #logmsg h3:first-child, #logmsg h4:first-child, #logmsg h5:first-child, #logmsg h6:first-child { margin-top: 0; }
#logmsg ul, #logmsg ol { padding: 0; list-style-position: inside; margin: 0 0 0 1em; }
#logmsg ul { text-indent: -1em; padding-left: 1em; }#logmsg ol { text-indent: -1.5em; padding-left: 1.5em; }
#logmsg > ul, #logmsg > ol { margin: 0 0 1em 0; }
#logmsg pre { background: #eee; padding: 1em; }
#logmsg blockquote { border: 1px solid #fa0; border-left-width: 10px; padding: 1em 1em 0 1em; background: white;}
#logmsg dl { margin: 0; }
#logmsg dt { font-weight: bold; }
#logmsg dd { margin: 0; padding: 0 0 0.5em 0; }
#logmsg dd:before { content:'\00bb';}
#logmsg table { border-spacing: 0px; border-collapse: collapse; border-top: 4px solid #fa0; border-bottom: 1px solid #fa0; background: #fff; }
#logmsg table th { text-align: left; font-weight: normal; padding: 0.2em 0.5em; border-top: 1px dotted #fa0; }
#logmsg table td { text-align: right; border-top: 1px dotted #fa0; padding: 0.2em 0.5em; }
#logmsg table thead th { text-align: center; border-bottom: 1px solid #fa0; }
#logmsg table th.Corner { text-align: left; }
#logmsg hr { border: none 0; border-top: 2px dashed #fa0; height: 1px; }
#header, #footer { color: #fff; background: #636; border: 1px #300 solid; padding: 6px; }
#patch { width: 100%; }
#patch h4 {font-family: verdana,arial,helvetica,sans-serif;font-size:10pt;padding:8px;background:#369;color:#fff;margin:0;}
#patch .propset h4, #patch .binary h4 {margin:0;}
#patch pre {padding:0;line-height:1.2em;margin:0;}
#patch .diff {width:100%;background:#eee;padding: 0 0 10px 0;overflow:auto;}
#patch .propset .diff, #patch .binary .diff  {padding:10px 0;}
#patch span {display:block;padding:0 10px;}
#patch .modfile, #patch .addfile, #patch .delfile, #patch .propset, #patch .binary, #patch .copfile {border:1px solid #ccc;margin:10px 0;}
#patch ins {background:#dfd;text-decoration:none;display:block;padding:0 10px;}
#patch del {background:#fdd;text-decoration:none;display:block;padding:0 10px;}
#patch .lines, .info {color:#888;background:#fff;}
--></style>
<div id="msg">
<dl class="meta">
<dt>Revision</dt> <dd><a href="http://trac.webkit.org/projects/webkit/changeset/283194">283194</a></dd>
<dt>Author</dt> <dd>katherine_cheney@apple.com</dd>
<dt>Date</dt> <dd>2021-09-28 14:13:01 -0700 (Tue, 28 Sep 2021)</dd>
</dl>

<h3>Log Message</h3>
<pre>PCM: different bundleID entries will override each other
https://bugs.webkit.org/show_bug.cgi?id=230839

Reviewed by Alex Christensen.

Source/WebKit:

We recently added a bundleID column to PCM tables. We want to make
sure entries with different bundleIDs do not override each other,
so we should make it a part of the unique constraint on both PCM
tables that contain it. This requires creating new tables and
migrating existing data to them. Luckily this code already exists
in the ITP database, and we can just move it to the shared
DatabaseUtilities class.

* NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.cpp:
(WebKit::ResourceLoadStatisticsDatabaseStore::expectedTableAndIndexQueries):
(WebKit::stripIndexQueryToMatchStoredValue): Deleted.
(WebKit::expectedTableAndIndexQueries): Deleted.
(WebKit::ResourceLoadStatisticsDatabaseStore::currentTableAndIndexQueries): Deleted.
(WebKit::insertDistinctValuesInTableStatement): Deleted.
(WebKit::ResourceLoadStatisticsDatabaseStore::migrateDataToNewTablesIfNecessary): Deleted.
* NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.h:
* NetworkProcess/DatabaseUtilities.cpp:
(WebKit::DatabaseUtilities::stripIndexQueryToMatchStoredValue):
(WebKit::DatabaseUtilities::currentTableAndIndexQueries):
(WebKit::insertDistinctValuesInTableStatement):
(WebKit::DatabaseUtilities::migrateDataToNewTablesIfNecessary):
* NetworkProcess/DatabaseUtilities.h:
* NetworkProcess/PrivateClickMeasurement/PrivateClickMeasurementDatabase.cpp:
(WebKit::PCM::Database::Database):
(WebKit::PCM::Database::expectedTableAndIndexQueries):
(WebKit::PCM::Database::createUniqueIndices):
(WebKit::PCM::Database::needsUpdatedSchema):
* NetworkProcess/PrivateClickMeasurement/PrivateClickMeasurementDatabase.h:

Tools:

API test coverage for the case of existing PCM data with a bundleID
column but an expired unique index.

* TestWebKitAPI/Tests/WebKitCocoa/PrivateClickMeasurement.mm:
(addUnattributedPCMv4):
(addAttributedPCMv4):
(dumpedPCM):
(pollUntilPCMIsMigrated):
(emptyPcmDBPath):
(createAndPopulatePCMObservedDomainTable):
(setUpFromResourceLoadStatisticsDatabase):
(setUpFromPCMDatabase):
(TEST):
(setUp): Deleted.</pre>

<h3>Modified Paths</h3>
<ul>
<li><a href="#trunkSourceWebKitChangeLog">trunk/Source/WebKit/ChangeLog</a></li>
<li><a href="#trunkSourceWebKitNetworkProcessClassifierResourceLoadStatisticsDatabaseStorecpp">trunk/Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.cpp</a></li>
<li><a href="#trunkSourceWebKitNetworkProcessClassifierResourceLoadStatisticsDatabaseStoreh">trunk/Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.h</a></li>
<li><a href="#trunkSourceWebKitNetworkProcessDatabaseUtilitiescpp">trunk/Source/WebKit/NetworkProcess/DatabaseUtilities.cpp</a></li>
<li><a href="#trunkSourceWebKitNetworkProcessDatabaseUtilitiesh">trunk/Source/WebKit/NetworkProcess/DatabaseUtilities.h</a></li>
<li><a href="#trunkSourceWebKitNetworkProcessPrivateClickMeasurementPrivateClickMeasurementDatabasecpp">trunk/Source/WebKit/NetworkProcess/PrivateClickMeasurement/PrivateClickMeasurementDatabase.cpp</a></li>
<li><a href="#trunkSourceWebKitNetworkProcessPrivateClickMeasurementPrivateClickMeasurementDatabaseh">trunk/Source/WebKit/NetworkProcess/PrivateClickMeasurement/PrivateClickMeasurementDatabase.h</a></li>
<li><a href="#trunkToolsChangeLog">trunk/Tools/ChangeLog</a></li>
<li><a href="#trunkToolsTestWebKitAPITestsWebKitCocoaPrivateClickMeasurementmm">trunk/Tools/TestWebKitAPI/Tests/WebKitCocoa/PrivateClickMeasurement.mm</a></li>
</ul>

</div>
<div id="patch">
<h3>Diff</h3>
<a id="trunkSourceWebKitChangeLog"></a>
<div class="modfile"><h4>Modified: trunk/Source/WebKit/ChangeLog (283193 => 283194)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/WebKit/ChangeLog    2021-09-28 21:10:21 UTC (rev 283193)
+++ trunk/Source/WebKit/ChangeLog       2021-09-28 21:13:01 UTC (rev 283194)
</span><span class="lines">@@ -1,3 +1,39 @@
</span><ins>+2021-09-28  Kate Cheney  <katherine_cheney@apple.com>
+
+        PCM: different bundleID entries will override each other
+        https://bugs.webkit.org/show_bug.cgi?id=230839
+
+        Reviewed by Alex Christensen.
+
+        We recently added a bundleID column to PCM tables. We want to make
+        sure entries with different bundleIDs do not override each other,
+        so we should make it a part of the unique constraint on both PCM
+        tables that contain it. This requires creating new tables and
+        migrating existing data to them. Luckily this code already exists
+        in the ITP database, and we can just move it to the shared
+        DatabaseUtilities class.
+
+        * NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.cpp:
+        (WebKit::ResourceLoadStatisticsDatabaseStore::expectedTableAndIndexQueries):
+        (WebKit::stripIndexQueryToMatchStoredValue): Deleted.
+        (WebKit::expectedTableAndIndexQueries): Deleted.
+        (WebKit::ResourceLoadStatisticsDatabaseStore::currentTableAndIndexQueries): Deleted.
+        (WebKit::insertDistinctValuesInTableStatement): Deleted.
+        (WebKit::ResourceLoadStatisticsDatabaseStore::migrateDataToNewTablesIfNecessary): Deleted.
+        * NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.h:
+        * NetworkProcess/DatabaseUtilities.cpp:
+        (WebKit::DatabaseUtilities::stripIndexQueryToMatchStoredValue):
+        (WebKit::DatabaseUtilities::currentTableAndIndexQueries):
+        (WebKit::insertDistinctValuesInTableStatement):
+        (WebKit::DatabaseUtilities::migrateDataToNewTablesIfNecessary):
+        * NetworkProcess/DatabaseUtilities.h:
+        * NetworkProcess/PrivateClickMeasurement/PrivateClickMeasurementDatabase.cpp:
+        (WebKit::PCM::Database::Database):
+        (WebKit::PCM::Database::expectedTableAndIndexQueries):
+        (WebKit::PCM::Database::createUniqueIndices):
+        (WebKit::PCM::Database::needsUpdatedSchema):
+        * NetworkProcess/PrivateClickMeasurement/PrivateClickMeasurementDatabase.h:
+
</ins><span class="cx"> 2021-09-28  Brent Fulgham  <bfulgham@apple.com>
</span><span class="cx"> 
</span><span class="cx">         Explicitly deny 'system-privilege' in the sandbox profile as a hardening measure 
</span></span></pre></div>
<a id="trunkSourceWebKitNetworkProcessClassifierResourceLoadStatisticsDatabaseStorecpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.cpp (283193 => 283194)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.cpp    2021-09-28 21:10:21 UTC (rev 283193)
+++ trunk/Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.cpp       2021-09-28 21:13:01 UTC (rev 283194)
</span><span class="lines">@@ -50,7 +50,6 @@
</span><span class="cx"> #include <wtf/CrossThreadCopier.h>
</span><span class="cx"> #include <wtf/DateMath.h>
</span><span class="cx"> #include <wtf/MathExtras.h>
</span><del>-#include <wtf/RobinHoodHashMap.h>
</del><span class="cx"> #include <wtf/Scope.h>
</span><span class="cx"> #include <wtf/StdSet.h>
</span><span class="cx"> #include <wtf/SuspendableWorkQueue.h>
</span><span class="lines">@@ -253,13 +252,8 @@
</span><span class="cx">     return schema.contains("REFERENCES TopLevelDomains");
</span><span class="cx"> }
</span><span class="cx"> 
</span><del>-static String stripIndexQueryToMatchStoredValue(const char* originalQuery)
</del><ins>+const MemoryCompactLookupOnlyRobinHoodHashMap<String, TableAndIndexPair>& ResourceLoadStatisticsDatabaseStore::expectedTableAndIndexQueries()
</ins><span class="cx"> {
</span><del>-    return String(originalQuery).replace("CREATE UNIQUE INDEX IF NOT EXISTS", "CREATE UNIQUE INDEX");
-}
-
-static const MemoryCompactLookupOnlyRobinHoodHashMap<String, TableAndIndexPair>& expectedTableAndIndexQueries()
-{
</del><span class="cx">     static auto expectedTableAndIndexQueries = makeNeverDestroyed(MemoryCompactLookupOnlyRobinHoodHashMap<String, TableAndIndexPair> {
</span><span class="cx">         { "ObservedDomains"_s, std::make_pair<String, std::optional<String>>(createObservedDomain, std::nullopt) },
</span><span class="cx">         { "TopLevelDomains"_s, std::make_pair<String, std::optional<String>>(createTopLevelDomains, std::nullopt) },
</span><span class="lines">@@ -279,6 +273,27 @@
</span><span class="cx">     return expectedTableAndIndexQueries;
</span><span class="cx"> }
</span><span class="cx"> 
</span><ins>+const Vector<String>& ResourceLoadStatisticsDatabaseStore::sortedTables()
+{
+    static auto sortedTables = makeNeverDestroyed(Vector<String> {
+        "ObservedDomains"_s,
+        "TopLevelDomains"_s,
+        "StorageAccessUnderTopFrameDomains"_s,
+        "TopFrameUniqueRedirectsTo"_s,
+        "TopFrameUniqueRedirectsToSinceSameSiteStrictEnforcement"_s,
+        "TopFrameUniqueRedirectsFrom"_s,
+        "TopFrameLinkDecorationsFrom"_s,
+        "TopFrameLoadedThirdPartyScripts"_s,
+        "SubframeUnderTopFrameDomains"_s,
+        "SubresourceUnderTopFrameDomains"_s,
+        "SubresourceUniqueRedirectsTo"_s,
+        "SubresourceUniqueRedirectsFrom"_s,
+        "OperatingDates"_s
+    });
+
+    return sortedTables;
+}
+
</ins><span class="cx"> template <typename ContainerType>
</span><span class="cx"> static String buildList(const ContainerType& values)
</span><span class="cx"> {
</span><span class="lines">@@ -367,52 +382,6 @@
</span><span class="cx">     }
</span><span class="cx"> }
</span><span class="cx"> 
</span><del>-TableAndIndexPair ResourceLoadStatisticsDatabaseStore::currentTableAndIndexQueries(const String& tableName)
-{
-    auto getTableStatement = m_database.prepareStatement("SELECT sql FROM sqlite_master WHERE tbl_name=? AND type = 'table'"_s);
-    if (!getTableStatement) {
-        RELEASE_LOG_ERROR(Network, "%p - ResourceLoadStatisticsDatabaseStore::currentTableAndIndexQueries Unable to prepare statement to fetch schema for the table, error message: %" PRIVATE_LOG_STRING, this, m_database.lastErrorMsg());
-        ASSERT_NOT_REACHED();
-        return { };
-    }
-
-    if (getTableStatement->bindText(1, tableName) != SQLITE_OK) {
-        RELEASE_LOG_ERROR(Network, "%p - ResourceLoadStatisticsDatabaseStore::currentTableAndIndexQueries Unable to bind statement to fetch schema for the table, error message: %" PRIVATE_LOG_STRING, this, m_database.lastErrorMsg());
-        ASSERT_NOT_REACHED();
-        return { };
-    }
-
-    if (getTableStatement->step() != SQLITE_ROW) {
-        RELEASE_LOG_ERROR(Network, "%p - ResourceLoadStatisticsDatabaseStore::currentTableAndIndexQueries error executing statement to fetch table schema, error message: %" PRIVATE_LOG_STRING, this, m_database.lastErrorMsg());
-        ASSERT_NOT_REACHED();
-        return { };
-    }
-
-    String createTableQuery = getTableStatement->columnText(0);
-
-    auto getIndexStatement = m_database.prepareStatement("SELECT sql FROM sqlite_master WHERE tbl_name=? AND type = 'index'"_s);
-    if (!getIndexStatement) {
-        RELEASE_LOG_ERROR(Network, "%p - ResourceLoadStatisticsDatabaseStore::currentTableAndIndexQueries Unable to prepare statement to fetch index for the table, error message: %" PRIVATE_LOG_STRING, this, m_database.lastErrorMsg());
-        ASSERT_NOT_REACHED();
-        return { };
-    }
-
-    if (getIndexStatement->bindText(1, tableName) != SQLITE_OK) {
-        RELEASE_LOG_ERROR(Network, "%p - ResourceLoadStatisticsDatabaseStore::currentTableAndIndexQueries Unable to bind statement to fetch index for the table, error message: %" PRIVATE_LOG_STRING, this, m_database.lastErrorMsg());
-        ASSERT_NOT_REACHED();
-        return { };
-    }
-
-    std::optional<String> index;
-    if (getIndexStatement->step() == SQLITE_ROW) {
-        auto rawIndex = String(getIndexStatement->columnText(0));
-        if (!rawIndex.isEmpty())
-            index = rawIndex;
-    }
-
-    return std::make_pair<String, std::optional<String>>(WTFMove(createTableQuery), WTFMove(index));
-}
-
</del><span class="cx"> bool ResourceLoadStatisticsDatabaseStore::missingUniqueIndices()
</span><span class="cx"> {
</span><span class="cx">     auto statement = m_database.prepareStatement("SELECT COUNT(*) FROM sqlite_master WHERE type = 'index'"_s);
</span><span class="lines">@@ -456,68 +425,6 @@
</span><span class="cx">     return false;
</span><span class="cx"> }
</span><span class="cx"> 
</span><del>-static Expected<SQLiteStatement, int> insertDistinctValuesInTableStatement(SQLiteDatabase& database, const String& table)
-{
-    if (table == "SubframeUnderTopFrameDomains")
-        return database.prepareStatement("INSERT INTO SubframeUnderTopFrameDomains SELECT subFrameDomainID, MAX(lastUpdated), topFrameDomainID FROM _SubframeUnderTopFrameDomains GROUP BY subFrameDomainID, topFrameDomainID"_s);
-
-    if (table == "SubresourceUnderTopFrameDomains")
-        return database.prepareStatement("INSERT INTO SubresourceUnderTopFrameDomains SELECT subresourceDomainID, MAX(lastUpdated), topFrameDomainID FROM _SubresourceUnderTopFrameDomains GROUP BY subresourceDomainID, topFrameDomainID"_s);
-
-    if (table == "SubresourceUniqueRedirectsTo")
-        return database.prepareStatement("INSERT INTO SubresourceUniqueRedirectsTo SELECT subresourceDomainID, MAX(lastUpdated), toDomainID FROM _SubresourceUniqueRedirectsTo GROUP BY subresourceDomainID, toDomainID"_s);
-
-    if (table == "TopFrameLinkDecorationsFrom")
-        return database.prepareStatement("INSERT INTO TopFrameLinkDecorationsFrom SELECT toDomainID, MAX(lastUpdated), fromDomainID FROM _TopFrameLinkDecorationsFrom GROUP BY toDomainID, fromDomainID"_s);
-
-    return database.prepareStatementSlow(makeString("INSERT INTO ", table, " SELECT DISTINCT * FROM _", table));
-}
-
-void ResourceLoadStatisticsDatabaseStore::migrateDataToNewTablesIfNecessary()
-{
-    if (!needsUpdatedSchema())
-        return;
-
-    auto transactionScope = beginTransactionIfNecessary();
-
-    for (auto& table : expectedTableAndIndexQueries().keys()) {
-        auto alterTable = m_database.prepareStatementSlow(makeString("ALTER TABLE ", table, " RENAME TO _", table));
-        if (!alterTable || alterTable->step() != SQLITE_DONE) {
-            RELEASE_LOG_ERROR(Network, "%p - ResourceLoadStatisticsDatabaseStore::migrateDataToNewTablesIfNecessary failed to rename table, error message: %s", this, m_database.lastErrorMsg());
-            ASSERT_NOT_REACHED();
-            return;
-        }
-    }
-
-    if (!createSchema()) {
-        RELEASE_LOG_ERROR(Network, "%p - ResourceLoadStatisticsDatabaseStore::migrateDataToNewTablesIfNecessary failed to create schema, error message: %s", this, m_database.lastErrorMsg());
-        ASSERT_NOT_REACHED();
-        return;
-    }
-
-    for (auto& table : expectedTableAndIndexQueries().keys()) {
-        auto migrateTableData = insertDistinctValuesInTableStatement(m_database, table);
-        if (!migrateTableData || migrateTableData->step() != SQLITE_DONE) {
-            RELEASE_LOG_ERROR(Network, "%p - ResourceLoadStatisticsDatabaseStore::migrateDataToNewTablesIfNecessary failed to migrate schema, error message: %s", this, m_database.lastErrorMsg());
-            ASSERT_NOT_REACHED();
-            return;
-        }
-
-        auto dropTableQuery = m_database.prepareStatementSlow(makeString("DROP TABLE _", table));
-        if (!dropTableQuery || dropTableQuery->step() != SQLITE_DONE) {
-            RELEASE_LOG_ERROR(Network, "%p - ResourceLoadStatisticsDatabaseStore::migrateDataToNewTablesIfNecessary failed to drop temporary tables, error message: %s", this, m_database.lastErrorMsg());
-            ASSERT_NOT_REACHED();
-            return;
-        }
-    }
-
-    if (!createUniqueIndices()) {
-        RELEASE_LOG_ERROR(Network, "%p - ResourceLoadStatisticsDatabaseStore::migrateDataToNewTablesIfNecessary failed to create unique indices, error message: %s", this, m_database.lastErrorMsg());
-        ASSERT_NOT_REACHED();
-        return;
-    }
-}
-
</del><span class="cx"> void ResourceLoadStatisticsDatabaseStore::migrateDataToPCMDatabaseIfNecessary()
</span><span class="cx"> {
</span><span class="cx">     if (!tableExists("UnattributedPrivateClickMeasurement")
</span></span></pre></div>
<a id="trunkSourceWebKitNetworkProcessClassifierResourceLoadStatisticsDatabaseStoreh"></a>
<div class="modfile"><h4>Modified: trunk/Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.h (283193 => 283194)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.h      2021-09-28 21:10:21 UTC (rev 283193)
+++ trunk/Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.h 2021-09-28 21:13:01 UTC (rev 283194)
</span><span class="lines">@@ -48,8 +48,6 @@
</span><span class="cx"> static constexpr size_t numberOfStatistics = 7;
</span><span class="cx"> static constexpr std::array<unsigned, numberOfBucketsPerStatistic> bucketSizes {{ 1, 3, 10, 50, 100 }};
</span><span class="cx"> 
</span><del>-typedef std::pair<String, std::optional<String>> TableAndIndexPair;
-
</del><span class="cx"> // This is always constructed / used / destroyed on the WebResourceLoadStatisticsStore's statistics queue.
</span><span class="cx"> class ResourceLoadStatisticsDatabaseStore final : public ResourceLoadStatisticsStore, public DatabaseUtilities {
</span><span class="cx"> public:
</span><span class="lines">@@ -118,6 +116,8 @@
</span><span class="cx">     static void interruptAllDatabases();
</span><span class="cx"> 
</span><span class="cx"> private:
</span><ins>+    const MemoryCompactLookupOnlyRobinHoodHashMap<String, TableAndIndexPair>& expectedTableAndIndexQueries() final;
+    const Vector<String>& sortedTables() final;
</ins><span class="cx">     void includeTodayAsOperatingDateIfNecessary() override;
</span><span class="cx">     void clearOperatingDates() override { }
</span><span class="cx">     bool hasStatisticsExpired(WallTime mostRecentUserInteractionTime, OperatingDatesWindow) const override;
</span><span class="lines">@@ -126,10 +126,8 @@
</span><span class="cx">     void openITPDatabase();
</span><span class="cx">     void addMissingTablesIfNecessary();
</span><span class="cx">     bool missingUniqueIndices();
</span><del>-    bool needsUpdatedSchema();
-    TableAndIndexPair currentTableAndIndexQueries(const String&);
</del><ins>+    bool needsUpdatedSchema() final;
</ins><span class="cx">     bool missingReferenceToObservedDomains();
</span><del>-    void migrateDataToNewTablesIfNecessary();
</del><span class="cx">     void migrateDataToPCMDatabaseIfNecessary();
</span><span class="cx">     bool tableExists(StringView);
</span><span class="cx">     void deleteTable(StringView);
</span><span class="lines">@@ -206,7 +204,7 @@
</span><span class="cx">     RegistrableDomainsToDeleteOrRestrictWebsiteDataFor registrableDomainsToDeleteOrRestrictWebsiteDataFor() override;
</span><span class="cx">     bool isDatabaseStore() const final { return true; }
</span><span class="cx"> 
</span><del>-    bool createUniqueIndices();
</del><ins>+    bool createUniqueIndices() final;
</ins><span class="cx">     bool createSchema() final;
</span><span class="cx">     String ensureAndMakeDomainList(const HashSet<RegistrableDomain>&);
</span><span class="cx">     std::optional<WallTime> mostRecentUserInteractionTime(const DomainData&);
</span></span></pre></div>
<a id="trunkSourceWebKitNetworkProcessDatabaseUtilitiescpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/WebKit/NetworkProcess/DatabaseUtilities.cpp (283193 => 283194)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/WebKit/NetworkProcess/DatabaseUtilities.cpp 2021-09-28 21:10:21 UTC (rev 283193)
+++ trunk/Source/WebKit/NetworkProcess/DatabaseUtilities.cpp    2021-09-28 21:13:01 UTC (rev 283194)
</span><span class="lines">@@ -187,4 +187,119 @@
</span><span class="cx">     return attribution;
</span><span class="cx"> }
</span><span class="cx"> 
</span><ins>+String DatabaseUtilities::stripIndexQueryToMatchStoredValue(const char* originalQuery)
+{
+    return String(originalQuery).replace("CREATE UNIQUE INDEX IF NOT EXISTS", "CREATE UNIQUE INDEX");
+}
+
+TableAndIndexPair DatabaseUtilities::currentTableAndIndexQueries(const String& tableName)
+{
+    auto getTableStatement = m_database.prepareStatement("SELECT sql FROM sqlite_master WHERE tbl_name=? AND type = 'table'"_s);
+    if (!getTableStatement) {
+        RELEASE_LOG_ERROR(PrivateClickMeasurement, "%p - DatabaseUtilities::currentTableAndIndexQueries Unable to prepare statement to fetch schema for the table, error message: %" PRIVATE_LOG_STRING, this, m_database.lastErrorMsg());
+        ASSERT_NOT_REACHED();
+        return { };
+    }
+
+    if (getTableStatement->bindText(1, tableName) != SQLITE_OK) {
+        RELEASE_LOG_ERROR(PrivateClickMeasurement, "%p - DatabaseUtilities::currentTableAndIndexQueries Unable to bind statement to fetch schema for the table, error message: %" PRIVATE_LOG_STRING, this, m_database.lastErrorMsg());
+        ASSERT_NOT_REACHED();
+        return { };
+    }
+
+    if (getTableStatement->step() != SQLITE_ROW) {
+        RELEASE_LOG_ERROR(PrivateClickMeasurement, "%p - DatabaseUtilities::currentTableAndIndexQueries error executing statement to fetch table schema, error message: %" PRIVATE_LOG_STRING, this, m_database.lastErrorMsg());
+        ASSERT_NOT_REACHED();
+        return { };
+    }
+
+    String createTableQuery = getTableStatement->columnText(0);
+
+    auto getIndexStatement = m_database.prepareStatement("SELECT sql FROM sqlite_master WHERE tbl_name=? AND type = 'index'"_s);
+    if (!getIndexStatement) {
+        RELEASE_LOG_ERROR(PrivateClickMeasurement, "%p - DatabaseUtilities::currentTableAndIndexQueries Unable to prepare statement to fetch index for the table, error message: %" PRIVATE_LOG_STRING, this, m_database.lastErrorMsg());
+        ASSERT_NOT_REACHED();
+        return { };
+    }
+
+    if (getIndexStatement->bindText(1, tableName) != SQLITE_OK) {
+        RELEASE_LOG_ERROR(PrivateClickMeasurement, "%p - DatabaseUtilities::currentTableAndIndexQueries Unable to bind statement to fetch index for the table, error message: %" PRIVATE_LOG_STRING, this, m_database.lastErrorMsg());
+        ASSERT_NOT_REACHED();
+        return { };
+    }
+
+    std::optional<String> index;
+    if (getIndexStatement->step() == SQLITE_ROW) {
+        auto rawIndex = String(getIndexStatement->columnText(0));
+        if (!rawIndex.isEmpty())
+            index = rawIndex;
+    }
+
+    return std::make_pair<String, std::optional<String>>(WTFMove(createTableQuery), WTFMove(index));
+}
+
+static Expected<WebCore::SQLiteStatement, int> insertDistinctValuesInTableStatement(WebCore::SQLiteDatabase& database, const String& table)
+{
+    if (table == "SubframeUnderTopFrameDomains")
+        return database.prepareStatement("INSERT INTO SubframeUnderTopFrameDomains SELECT subFrameDomainID, MAX(lastUpdated), topFrameDomainID FROM _SubframeUnderTopFrameDomains GROUP BY subFrameDomainID, topFrameDomainID"_s);
+
+    if (table == "SubresourceUnderTopFrameDomains")
+        return database.prepareStatement("INSERT INTO SubresourceUnderTopFrameDomains SELECT subresourceDomainID, MAX(lastUpdated), topFrameDomainID FROM _SubresourceUnderTopFrameDomains GROUP BY subresourceDomainID, topFrameDomainID"_s);
+
+    if (table == "SubresourceUniqueRedirectsTo")
+        return database.prepareStatement("INSERT INTO SubresourceUniqueRedirectsTo SELECT subresourceDomainID, MAX(lastUpdated), toDomainID FROM _SubresourceUniqueRedirectsTo GROUP BY subresourceDomainID, toDomainID"_s);
+
+    if (table == "TopFrameLinkDecorationsFrom")
+        return database.prepareStatement("INSERT INTO TopFrameLinkDecorationsFrom SELECT toDomainID, MAX(lastUpdated), fromDomainID FROM _TopFrameLinkDecorationsFrom GROUP BY toDomainID, fromDomainID"_s);
+
+    return database.prepareStatementSlow(makeString("INSERT INTO ", table, " SELECT DISTINCT * FROM _", table));
+}
+
+void DatabaseUtilities::migrateDataToNewTablesIfNecessary()
+{
+    if (!needsUpdatedSchema())
+        return;
+
+    auto transactionScope = beginTransactionIfNecessary();
+
+    for (auto& table : expectedTableAndIndexQueries().keys()) {
+        auto alterTable = m_database.prepareStatementSlow(makeString("ALTER TABLE ", table, " RENAME TO _", table));
+        if (!alterTable || alterTable->step() != SQLITE_DONE) {
+            RELEASE_LOG_ERROR(PrivateClickMeasurement, "%p - DatabaseUtilities::migrateDataToNewTablesIfNecessary failed to rename table, error message: %s", this, m_database.lastErrorMsg());
+            ASSERT_NOT_REACHED();
+            return;
+        }
+    }
+
+    if (!createSchema()) {
+        ASSERT_NOT_REACHED();
+        return;
+    }
+
+    // Maintain the order of tables to make sure the ObservedDomains table is created first. Other tables have foreign key constraints referencing it.
+    for (auto& table : sortedTables()) {
+        auto migrateTableData = insertDistinctValuesInTableStatement(m_database, table);
+        if (!migrateTableData || migrateTableData->step() != SQLITE_DONE) {
+            RELEASE_LOG_ERROR(PrivateClickMeasurement, "%p - DatabaseUtilities::migrateDataToNewTablesIfNecessary (table %s) failed to migrate schema, error message: %s", this, table.utf8().data(), m_database.lastErrorMsg());
+            ASSERT_NOT_REACHED();
+            return;
+        }
+    }
+
+    // Drop all tables at the end to avoid trashing data that references data in other tables.
+    for (auto& table : sortedTables()) {
+        auto dropTableQuery = m_database.prepareStatementSlow(makeString("DROP TABLE _", table));
+        if (!dropTableQuery || dropTableQuery->step() != SQLITE_DONE) {
+            RELEASE_LOG_ERROR(PrivateClickMeasurement, "%p - DatabaseUtilities::migrateDataToNewTablesIfNecessary failed to drop temporary tables, error message: %s", this, m_database.lastErrorMsg());
+            ASSERT_NOT_REACHED();
+            return;
+        }
+    }
+
+    if (!createUniqueIndices()) {
+        RELEASE_LOG_ERROR(PrivateClickMeasurement, "%p - DatabaseUtilities::migrateDataToNewTablesIfNecessary failed to create unique indices, error message: %s", this, m_database.lastErrorMsg());
+        ASSERT_NOT_REACHED();
+    }
+}
+
</ins><span class="cx"> } // namespace WebKit
</span></span></pre></div>
<a id="trunkSourceWebKitNetworkProcessDatabaseUtilitiesh"></a>
<div class="modfile"><h4>Modified: trunk/Source/WebKit/NetworkProcess/DatabaseUtilities.h (283193 => 283194)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/WebKit/NetworkProcess/DatabaseUtilities.h   2021-09-28 21:10:21 UTC (rev 283193)
+++ trunk/Source/WebKit/NetworkProcess/DatabaseUtilities.h      2021-09-28 21:13:01 UTC (rev 283194)
</span><span class="lines">@@ -27,6 +27,7 @@
</span><span class="cx"> 
</span><span class="cx"> #include <WebCore/SQLiteDatabase.h>
</span><span class="cx"> #include <WebCore/SQLiteTransaction.h>
</span><ins>+#include <wtf/RobinHoodHashMap.h>
</ins><span class="cx"> #include <wtf/Scope.h>
</span><span class="cx"> 
</span><span class="cx"> namespace WebCore {
</span><span class="lines">@@ -39,6 +40,8 @@
</span><span class="cx"> 
</span><span class="cx"> enum class PrivateClickMeasurementAttributionType : bool;
</span><span class="cx"> 
</span><ins>+using TableAndIndexPair = std::pair<String, std::optional<String>>;
+
</ins><span class="cx"> class DatabaseUtilities {
</span><span class="cx"> protected:
</span><span class="cx">     DatabaseUtilities(String&& storageFilePath);
</span><span class="lines">@@ -52,8 +55,15 @@
</span><span class="cx">     void close();
</span><span class="cx">     void interrupt();
</span><span class="cx">     virtual bool createSchema() = 0;
</span><ins>+    virtual bool createUniqueIndices() = 0;
</ins><span class="cx">     virtual void destroyStatements() = 0;
</span><span class="cx">     virtual String getDomainStringFromDomainID(unsigned) const = 0;
</span><ins>+    virtual bool needsUpdatedSchema() = 0;
+    virtual const MemoryCompactLookupOnlyRobinHoodHashMap<String, TableAndIndexPair>& expectedTableAndIndexQueries() = 0;
+    virtual const Vector<String>& sortedTables() = 0;
+    TableAndIndexPair currentTableAndIndexQueries(const String&);
+    String stripIndexQueryToMatchStoredValue(const char* originalQuery);
+    void migrateDataToNewTablesIfNecessary();
</ins><span class="cx"> 
</span><span class="cx">     WebCore::PrivateClickMeasurement buildPrivateClickMeasurementFromDatabase(WebCore::SQLiteStatement&, PrivateClickMeasurementAttributionType) const;
</span><span class="cx"> 
</span></span></pre></div>
<a id="trunkSourceWebKitNetworkProcessPrivateClickMeasurementPrivateClickMeasurementDatabasecpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/WebKit/NetworkProcess/PrivateClickMeasurement/PrivateClickMeasurementDatabase.cpp (283193 => 283194)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/WebKit/NetworkProcess/PrivateClickMeasurement/PrivateClickMeasurementDatabase.cpp   2021-09-28 21:10:21 UTC (rev 283193)
+++ trunk/Source/WebKit/NetworkProcess/PrivateClickMeasurement/PrivateClickMeasurementDatabase.cpp      2021-09-28 21:13:01 UTC (rev 283194)
</span><span class="lines">@@ -68,8 +68,8 @@
</span><span class="cx">     "earliestTimeToSendToSource REAL, token TEXT, signature TEXT, keyID TEXT, earliestTimeToSendToDestination REAL, sourceApplicationBundleID TEXT, "
</span><span class="cx">     "FOREIGN KEY(sourceSiteDomainID) REFERENCES PCMObservedDomains(domainID) ON DELETE CASCADE, FOREIGN KEY(destinationSiteDomainID) REFERENCES "
</span><span class="cx">     "PCMObservedDomains(domainID) ON DELETE CASCADE)"_s;
</span><del>-constexpr auto createUniqueIndexUnattributedPrivateClickMeasurement = "CREATE UNIQUE INDEX IF NOT EXISTS UnattributedPrivateClickMeasurement_sourceSiteDomainID_destinationSiteDomainID on UnattributedPrivateClickMeasurement ( sourceSiteDomainID, destinationSiteDomainID )"_s;
-constexpr auto createUniqueIndexAttributedPrivateClickMeasurement = "CREATE UNIQUE INDEX IF NOT EXISTS AttributedPrivateClickMeasurement_sourceSiteDomainID_destinationSiteDomainID on AttributedPrivateClickMeasurement ( sourceSiteDomainID, destinationSiteDomainID )"_s;
</del><ins>+constexpr auto createUniqueIndexUnattributedPrivateClickMeasurement = "CREATE UNIQUE INDEX IF NOT EXISTS UnattributedPrivateClickMeasurement_sourceSiteDomainID_destinationSiteDomainID_sourceApplicationBundleID on UnattributedPrivateClickMeasurement ( sourceSiteDomainID, destinationSiteDomainID, sourceApplicationBundleID )"_s;
+constexpr auto createUniqueIndexAttributedPrivateClickMeasurement = "CREATE UNIQUE INDEX IF NOT EXISTS AttributedPrivateClickMeasurement_sourceSiteDomainID_destinationSiteDomainID_sourceApplicationBundleID on AttributedPrivateClickMeasurement ( sourceSiteDomainID, destinationSiteDomainID, sourceApplicationBundleID )"_s;
</ins><span class="cx"> constexpr auto createPCMObservedDomain = "CREATE TABLE PCMObservedDomains ("
</span><span class="cx">     "domainID INTEGER PRIMARY KEY, registrableDomain TEXT NOT NULL UNIQUE ON CONFLICT FAIL)"_s;
</span><span class="cx"> constexpr auto insertObservedDomainQuery = "INSERT INTO PCMObservedDomains (registrableDomain) VALUES (?)"_s;
</span><span class="lines">@@ -89,6 +89,7 @@
</span><span class="cx">     openDatabaseAndCreateSchemaIfNecessary();
</span><span class="cx">     enableForeignKeys();
</span><span class="cx">     addBundleIDColumnIfNecessary();
</span><ins>+    migrateDataToNewTablesIfNecessary();
</ins><span class="cx">     allDatabases().add(this);
</span><span class="cx"> }
</span><span class="cx"> 
</span><span class="lines">@@ -99,6 +100,28 @@
</span><span class="cx">     allDatabases().remove(this);
</span><span class="cx"> }
</span><span class="cx"> 
</span><ins>+const MemoryCompactLookupOnlyRobinHoodHashMap<String, TableAndIndexPair>& Database::expectedTableAndIndexQueries()
+{
+    static auto expectedTableAndIndexQueries = makeNeverDestroyed(MemoryCompactLookupOnlyRobinHoodHashMap<String, TableAndIndexPair> {
+        { "PCMObservedDomains"_s, std::make_pair<String, std::optional<String>>(createPCMObservedDomain, std::nullopt) },
+        { "UnattributedPrivateClickMeasurement"_s, std::make_pair<String, std::optional<String>>(createUnattributedPrivateClickMeasurement, stripIndexQueryToMatchStoredValue(createUniqueIndexUnattributedPrivateClickMeasurement)) },
+        { "AttributedPrivateClickMeasurement"_s, std::make_pair<String, std::optional<String>>(createAttributedPrivateClickMeasurement, stripIndexQueryToMatchStoredValue(createUniqueIndexAttributedPrivateClickMeasurement)) },
+    });
+
+    return expectedTableAndIndexQueries;
+}
+
+const Vector<String>& Database::sortedTables()
+{
+    static auto sortedTables = makeNeverDestroyed(Vector<String> {
+        "PCMObservedDomains"_s,
+        "UnattributedPrivateClickMeasurement"_s,
+        "AttributedPrivateClickMeasurement"_s
+    });
+
+    return sortedTables;
+}
+
</ins><span class="cx"> void Database::interruptAllDatabases()
</span><span class="cx"> {
</span><span class="cx">     ASSERT(!RunLoop::isMain());
</span><span class="lines">@@ -106,6 +129,16 @@
</span><span class="cx">         database->interrupt();
</span><span class="cx"> }
</span><span class="cx"> 
</span><ins>+bool Database::createUniqueIndices()
+{
+    if (!m_database.executeCommand(createUniqueIndexUnattributedPrivateClickMeasurement)
+        || !m_database.executeCommand(createUniqueIndexAttributedPrivateClickMeasurement)) {
+        LOG_ERROR("Error creating indexes");
+        return false;
+    }
+    return true;
+}
+
</ins><span class="cx"> bool Database::createSchema()
</span><span class="cx"> {
</span><span class="cx">     ASSERT(!RunLoop::isMain());
</span><span class="lines">@@ -674,7 +707,17 @@
</span><span class="cx">     checkColumns(attributedTableName);
</span><span class="cx">     checkColumns(unattributedTableName);
</span><span class="cx"> }
</span><ins>+bool Database::needsUpdatedSchema()
+{
+    // FIXME: Remove this at the end of 2021. No public release was made with the schema missing sourceApplicationBundleID, so this is only needed to migrate internal users who updated in September 2021.
+    for (auto& table : expectedTableAndIndexQueries().keys()) {
+        if (currentTableAndIndexQueries(table) != expectedTableAndIndexQueries().get(table))
+            return true;
+    }
</ins><span class="cx"> 
</span><ins>+    return false;
+}
+
</ins><span class="cx"> Vector<String> Database::columnsForTable(const String& tableName)
</span><span class="cx"> {
</span><span class="cx">     auto statement = m_database.prepareStatementSlow(makeString("PRAGMA table_info(", tableName, ")"));
</span></span></pre></div>
<a id="trunkSourceWebKitNetworkProcessPrivateClickMeasurementPrivateClickMeasurementDatabaseh"></a>
<div class="modfile"><h4>Modified: trunk/Source/WebKit/NetworkProcess/PrivateClickMeasurement/PrivateClickMeasurementDatabase.h (283193 => 283194)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/WebKit/NetworkProcess/PrivateClickMeasurement/PrivateClickMeasurementDatabase.h     2021-09-28 21:10:21 UTC (rev 283193)
+++ trunk/Source/WebKit/NetworkProcess/PrivateClickMeasurement/PrivateClickMeasurementDatabase.h        2021-09-28 21:13:01 UTC (rev 283194)
</span><span class="lines">@@ -77,6 +77,11 @@
</span><span class="cx">     String getDomainStringFromDomainID(DomainID) const final;
</span><span class="cx"> 
</span><span class="cx">     void addBundleIDColumnIfNecessary();
</span><ins>+    bool needsUpdatedSchema() final;
+    bool createUniqueIndices() final;
+    const MemoryCompactLookupOnlyRobinHoodHashMap<String, TableAndIndexPair>& expectedTableAndIndexQueries() final;
+    const Vector<String>& sortedTables() final;
+
</ins><span class="cx">     Vector<String> columnsForTable(const String& tableName);
</span><span class="cx">     void addMissingColumnToTable(const String& tableName, const String& columnName);
</span><span class="cx"> 
</span></span></pre></div>
<a id="trunkToolsChangeLog"></a>
<div class="modfile"><h4>Modified: trunk/Tools/ChangeLog (283193 => 283194)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Tools/ChangeLog    2021-09-28 21:10:21 UTC (rev 283193)
+++ trunk/Tools/ChangeLog       2021-09-28 21:13:01 UTC (rev 283194)
</span><span class="lines">@@ -1,3 +1,25 @@
</span><ins>+2021-09-28  Kate Cheney  <katherine_cheney@apple.com>
+
+        PCM: different bundleID entries will override each other
+        https://bugs.webkit.org/show_bug.cgi?id=230839
+
+        Reviewed by Alex Christensen.
+
+        API test coverage for the case of existing PCM data with a bundleID
+        column but an expired unique index.
+
+        * TestWebKitAPI/Tests/WebKitCocoa/PrivateClickMeasurement.mm:
+        (addUnattributedPCMv4):
+        (addAttributedPCMv4):
+        (dumpedPCM):
+        (pollUntilPCMIsMigrated):
+        (emptyPcmDBPath):
+        (createAndPopulatePCMObservedDomainTable):
+        (setUpFromResourceLoadStatisticsDatabase):
+        (setUpFromPCMDatabase):
+        (TEST):
+        (setUp): Deleted.
+
</ins><span class="cx"> 2021-09-28  Eddy Wong  <eddy_wong@apple.com>
</span><span class="cx"> 
</span><span class="cx">         Added GlobalSign R3/R5 Root CA cert to webkitcorepy to resolve certain pip module download SSL error.
</span></span></pre></div>
<a id="trunkToolsTestWebKitAPITestsWebKitCocoaPrivateClickMeasurementmm"></a>
<div class="modfile"><h4>Modified: trunk/Tools/TestWebKitAPI/Tests/WebKitCocoa/PrivateClickMeasurement.mm (283193 => 283194)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Tools/TestWebKitAPI/Tests/WebKitCocoa/PrivateClickMeasurement.mm   2021-09-28 21:10:21 UTC (rev 283193)
+++ trunk/Tools/TestWebKitAPI/Tests/WebKitCocoa/PrivateClickMeasurement.mm      2021-09-28 21:13:01 UTC (rev 283194)
</span><span class="lines">@@ -164,6 +164,49 @@
</span><span class="cx">     addValuesToTable<7>(database, insertUnattributedPrivateClickMeasurementQueryV3, { 2, 3, 43, 1.0, "test token", "test signature", "test key id" });
</span><span class="cx"> }
</span><span class="cx"> 
</span><ins>+static void addUnattributedPCMv4(WebCore::SQLiteDatabase& database)
+{
+    constexpr auto createUnattributedPrivateClickMeasurementV4 = "CREATE TABLE UnattributedPrivateClickMeasurement ("
+        "sourceSiteDomainID INTEGER NOT NULL, destinationSiteDomainID INTEGER NOT NULL, sourceID INTEGER NOT NULL, "
+        "timeOfAdClick REAL NOT NULL, token TEXT, signature TEXT, keyID TEXT, sourceApplicationBundleID TEXT, FOREIGN KEY(sourceSiteDomainID) "
+        "REFERENCES PCMObservedDomains(domainID) ON DELETE CASCADE, FOREIGN KEY(destinationSiteDomainID) REFERENCES "
+        "PCMObservedDomains(domainID) ON DELETE CASCADE)"_s;
+
+    EXPECT_TRUE(database.executeCommand(createUnattributedPrivateClickMeasurementV4));
+
+    constexpr auto insertUnattributedPrivateClickMeasurementQueryV4 = "INSERT OR REPLACE INTO UnattributedPrivateClickMeasurement (sourceSiteDomainID, destinationSiteDomainID, "
+        "sourceID, timeOfAdClick, token, signature, keyID, sourceApplicationBundleID) VALUES (?, ?, ?, ?, ?, ?, ?, ?)"_s;
+    
+#if PLATFORM(MAC)
+    auto bundleID = "com.apple.Safari";
+#else
+    auto bundleID = "com.apple.mobilesafari";
+#endif
+    addValuesToTable<8>(database, insertUnattributedPrivateClickMeasurementQueryV4, { 2, 3, 43, 1.0, "test token", "test signature", "test key id", bundleID });
+}
+
+static void addAttributedPCMv4(WebCore::SQLiteDatabase& database)
+{
+    constexpr auto createAttributedPrivateClickMeasurementV4 = "CREATE TABLE AttributedPrivateClickMeasurement ("
+        "sourceSiteDomainID INTEGER NOT NULL, destinationSiteDomainID INTEGER NOT NULL, sourceID INTEGER NOT NULL, "
+        "attributionTriggerData INTEGER NOT NULL, priority INTEGER NOT NULL, timeOfAdClick REAL NOT NULL, "
+        "earliestTimeToSendToSource REAL, token TEXT, signature TEXT, keyID TEXT, earliestTimeToSendToDestination REAL, sourceApplicationBundleID TEXT,"
+        "FOREIGN KEY(sourceSiteDomainID) REFERENCES PCMObservedDomains(domainID) ON DELETE CASCADE, FOREIGN KEY(destinationSiteDomainID) REFERENCES "
+        "PCMObservedDomains(domainID) ON DELETE CASCADE)"_s;
+
+    EXPECT_TRUE(database.executeCommand(createAttributedPrivateClickMeasurementV4));
+
+    constexpr auto insertAttributedPrivateClickMeasurementQueryV4 = "INSERT OR REPLACE INTO AttributedPrivateClickMeasurement (sourceSiteDomainID, destinationSiteDomainID, "
+        "sourceID, attributionTriggerData, priority, timeOfAdClick, earliestTimeToSendToSource, token, signature, keyID, earliestTimeToSendToDestination, sourceApplicationBundleID) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)"_s;
+
+#if PLATFORM(MAC)
+    auto bundleID = "com.apple.Safari";
+#else
+    auto bundleID = "com.apple.mobilesafari";
+#endif
+    addValuesToTable<12>(database, insertAttributedPrivateClickMeasurementQueryV4, { 1, 2, 42, 14, 7, 1.0, earliestTimeToSend(), "test token", "test signature", "test key id", earliestTimeToSend(), bundleID });
+}
+
</ins><span class="cx"> static RetainPtr<NSString> dumpedPCM(WKWebView *webView)
</span><span class="cx"> {
</span><span class="cx">     __block RetainPtr<NSString> pcm;
</span><span class="lines">@@ -172,14 +215,18 @@
</span><span class="cx">     }];
</span><span class="cx">     while (!pcm)
</span><span class="cx">         TestWebKitAPI::Util::spinRunLoop();
</span><ins>+    
</ins><span class="cx">     return pcm;
</span><span class="cx"> }
</span><span class="cx"> 
</span><del>-static void pollUntilPCMIsMigrated(WKWebView *webView)
</del><ins>+enum class MigratingFromResourceLoadStatistics : bool { No, Yes };
+static void pollUntilPCMIsMigrated(WKWebView *webView, MigratingFromResourceLoadStatistics migratingFromResourceLoadStatistics)
</ins><span class="cx"> {
</span><del>-    // This query is the first thing to open the old database, so migration has not happened yet.
-    const char* emptyPCMDatabase = "\nNo stored Private Click Measurement data.\n";
-    EXPECT_WK_STREQ(dumpedPCM(webView).get(), emptyPCMDatabase);
</del><ins>+    if (migratingFromResourceLoadStatistics == MigratingFromResourceLoadStatistics::Yes) {
+        // This query is the first thing to open the old database, so migration has not happened yet.
+        const char* emptyPCMDatabase = "\nNo stored Private Click Measurement data.\n";
+        EXPECT_WK_STREQ(dumpedPCM(webView).get(), emptyPCMDatabase);
+    }
</ins><span class="cx"> 
</span><span class="cx">     NSString *expectedMigratedPCMDatabase = @""
</span><span class="cx">         "Unattributed Private Click Measurements:\n"
</span><span class="lines">@@ -224,6 +271,17 @@
</span><span class="cx">     return fileURL.path;
</span><span class="cx"> }
</span><span class="cx"> 
</span><ins>+static NSString *emptyPcmDBPath()
+{
+    NSFileManager *defaultFileManager = NSFileManager.defaultManager;
+    NSURL *itpRootURL = WKWebsiteDataStore.defaultDataStore._configuration._resourceLoadStatisticsDirectory;
+    NSURL *fileURL = [itpRootURL URLByAppendingPathComponent:@"pcm.db"];
+    [defaultFileManager removeItemAtPath:itpRootURL.path error:nil];
+    EXPECT_FALSE([defaultFileManager fileExistsAtPath:itpRootURL.path]);
+    [defaultFileManager createDirectoryAtURL:itpRootURL withIntermediateDirectories:YES attributes:nil error:nil];
+    return fileURL.path;
+}
+
</ins><span class="cx"> static void cleanUp()
</span><span class="cx"> {
</span><span class="cx">     NSFileManager *defaultFileManager = NSFileManager.defaultManager;
</span><span class="lines">@@ -254,8 +312,24 @@
</span><span class="cx">     addObservedDomain("www.webkit.org");
</span><span class="cx"> }
</span><span class="cx"> 
</span><del>-void setUp(void(*addUnattributedPCM)(WebCore::SQLiteDatabase&), void(*addAttributedPCM)(WebCore::SQLiteDatabase&))
</del><ins>+static void createAndPopulatePCMObservedDomainTable(WebCore::SQLiteDatabase& database)
</ins><span class="cx"> {
</span><ins>+    auto addObservedDomain = [&](const char* domain) {
+        constexpr auto insertObservedDomainQuery = "INSERT INTO PCMObservedDomains (registrableDomain) VALUES (?)"_s;
+        addValuesToTable<1>(database, insertObservedDomainQuery, { domain });
+    };
+
+    constexpr auto createPCMObservedDomain = "CREATE TABLE PCMObservedDomains ("
+        "domainID INTEGER PRIMARY KEY, registrableDomain TEXT NOT NULL UNIQUE ON CONFLICT FAIL)"_s;
+
+    EXPECT_TRUE(database.executeCommand(createPCMObservedDomain));
+    addObservedDomain("example.com");
+    addObservedDomain("webkit.org");
+    addObservedDomain("www.webkit.org");
+}
+
+void setUpFromResourceLoadStatisticsDatabase(void(*addUnattributedPCM)(WebCore::SQLiteDatabase&), void(*addAttributedPCM)(WebCore::SQLiteDatabase&))
+{
</ins><span class="cx">     WebCore::SQLiteDatabase database;
</span><span class="cx">     EXPECT_TRUE(database.open(emptyObservationsDBPath()));
</span><span class="cx">     createAndPopulateObservedDomainTable(database);
</span><span class="lines">@@ -264,26 +338,44 @@
</span><span class="cx">     database.close();
</span><span class="cx"> }
</span><span class="cx"> 
</span><ins>+void setUpFromPCMDatabase(void(*addUnattributedPCM)(WebCore::SQLiteDatabase&), void(*addAttributedPCM)(WebCore::SQLiteDatabase&))
+{
+    WebCore::SQLiteDatabase database;
+    EXPECT_TRUE(database.open(emptyPcmDBPath()));
+    createAndPopulatePCMObservedDomainTable(database);
+    addUnattributedPCM(database);
+    addAttributedPCM(database);
+    database.close();
+}
+
</ins><span class="cx"> TEST(PrivateClickMeasurement, MigrateFromResourceLoadStatistics1)
</span><span class="cx"> {
</span><del>-    setUp(addUnattributedPCMv1, addAttributedPCMv1);
</del><ins>+    setUpFromResourceLoadStatisticsDatabase(addUnattributedPCMv1, addAttributedPCMv1);
</ins><span class="cx">     auto webView = webViewWithResourceLoadStatisticsEnabledInNetworkProcess();
</span><del>-    pollUntilPCMIsMigrated(webView.get());
</del><ins>+    pollUntilPCMIsMigrated(webView.get(), MigratingFromResourceLoadStatistics::Yes);
</ins><span class="cx">     cleanUp();
</span><span class="cx"> }
</span><span class="cx"> 
</span><span class="cx"> TEST(PrivateClickMeasurement, MigrateFromResourceLoadStatistics2)
</span><span class="cx"> {
</span><del>-    setUp(addUnattributedPCMv2, addAttributedPCMv2);
</del><ins>+    setUpFromResourceLoadStatisticsDatabase(addUnattributedPCMv2, addAttributedPCMv2);
</ins><span class="cx">     auto webView = webViewWithResourceLoadStatisticsEnabledInNetworkProcess();
</span><del>-    pollUntilPCMIsMigrated(webView.get());
</del><ins>+    pollUntilPCMIsMigrated(webView.get(), MigratingFromResourceLoadStatistics::Yes);
</ins><span class="cx">     cleanUp();
</span><span class="cx"> }
</span><span class="cx"> 
</span><span class="cx"> TEST(PrivateClickMeasurement, MigrateFromResourceLoadStatistics3)
</span><span class="cx"> {
</span><del>-    setUp(addUnattributedPCMv3, addAttributedPCMv3);
</del><ins>+    setUpFromResourceLoadStatisticsDatabase(addUnattributedPCMv3, addAttributedPCMv3);
</ins><span class="cx">     auto webView = webViewWithResourceLoadStatisticsEnabledInNetworkProcess();
</span><del>-    pollUntilPCMIsMigrated(webView.get());
</del><ins>+    pollUntilPCMIsMigrated(webView.get(), MigratingFromResourceLoadStatistics::Yes);
</ins><span class="cx">     cleanUp();
</span><span class="cx"> }
</span><ins>+
+TEST(PrivateClickMeasurement, MigrateFromPCM1)
+{
+    setUpFromPCMDatabase(addUnattributedPCMv4, addAttributedPCMv4);
+    auto webView = webViewWithResourceLoadStatisticsEnabledInNetworkProcess();
+    pollUntilPCMIsMigrated(webView.get(), MigratingFromResourceLoadStatistics::No);
+    cleanUp();
+}
</ins></span></pre>
</div>
</div>

</body>
</html>