<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
"http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head><meta http-equiv="content-type" content="text/html; charset=utf-8" />
<title>[287949] trunk/Tools</title>
</head>
<body>

<style type="text/css"><!--
#msg dl.meta { border: 1px #006 solid; background: #369; padding: 6px; color: #fff; }
#msg dl.meta dt { float: left; width: 6em; font-weight: bold; }
#msg dt:after { content:':';}
#msg dl, #msg dt, #msg ul, #msg li, #header, #footer, #logmsg { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt;  }
#msg dl a { font-weight: bold}
#msg dl a:link    { color:#fc3; }
#msg dl a:active  { color:#ff0; }
#msg dl a:visited { color:#cc6; }
h3 { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt; font-weight: bold; }
#msg pre { overflow: auto; background: #ffc; border: 1px #fa0 solid; padding: 6px; }
#logmsg { background: #ffc; border: 1px #fa0 solid; padding: 1em 1em 0 1em; }
#logmsg p, #logmsg pre, #logmsg blockquote { margin: 0 0 1em 0; }
#logmsg p, #logmsg li, #logmsg dt, #logmsg dd { line-height: 14pt; }
#logmsg h1, #logmsg h2, #logmsg h3, #logmsg h4, #logmsg h5, #logmsg h6 { margin: .5em 0; }
#logmsg h1:first-child, #logmsg h2:first-child, #logmsg h3:first-child, #logmsg h4:first-child, #logmsg h5:first-child, #logmsg h6:first-child { margin-top: 0; }
#logmsg ul, #logmsg ol { padding: 0; list-style-position: inside; margin: 0 0 0 1em; }
#logmsg ul { text-indent: -1em; padding-left: 1em; }#logmsg ol { text-indent: -1.5em; padding-left: 1.5em; }
#logmsg > ul, #logmsg > ol { margin: 0 0 1em 0; }
#logmsg pre { background: #eee; padding: 1em; }
#logmsg blockquote { border: 1px solid #fa0; border-left-width: 10px; padding: 1em 1em 0 1em; background: white;}
#logmsg dl { margin: 0; }
#logmsg dt { font-weight: bold; }
#logmsg dd { margin: 0; padding: 0 0 0.5em 0; }
#logmsg dd:before { content:'\00bb';}
#logmsg table { border-spacing: 0px; border-collapse: collapse; border-top: 4px solid #fa0; border-bottom: 1px solid #fa0; background: #fff; }
#logmsg table th { text-align: left; font-weight: normal; padding: 0.2em 0.5em; border-top: 1px dotted #fa0; }
#logmsg table td { text-align: right; border-top: 1px dotted #fa0; padding: 0.2em 0.5em; }
#logmsg table thead th { text-align: center; border-bottom: 1px solid #fa0; }
#logmsg table th.Corner { text-align: left; }
#logmsg hr { border: none 0; border-top: 2px dashed #fa0; height: 1px; }
#header, #footer { color: #fff; background: #636; border: 1px #300 solid; padding: 6px; }
#patch { width: 100%; }
#patch h4 {font-family: verdana,arial,helvetica,sans-serif;font-size:10pt;padding:8px;background:#369;color:#fff;margin:0;}
#patch .propset h4, #patch .binary h4 {margin:0;}
#patch pre {padding:0;line-height:1.2em;margin:0;}
#patch .diff {width:100%;background:#eee;padding: 0 0 10px 0;overflow:auto;}
#patch .propset .diff, #patch .binary .diff  {padding:10px 0;}
#patch span {display:block;padding:0 10px;}
#patch .modfile, #patch .addfile, #patch .delfile, #patch .propset, #patch .binary, #patch .copfile {border:1px solid #ccc;margin:10px 0;}
#patch ins {background:#dfd;text-decoration:none;display:block;padding:0 10px;}
#patch del {background:#fdd;text-decoration:none;display:block;padding:0 10px;}
#patch .lines, .info {color:#888;background:#fff;}
--></style>
<div id="msg">
<dl class="meta">
<dt>Revision</dt> <dd><a href="http://trac.webkit.org/projects/webkit/changeset/287949">287949</a></dd>
<dt>Author</dt> <dd>jbedard@apple.com</dd>
<dt>Date</dt> <dd>2022-01-12 13:17:51 -0800 (Wed, 12 Jan 2022)</dd>
</dl>

<h3>Log Message</h3>
<pre>[EWS] Accept GitHub hooks (Part 1)
https://bugs.webkit.org/show_bug.cgi?id=235033
<rdar://problem/87345269>

Reviewed by Aakash Jain

Trigger a subset of queues for incoming GitHub pull-requests.

* Tools/CISupport/ews-build/config.json: Enable webkitperl, webkitpy, services and style checks.
* Tools/CISupport/ews-build/loadConfig_unittest.py:
(ConfigDotJSONTest):
(ConfigDotJSONTest.test_multiple_scheduers_for_builder):
* Tools/CISupport/ews-build/master.cfg: Accept GitHub hooks, specify variables
to extract from incoming request.
* Tools/CISupport/ews-build/steps.py:
(GitHub):
(GitHub.repository_urls): Concatenate all valid projects with the base URL.
(GitHub.pr_url): Given a repository and pull-request number, return the URL
for that pull-request.
(ConfigureBuild.add_pr_details):
(GitHubMixin): Deleted.

Canonical link: https://commits.webkit.org/245977@main</pre>

<h3>Modified Paths</h3>
<ul>
<li><a href="#trunkToolsCISupportewsbuildconfigjson">trunk/Tools/CISupport/ews-build/config.json</a></li>
<li><a href="#trunkToolsCISupportewsbuildloadConfig_unittestpy">trunk/Tools/CISupport/ews-build/loadConfig_unittest.py</a></li>
<li><a href="#trunkToolsCISupportewsbuildmastercfg">trunk/Tools/CISupport/ews-build/master.cfg</a></li>
<li><a href="#trunkToolsCISupportewsbuildstepspy">trunk/Tools/CISupport/ews-build/steps.py</a></li>
<li><a href="#trunkToolsChangeLog">trunk/Tools/ChangeLog</a></li>
</ul>

</div>
<div id="patch">
<h3>Diff</h3>
<a id="trunkToolsCISupportewsbuildconfigjson"></a>
<div class="modfile"><h4>Modified: trunk/Tools/CISupport/ews-build/config.json (287948 => 287949)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Tools/CISupport/ews-build/config.json      2022-01-12 21:17:49 UTC (rev 287948)
+++ trunk/Tools/CISupport/ews-build/config.json 2022-01-12 21:17:51 UTC (rev 287949)
</span><span class="lines">@@ -347,6 +347,13 @@
</span><span class="cx">       "builderNames": ["Commit-Queue"]
</span><span class="cx">     },
</span><span class="cx">     {
</span><ins>+      "type": "AnyBranchScheduler", "name": "pullrequest",
+      "builderNames": [
+            "Services-EWS", "Style-EWS", "Style-EWS",
+            "WebKitPerl-Tests-EWS", "WebKitPy-Tests-EWS"
+      ]
+    },
+    {
</ins><span class="cx">       "type": "Triggerable", "name": "macos-catalina-debug-build-ews",
</span><span class="cx">       "builderNames": ["macOS-Catalina-Debug-Build-EWS"]
</span><span class="cx">     },
</span></span></pre></div>
<a id="trunkToolsCISupportewsbuildloadConfig_unittestpy"></a>
<div class="modfile"><h4>Modified: trunk/Tools/CISupport/ews-build/loadConfig_unittest.py (287948 => 287949)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Tools/CISupport/ews-build/loadConfig_unittest.py   2022-01-12 21:17:49 UTC (rev 287948)
+++ trunk/Tools/CISupport/ews-build/loadConfig_unittest.py      2022-01-12 21:17:51 UTC (rev 287949)
</span><span class="lines">@@ -31,6 +31,8 @@
</span><span class="cx"> 
</span><span class="cx"> 
</span><span class="cx"> class ConfigDotJSONTest(unittest.TestCase):
</span><ins>+    DUPLICATED_TRIGGERS = ['try', 'pullrequest']
+
</ins><span class="cx">     def get_config(self):
</span><span class="cx">         cwd = os.path.dirname(os.path.abspath(__file__))
</span><span class="cx">         with open(os.path.join(cwd, 'config.json'), 'r') as config:
</span><span class="lines">@@ -72,7 +74,10 @@
</span><span class="cx">             if scheduler['name'] in triggered_by_schedulers:
</span><span class="cx">                 continue
</span><span class="cx">             for buildername in scheduler.get('builderNames'):
</span><del>-                self.assertTrue(buildername not in builder_to_schduler_map, 'builder {} appears multiple times in schedulers.'.format(buildername))
</del><ins>+                if scheduler.get('name') in self.DUPLICATED_TRIGGERS and builder_to_schduler_map.get(buildername):
+                    self.assertTrue(builder_to_schduler_map[buildername] in self.DUPLICATED_TRIGGERS)
+                else:
+                    self.assertTrue(buildername not in builder_to_schduler_map, 'builder {} appears multiple times in schedulers.'.format(buildername))
</ins><span class="cx">                 builder_to_schduler_map[buildername] = scheduler.get('name')
</span><span class="cx"> 
</span><span class="cx">     def test_schduler_contains_valid_builder_name(self):
</span></span></pre></div>
<a id="trunkToolsCISupportewsbuildmastercfg"></a>
<div class="modfile"><h4>Modified: trunk/Tools/CISupport/ews-build/master.cfg (287948 => 287949)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Tools/CISupport/ews-build/master.cfg       2022-01-12 21:17:49 UTC (rev 287948)
+++ trunk/Tools/CISupport/ews-build/master.cfg  2022-01-12 21:17:51 UTC (rev 287949)
</span><span class="lines">@@ -8,6 +8,18 @@
</span><span class="cx"> import loadConfig
</span><span class="cx"> from events import Events
</span><span class="cx"> 
</span><ins>+
+def load_password(name):
+    if os.getenv(name):
+        return os.getenv(name)
+    try:
+        passwords = json.load(open('passwords.json'))
+        return passwords.get(name, None)
+    except Exception as e:
+        print('Error in finding {} in passwords.json'.format(name))
+        return None
+
+
</ins><span class="cx"> # This is work-around for Twisted having a small size limit for patches. See https://bugs.webkit.org/show_bug.cgi?id=198851#c5
</span><span class="cx"> from twisted.spread import banana
</span><span class="cx"> banana.SIZE_LIMIT = 100 * 1024 * 1024
</span><span class="lines">@@ -28,6 +40,18 @@
</span><span class="cx">     'Builders.buildFetchLimit': 1000,
</span><span class="cx">     'Workers.showWorkerBuilders': True,
</span><span class="cx"> }
</span><ins>+c['www']['change_hook_dialects'] = dict(
+    github=dict(
+        secret=load_password('GITHUB_HOOK_SECRET'),
+        github_property_whitelist=[
+            'github.number',
+            'github.title',
+            'github.head.ref',
+            'github.base.sha',
+            'github.head.repo.full_name',
+        ],
+    ),
+)
</ins><span class="cx"> 
</span><span class="cx"> if not is_test_mode_enabled:
</span><span class="cx">     admin_username = os.getenv('EWS_ADMIN_USERNAME')
</span></span></pre></div>
<a id="trunkToolsCISupportewsbuildstepspy"></a>
<div class="modfile"><h4>Modified: trunk/Tools/CISupport/ews-build/steps.py (287948 => 287949)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Tools/CISupport/ews-build/steps.py 2022-01-12 21:17:49 UTC (rev 287948)
+++ trunk/Tools/CISupport/ews-build/steps.py    2022-01-12 21:17:51 UTC (rev 287949)
</span><span class="lines">@@ -55,7 +55,8 @@
</span><span class="cx"> WithProperties = properties.WithProperties
</span><span class="cx"> Interpolate = properties.Interpolate
</span><span class="cx"> BRANCH_PR_RE = re.compile(r'^refs/pull/(?P<id>\d+)/merge$')
</span><del>-GITHUB_REPOSITORIES = ['https://github.com/WebKit/WebKit']
</del><ins>+GITHUB_URL = 'https://github.com/'
+GITHUB_PROJECTS = ['WebKit/WebKit']
</ins><span class="cx"> 
</span><span class="cx"> 
</span><span class="cx"> class BufferLogHeaderObserver(logobserver.BufferLogObserver):
</span><span class="lines">@@ -71,32 +72,24 @@
</span><span class="cx">         return self._get(self.headers)
</span><span class="cx"> 
</span><span class="cx"> 
</span><del>-class GitHubMixin(object):
-    def pr_url(self, pr_number=None):
-        pr_number = pr_number or self.get_pull_request_number()
-        if not pr_number:
-            return ''
-        return '{}/pull/{}'.format(self.getProperty('repository', '-'), pr_number)
</del><ins>+class GitHub(object):
+    @classmethod
+    def repository_urls(cls):
+        return [GITHUB_URL + project for project in GITHUB_PROJECTS]
</ins><span class="cx"> 
</span><del>-    def get_pull_request_number(self):
-        pr_number = self.getProperty('pull_request')
-        if pr_number:
-            return int(pr_number)
</del><ins>+    @classmethod
+    def pr_url(cls, pr_number, repository_url=None):
+        if not repository_url:
+            repository_url = '{}{}'.format(GITHUB_URL, GITHUB_PROJECTS[0])
</ins><span class="cx"> 
</span><del>-        if self.getProperty('event') != 'pull_request':
-            return None
-        if self.getProperty('repository') not in GITHUB_REPOSITORIES:
-            return None
</del><ins>+        if repository_url not in GitHub.repository_urls():
+            return ''
+        if not pr_number or not isinstance(pr_number, int):
+            return ''
+        return '{}pull/{}'.format(repository_url, pr_number)
</ins><span class="cx"> 
</span><del>-        match = BRANCH_PR_RE.match(self.getProperty('branch', ''))
-        if not match:
-            return None
-        pr_number = int(match.group('id'))
-        self.setProperty('pull_request', pr_number)
-        return pr_number
</del><span class="cx"> 
</span><del>-
-class ConfigureBuild(buildstep.BuildStep, GitHubMixin):
</del><ins>+class ConfigureBuild(buildstep.BuildStep):
</ins><span class="cx">     name = 'configure-build'
</span><span class="cx">     description = ['configuring build']
</span><span class="cx">     descriptionDone = ['Configured build']
</span><span class="lines">@@ -147,9 +140,9 @@
</span><span class="cx">             self.addURL('Patch {}'.format(patch_id), Bugzilla.patch_url(patch_id))
</span><span class="cx"> 
</span><span class="cx">     def add_pr_details(self):
</span><del>-        pr_number = self.get_pull_request_number()
</del><ins>+        pr_number = self.getProperty('github.number')
</ins><span class="cx">         if pr_number:
</span><del>-            self.addURL('Pull request {}'.format(pr_number), self.pr_url(pr_number=pr_number))
</del><ins>+            self.addURL('Pull request {}'.format(pr_number), GitHub.pr_url(pr_number=pr_number, repository_url=self.getProperty('repository')) or '')
</ins><span class="cx"> 
</span><span class="cx"> 
</span><span class="cx"> class CheckOutSource(git.Git):
</span></span></pre></div>
<a id="trunkToolsChangeLog"></a>
<div class="modfile"><h4>Modified: trunk/Tools/ChangeLog (287948 => 287949)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Tools/ChangeLog    2022-01-12 21:17:49 UTC (rev 287948)
+++ trunk/Tools/ChangeLog       2022-01-12 21:17:51 UTC (rev 287949)
</span><span class="lines">@@ -1,3 +1,27 @@
</span><ins>+2022-01-12  Jonathan Bedard  <jbedard@apple.com>
+
+        [EWS] Accept GitHub hooks (Part 1)
+        https://bugs.webkit.org/show_bug.cgi?id=235033
+        <rdar://problem/87345269>
+
+        Reviewed by Aakash Jain.
+
+        Trigger a subset of queues for incoming GitHub pull-requests.
+
+        * CISupport/ews-build/config.json: Enable webkitperl, webkitpy, services and style checks.
+        * CISupport/ews-build/loadConfig_unittest.py:
+        (ConfigDotJSONTest):
+        (ConfigDotJSONTest.test_multiple_scheduers_for_builder):
+        * CISupport/ews-build/master.cfg: Accept GitHub hooks, specify variables
+        to extract from incoming request.
+        * CISupport/ews-build/steps.py:
+        (GitHub):
+        (GitHub.repository_urls): Concatenate all valid projects with the base URL.
+        (GitHub.pr_url): Given a repository and pull-request number, return the URL
+        for that pull-request.
+        (ConfigureBuild.add_pr_details):
+        (GitHubMixin): Deleted.
+
</ins><span class="cx"> 2022-01-12  Alex Christensen  <achristensen@webkit.org>
</span><span class="cx"> 
</span><span class="cx">         Build WebKitSwift when building with make or build-webkit
</span></span></pre>
</div>
</div>

</body>
</html>