<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
"http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head><meta http-equiv="content-type" content="text/html; charset=utf-8" />
<title>[276997] trunk/Tools</title>
</head>
<body>

<style type="text/css"><!--
#msg dl.meta { border: 1px #006 solid; background: #369; padding: 6px; color: #fff; }
#msg dl.meta dt { float: left; width: 6em; font-weight: bold; }
#msg dt:after { content:':';}
#msg dl, #msg dt, #msg ul, #msg li, #header, #footer, #logmsg { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt;  }
#msg dl a { font-weight: bold}
#msg dl a:link    { color:#fc3; }
#msg dl a:active  { color:#ff0; }
#msg dl a:visited { color:#cc6; }
h3 { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt; font-weight: bold; }
#msg pre { overflow: auto; background: #ffc; border: 1px #fa0 solid; padding: 6px; }
#logmsg { background: #ffc; border: 1px #fa0 solid; padding: 1em 1em 0 1em; }
#logmsg p, #logmsg pre, #logmsg blockquote { margin: 0 0 1em 0; }
#logmsg p, #logmsg li, #logmsg dt, #logmsg dd { line-height: 14pt; }
#logmsg h1, #logmsg h2, #logmsg h3, #logmsg h4, #logmsg h5, #logmsg h6 { margin: .5em 0; }
#logmsg h1:first-child, #logmsg h2:first-child, #logmsg h3:first-child, #logmsg h4:first-child, #logmsg h5:first-child, #logmsg h6:first-child { margin-top: 0; }
#logmsg ul, #logmsg ol { padding: 0; list-style-position: inside; margin: 0 0 0 1em; }
#logmsg ul { text-indent: -1em; padding-left: 1em; }#logmsg ol { text-indent: -1.5em; padding-left: 1.5em; }
#logmsg > ul, #logmsg > ol { margin: 0 0 1em 0; }
#logmsg pre { background: #eee; padding: 1em; }
#logmsg blockquote { border: 1px solid #fa0; border-left-width: 10px; padding: 1em 1em 0 1em; background: white;}
#logmsg dl { margin: 0; }
#logmsg dt { font-weight: bold; }
#logmsg dd { margin: 0; padding: 0 0 0.5em 0; }
#logmsg dd:before { content:'\00bb';}
#logmsg table { border-spacing: 0px; border-collapse: collapse; border-top: 4px solid #fa0; border-bottom: 1px solid #fa0; background: #fff; }
#logmsg table th { text-align: left; font-weight: normal; padding: 0.2em 0.5em; border-top: 1px dotted #fa0; }
#logmsg table td { text-align: right; border-top: 1px dotted #fa0; padding: 0.2em 0.5em; }
#logmsg table thead th { text-align: center; border-bottom: 1px solid #fa0; }
#logmsg table th.Corner { text-align: left; }
#logmsg hr { border: none 0; border-top: 2px dashed #fa0; height: 1px; }
#header, #footer { color: #fff; background: #636; border: 1px #300 solid; padding: 6px; }
#patch { width: 100%; }
#patch h4 {font-family: verdana,arial,helvetica,sans-serif;font-size:10pt;padding:8px;background:#369;color:#fff;margin:0;}
#patch .propset h4, #patch .binary h4 {margin:0;}
#patch pre {padding:0;line-height:1.2em;margin:0;}
#patch .diff {width:100%;background:#eee;padding: 0 0 10px 0;overflow:auto;}
#patch .propset .diff, #patch .binary .diff  {padding:10px 0;}
#patch span {display:block;padding:0 10px;}
#patch .modfile, #patch .addfile, #patch .delfile, #patch .propset, #patch .binary, #patch .copfile {border:1px solid #ccc;margin:10px 0;}
#patch ins {background:#dfd;text-decoration:none;display:block;padding:0 10px;}
#patch del {background:#fdd;text-decoration:none;display:block;padding:0 10px;}
#patch .lines, .info {color:#888;background:#fff;}
--></style>
<div id="msg">
<dl class="meta">
<dt>Revision</dt> <dd><a href="http://trac.webkit.org/projects/webkit/changeset/276997">276997</a></dd>
<dt>Author</dt> <dd>jbedard@apple.com</dd>
<dt>Date</dt> <dd>2021-05-04 17:00:14 -0700 (Tue, 04 May 2021)</dd>
</dl>

<h3>Log Message</h3>
<pre>[webkitscmpy] Scope classes used in TaskPool to object
https://bugs.webkit.org/show_bug.cgi?id=225361
<rdar://problem/77514131>

Reviewed by Dewei Zhu.

The classes used by TaskPool must be stand-alone because they are passed between processes.
However, it makes sense that those classes are scoped to the TaskPool object since their
usage should always be associated with the TaskPool object

* Scripts/libraries/webkitcorepy/setup.py:
* Scripts/libraries/webkitcorepy/webkitcorepy/__init__.py:
* Scripts/libraries/webkitcorepy/webkitcorepy/task_pool.py:
(_Message): Renamed from Message.
(_Task): Renamed from Task.
(_Result): Renamed from Result.
(_Log): Renamed from Log.
(_Print): Renamed from Print.
(_State): Renamed from State.
(_ChildException): Renamed from ChildException.
(_BiDirectionalQueue): Renamed from BiDirectionalQueue.
(_Process): Renamed from Process.
(TaskPool):
(TaskPool.do):
(Message): Renamed to _Message..
(Task): Renamed to _Task.
(Result): Renamed to _Result.
(Log): Renamed to _Log.
(Print): Renamed to _Print.
(State): Renamed to _State.
(ChildException): Renamed to _ChildException.
(BiDirectionalQueue): Renamed to _BiDirectionalQueue.
(Process): Renamed to _Process.</pre>

<h3>Modified Paths</h3>
<ul>
<li><a href="#trunkToolsChangeLog">trunk/Tools/ChangeLog</a></li>
<li><a href="#trunkToolsScriptslibrarieswebkitcorepysetuppy">trunk/Tools/Scripts/libraries/webkitcorepy/setup.py</a></li>
<li><a href="#trunkToolsScriptslibrarieswebkitcorepywebkitcorepy__init__py">trunk/Tools/Scripts/libraries/webkitcorepy/webkitcorepy/__init__.py</a></li>
<li><a href="#trunkToolsScriptslibrarieswebkitcorepywebkitcorepytask_poolpy">trunk/Tools/Scripts/libraries/webkitcorepy/webkitcorepy/task_pool.py</a></li>
</ul>

</div>
<div id="patch">
<h3>Diff</h3>
<a id="trunkToolsChangeLog"></a>
<div class="modfile"><h4>Modified: trunk/Tools/ChangeLog (276996 => 276997)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Tools/ChangeLog    2021-05-04 23:58:22 UTC (rev 276996)
+++ trunk/Tools/ChangeLog       2021-05-05 00:00:14 UTC (rev 276997)
</span><span class="lines">@@ -1,3 +1,39 @@
</span><ins>+2021-05-04  Jonathan Bedard  <jbedard@apple.com>
+
+        [webkitscmpy] Scope classes used in TaskPool to object
+        https://bugs.webkit.org/show_bug.cgi?id=225361
+        <rdar://problem/77514131>
+
+        Reviewed by Dewei Zhu.
+
+        The classes used by TaskPool must be stand-alone because they are passed between processes.
+        However, it makes sense that those classes are scoped to the TaskPool object since their
+        usage should always be associated with the TaskPool object
+
+        * Scripts/libraries/webkitcorepy/setup.py:
+        * Scripts/libraries/webkitcorepy/webkitcorepy/__init__.py:
+        * Scripts/libraries/webkitcorepy/webkitcorepy/task_pool.py:
+        (_Message): Renamed from Message.
+        (_Task): Renamed from Task.
+        (_Result): Renamed from Result.
+        (_Log): Renamed from Log.
+        (_Print): Renamed from Print.
+        (_State): Renamed from State.
+        (_ChildException): Renamed from ChildException.
+        (_BiDirectionalQueue): Renamed from BiDirectionalQueue.
+        (_Process): Renamed from Process.
+        (TaskPool):
+        (TaskPool.do):
+        (Message): Renamed to _Message..
+        (Task): Renamed to _Task.
+        (Result): Renamed to _Result.
+        (Log): Renamed to _Log.
+        (Print): Renamed to _Print.
+        (State): Renamed to _State.
+        (ChildException): Renamed to _ChildException.
+        (BiDirectionalQueue): Renamed to _BiDirectionalQueue.
+        (Process): Renamed to _Process.
+
</ins><span class="cx"> 2021-05-04  Chris Dumez  <cdumez@apple.com>
</span><span class="cx"> 
</span><span class="cx">         Unreviewed, try and re-enable AppleLanguagesTest.UpdateAppleLanguages on Apple Silicon.
</span></span></pre></div>
<a id="trunkToolsScriptslibrarieswebkitcorepysetuppy"></a>
<div class="modfile"><h4>Modified: trunk/Tools/Scripts/libraries/webkitcorepy/setup.py (276996 => 276997)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Tools/Scripts/libraries/webkitcorepy/setup.py      2021-05-04 23:58:22 UTC (rev 276996)
+++ trunk/Tools/Scripts/libraries/webkitcorepy/setup.py 2021-05-05 00:00:14 UTC (rev 276997)
</span><span class="lines">@@ -30,7 +30,7 @@
</span><span class="cx"> 
</span><span class="cx"> setup(
</span><span class="cx">     name='webkitcorepy',
</span><del>-    version='0.5.12',
</del><ins>+    version='0.5.13',
</ins><span class="cx">     description='Library containing various Python support classes and functions.',
</span><span class="cx">     long_description=readme(),
</span><span class="cx">     classifiers=[
</span></span></pre></div>
<a id="trunkToolsScriptslibrarieswebkitcorepywebkitcorepy__init__py"></a>
<div class="modfile"><h4>Modified: trunk/Tools/Scripts/libraries/webkitcorepy/webkitcorepy/__init__.py (276996 => 276997)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Tools/Scripts/libraries/webkitcorepy/webkitcorepy/__init__.py      2021-05-04 23:58:22 UTC (rev 276996)
+++ trunk/Tools/Scripts/libraries/webkitcorepy/webkitcorepy/__init__.py 2021-05-05 00:00:14 UTC (rev 276997)
</span><span class="lines">@@ -37,7 +37,7 @@
</span><span class="cx"> from webkitcorepy.task_pool import TaskPool
</span><span class="cx"> from webkitcorepy.credentials import credentials
</span><span class="cx"> 
</span><del>-version = Version(0, 5, 12)
</del><ins>+version = Version(0, 5, 13)
</ins><span class="cx"> 
</span><span class="cx"> from webkitcorepy.autoinstall import Package, AutoInstall
</span><span class="cx"> if sys.version_info > (3, 0):
</span></span></pre></div>
<a id="trunkToolsScriptslibrarieswebkitcorepywebkitcorepytask_poolpy"></a>
<div class="modfile"><h4>Modified: trunk/Tools/Scripts/libraries/webkitcorepy/webkitcorepy/task_pool.py (276996 => 276997)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Tools/Scripts/libraries/webkitcorepy/webkitcorepy/task_pool.py     2021-05-04 23:58:22 UTC (rev 276996)
+++ trunk/Tools/Scripts/libraries/webkitcorepy/webkitcorepy/task_pool.py        2021-05-05 00:00:14 UTC (rev 276997)
</span><span class="lines">@@ -30,17 +30,17 @@
</span><span class="cx"> from webkitcorepy import OutputCapture, Timeout, log
</span><span class="cx"> 
</span><span class="cx"> 
</span><del>-class Message(object):
</del><ins>+class _Message(object):
</ins><span class="cx">     def __init__(self, who=None):
</span><del>-        self.who = who or Process.name
</del><ins>+        self.who = who or _Process.name
</ins><span class="cx"> 
</span><span class="cx">     def __call__(self, caller):
</span><span class="cx">         NotImplemented()
</span><span class="cx"> 
</span><span class="cx"> 
</span><del>-class Task(Message):
</del><ins>+class _Task(_Message):
</ins><span class="cx">     def __init__(self, function, id, *args, **kwargs):
</span><del>-        super(Task, self).__init__()
</del><ins>+        super(_Task, self).__init__()
</ins><span class="cx"> 
</span><span class="cx">         self.function = function
</span><span class="cx">         self.id = id
</span><span class="lines">@@ -51,9 +51,9 @@
</span><span class="cx">         return self.function(*self.args, **self.kwargs)
</span><span class="cx"> 
</span><span class="cx"> 
</span><del>-class Result(Message):
</del><ins>+class _Result(_Message):
</ins><span class="cx">     def __init__(self, value, id):
</span><del>-        super(Result, self).__init__()
</del><ins>+        super(_Result, self).__init__()
</ins><span class="cx">         self.value = value
</span><span class="cx">         self.id = id
</span><span class="cx"> 
</span><span class="lines">@@ -63,9 +63,9 @@
</span><span class="cx">         return self.value
</span><span class="cx"> 
</span><span class="cx"> 
</span><del>-class Log(Message):
</del><ins>+class _Log(_Message):
</ins><span class="cx">     def __init__(self, record):
</span><del>-        super(Log, self).__init__()
</del><ins>+        super(_Log, self).__init__()
</ins><span class="cx">         self.record = record
</span><span class="cx"> 
</span><span class="cx">     def __call__(self, caller):
</span><span class="lines">@@ -72,12 +72,12 @@
</span><span class="cx">         logging.getLogger(self.record.name).log(self.record.levelno, '{} {}'.format(self.who, self.record.getMessage()))
</span><span class="cx"> 
</span><span class="cx"> 
</span><del>-class Print(Message):
</del><ins>+class _Print(_Message):
</ins><span class="cx">     stdout = 1
</span><span class="cx">     stderr = 2
</span><span class="cx"> 
</span><span class="cx">     def __init__(self, lines, stream=stdout):
</span><del>-        super(Print, self).__init__()
</del><ins>+        super(_Print, self).__init__()
</ins><span class="cx">         self.lines = lines
</span><span class="cx">         self.stream = stream
</span><span class="cx"> 
</span><span class="lines">@@ -90,32 +90,32 @@
</span><span class="cx">             stream.write('{}\n'.format(line))
</span><span class="cx"> 
</span><span class="cx"> 
</span><del>-class State(Message):
</del><ins>+class _State(_Message):
</ins><span class="cx">     STARTING, STOPPING = 1, 0
</span><span class="cx">     STATES = [STARTING, STOPPING]
</span><span class="cx"> 
</span><span class="cx">     def __init__(self, state):
</span><del>-        super(State, self).__init__()
</del><ins>+        super(_State, self).__init__()
</ins><span class="cx">         self.state = state
</span><span class="cx"> 
</span><span class="cx">     def __call__(self, caller):
</span><span class="cx">         log.info('{} {}'.format(
</span><span class="cx">             self.who, {
</span><del>-                State.STARTING: 'starting',
-                State.STOPPING: 'stopping',
</del><ins>+                self.STARTING: 'starting',
+                self.STOPPING: 'stopping',
</ins><span class="cx">             }.get(self.state, self.state),
</span><span class="cx">         ))
</span><span class="cx">         if caller:
</span><span class="cx">             caller._started += {
</span><del>-                State.STARTING: 1,
-                State.STOPPING: -1,
</del><ins>+                self.STARTING: 1,
+                self.STOPPING: -1,
</ins><span class="cx">             }.get(self.state, 0)
</span><span class="cx">         return self.state
</span><span class="cx"> 
</span><span class="cx"> 
</span><del>-class ChildException(Message):
</del><ins>+class _ChildException(_Message):
</ins><span class="cx">     def __init__(self, exc_info=None):
</span><del>-        super(ChildException, self).__init__()
</del><ins>+        super(_ChildException, self).__init__()
</ins><span class="cx">         self.exc_info = exc_info or sys.exc_info()
</span><span class="cx"> 
</span><span class="cx">     def __call__(self, caller):
</span><span class="lines">@@ -128,7 +128,7 @@
</span><span class="cx">         reraise(*self.exc_info)
</span><span class="cx"> 
</span><span class="cx"> 
</span><del>-class BiDirectionalQueue(object):
</del><ins>+class _BiDirectionalQueue(object):
</ins><span class="cx">     def __init__(self, outgoing=None, incoming=None):
</span><span class="cx">         self.outgoing = outgoing or multiprocessing.Queue()
</span><span class="cx">         self.incoming = incoming or multiprocessing.Queue()
</span><span class="lines">@@ -147,17 +147,18 @@
</span><span class="cx">             return self.incoming.get()
</span><span class="cx"> 
</span><span class="cx"> 
</span><del>-class Process(object):
</del><ins>+class _Process(object):
</ins><span class="cx">     name = None
</span><span class="cx">     working = False
</span><ins>+    queue = None
</ins><span class="cx"> 
</span><span class="cx">     class LogHandler(logging.Handler):
</span><span class="cx">         def __init__(self, queue, **kwargs):
</span><span class="cx">             self._queue = queue
</span><del>-            super(Process.LogHandler, self).__init__(**kwargs)
</del><ins>+            super(_Process.LogHandler, self).__init__(**kwargs)
</ins><span class="cx"> 
</span><span class="cx">         def emit(self, record):
</span><del>-            self._queue.send(Log(record))
</del><ins>+            self._queue.send(_Log(record))
</ins><span class="cx"> 
</span><span class="cx">     class Stream(io.IOBase):
</span><span class="cx">         def __init__(self, handle, queue):
</span><span class="lines">@@ -169,7 +170,7 @@
</span><span class="cx"> 
</span><span class="cx">         def flush(self):
</span><span class="cx">             if self.cache is not None:
</span><del>-                self._queue.send(Print(lines=[self.cache], stream=self.handle))
</del><ins>+                self._queue.send(_Print(lines=[self.cache], stream=self.handle))
</ins><span class="cx">                 self.cache = None
</span><span class="cx"> 
</span><span class="cx">         def writelines(self, lines):
</span><span class="lines">@@ -185,7 +186,7 @@
</span><span class="cx">                 elif c not in ['\r', '\0']:
</span><span class="cx">                     self.cache = c if self.cache is None else (self.cache + c)
</span><span class="cx">             if to_be_printed:
</span><del>-                self._queue.send(Print(lines=to_be_printed, stream=self.handle))
</del><ins>+                self._queue.send(_Print(lines=to_be_printed, stream=self.handle))
</ins><span class="cx">             return len(data)
</span><span class="cx"> 
</span><span class="cx">         @property
</span><span class="lines">@@ -252,9 +253,10 @@
</span><span class="cx">         logger.addHandler(cls.LogHandler(queue))
</span><span class="cx">         logger.setLevel(loglevel)
</span><span class="cx"> 
</span><del>-        queue.send(State(State.STARTING))
</del><ins>+        cls.queue = queue
+        queue.send(_State(_State.STARTING))
</ins><span class="cx"> 
</span><del>-        with OutputCapture.ReplaceSysStream('stderr', cls.Stream(Print.stderr, queue)), OutputCapture.ReplaceSysStream('stdout', cls.Stream(Print.stdout, queue)):
</del><ins>+        with OutputCapture.ReplaceSysStream('stderr', cls.Stream(_Print.stderr, queue)), OutputCapture.ReplaceSysStream('stdout', cls.Stream(_Print.stdout, queue)):
</ins><span class="cx">             try:
</span><span class="cx">                 pickling_support.install()
</span><span class="cx">                 if setup:
</span><span class="lines">@@ -264,11 +266,11 @@
</span><span class="cx">                     task = queue.receive()
</span><span class="cx">                     if not task:
</span><span class="cx">                         break
</span><del>-                    queue.send(Result(value=task(None), id=task.id))
</del><ins>+                    queue.send(_Result(value=task(None), id=task.id))
</ins><span class="cx"> 
</span><span class="cx">             except BaseException:
</span><span class="cx">                 typ, exception, traceback = sys.exc_info()
</span><del>-                queue.send(ChildException(exc_info=(
</del><ins>+                queue.send(_ChildException(exc_info=(
</ins><span class="cx">                     typ, typ('{} (from {})'.format(str(exception), name)), traceback,
</span><span class="cx">                 )))
</span><span class="cx"> 
</span><span class="lines">@@ -277,11 +279,22 @@
</span><span class="cx">                     teardown(*teardownargs, **teardownkwargs)
</span><span class="cx">                 sys.stdout.flush()
</span><span class="cx">                 sys.stderr.flush()
</span><del>-                queue.send(State(State.STOPPING))
</del><ins>+                queue.send(_State(_State.STOPPING))
+                cls.queue = None
</ins><span class="cx"> 
</span><span class="cx"> 
</span><span class="cx"> class TaskPool(object):
</span><ins>+    Message = _Message
+    Task = _Task
+    Result = _Result
+    Log = _Log
+    Print = _Print
+    State = _State
+    ChildException = _ChildException
+    BiDirectionalQueue = _BiDirectionalQueue
+    Process = _Process
</ins><span class="cx"> 
</span><ins>+
</ins><span class="cx">     class Exception(RuntimeError):
</span><span class="cx">         pass
</span><span class="cx"> 
</span><span class="lines">@@ -294,20 +307,20 @@
</span><span class="cx">         import tblib
</span><span class="cx"> 
</span><span class="cx">         name = name or 'worker'
</span><del>-        if name == Process.name:
</del><ins>+        if name == self.Process.name:
</ins><span class="cx">             raise ValueError("Parent process is already named {}".format(name))
</span><span class="cx"> 
</span><span class="cx">         if workers < 1:
</span><span class="cx">             raise ValueError('TaskPool requires positive number of workers')
</span><span class="cx"> 
</span><del>-        self.queue = BiDirectionalQueue()
</del><ins>+        self.queue = self.BiDirectionalQueue()
</ins><span class="cx"> 
</span><span class="cx">         self.workers = [multiprocessing.Process(
</span><del>-            target=Process.main,
</del><ins>+            target=self.Process.main,
</ins><span class="cx">             args=(
</span><span class="cx">                 '{}/{}'.format(name, count), logging.getLogger().getEffectiveLevel(),
</span><span class="cx">                 setup, setupargs, setupkwargs,
</span><del>-                BiDirectionalQueue(outgoing=self.queue.incoming, incoming=self.queue.outgoing),
</del><ins>+                self.BiDirectionalQueue(outgoing=self.queue.incoming, incoming=self.queue.outgoing),
</ins><span class="cx">                 teardown, teardownargs, teardownkwargs,
</span><span class="cx">             ),
</span><span class="cx">         ) for count in range(workers)]
</span><span class="lines">@@ -330,7 +343,7 @@
</span><span class="cx">         callback = kwargs.pop('callback', None)
</span><span class="cx">         if callback:
</span><span class="cx">             self.callbacks[self._id_count] = callback
</span><del>-        self.queue.send(Task(function, self._id_count, *args, **kwargs))
</del><ins>+        self.queue.send(self.Task(function, self._id_count, *args, **kwargs))
</ins><span class="cx">         self._id_count += 1
</span><span class="cx"> 
</span><span class="cx">         # For every block of tasks passed to our workers, we need consume messages so we don't get deadlocked
</span></span></pre>
</div>
</div>

</body>
</html>