<div dir="ltr">All of this makes sense, but the downsides is that it does not ensure convergence between the ports.<div><div>If this situation is expected to stay for some time, I wonder whether some middle ground may be found for ports that are build-stable.</div>
<div><br></div><div>For instance, having status information on new/modified tests for all ports would be useful and not that expensive.<br></div><div>When tests do not pass on some ports, updating the test expectations (and creating bug entries) would be an improvement over the current situation.<br>
</div><div><br></div><div><div class="gmail_extra"><br><div class="gmail_quote">2014-02-10 10:25 GMT+01:00 Osztrogonác Csaba <span dir="ltr"><<a href="mailto:oszi@inf.u-szeged.hu" target="_blank">oszi@inf.u-szeged.hu</a>></span>:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div>On 02/10/2014 09:51 AM, youenn fablet wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Is it by design that only mac bots run regression tests? Technical<br>
issue? Lack of resources?<br>
</blockquote>
<br></div>
Technically, it's so easy to make an EWS to run layout tests<br>
too with adding a "runTests": true" to the ews.json file:<br>
<a href="https://trac.webkit.org/browser/trunk/Tools/Scripts/webkitpy/common/config/ews.json" target="_blank">https://trac.webkit.org/<u></u>browser/trunk/Tools/Scripts/<u></u>webkitpy/common/config/ews.<u></u>json</a><br>
<br>
But in my opinion it wouldn't be a good idea to enable layout tests<br>
on Windows, GTK, EFL EWS bots, because it would make them absolutely<br>
useless and we would lost the information if a patch builds or not.<br>
<br>
To have a quite stable and working tester EWS, the buildbot for the<br>
given platform must be green _almost all the time_. If there is at<br>
least one failing test, the testing is at least twice slower, because<br>
the EWS runs the test with the patch and then without the patch to<br>
check if the list of the failing tests are same. Additionally the<br>
given port must be very stable. If there are any small flakiness,<br>
the EWS wouldn't pass ever and would stuck in an infinite loop.<br>
<br>
The rough true is that now only the Mac platform is stable and green<br>
enough to have tester EWS bots. (There are ~210 +/-5 failures on the<br>
Windows bots from the cstack merge, ~205 +/-10 failures on EFL-WK1 long<br>
time ago, ~80 +/- 2 failures on GTK-WK1 lone time ago, ~ 60 +/- 5<br>
failures on GTK-WK2, ...)<br>
<br>
Additionally to have tester EWS, port maintainers should have to setup<br>
many new hardware (min. 4-8 machines with 4/8 cores per port to have<br>
acceptable runtime) and EWS runtime would be much more slower than<br>
the runtime of build only EWS bots, because bulding + running tests<br>
take ~ an hour everywhere.<br>
<br>
Ossy<div><div><br>
<br>
______________________________<u></u>_________________<br>
webkit-dev mailing list<br>
<a href="mailto:webkit-dev@lists.webkit.org" target="_blank">webkit-dev@lists.webkit.org</a><br>
<a href="https://lists.webkit.org/mailman/listinfo/webkit-dev" target="_blank">https://lists.webkit.org/<u></u>mailman/listinfo/webkit-dev</a><br>
</div></div></blockquote></div><br></div></div><div class="gmail_extra"><br></div></div></div>