[webkit-dev] EWS doesn't lie!

[Tim Horton]

On Feb 10, 2014, at 12:51 AM, youenn fablet <youennf at gmail.com> wrote:

> Is it by design that only mac bots run regression tests?

Certainly not.

> Technical issue?

I think so; look at http://build.webkit.org/dashboard/; the Mac port is the only one that’s even close to green enough to make running the tests on EWS worthwhile (i.e. insufficiently noisy that new failures are possible to distinguish).

> Lack of resources?

Could be this too, but I can’t speak for the other ports. It takes a good number of machines to keep up (and Mac starts falling behind as soon as someone introduces a flaky test because of EWS’s slightly odd machinery).

> 
> 2014-01-30 9:05 GMT+01:00 Alexey Proskuryakov <ap at webkit.org>:
> Hi WebKit hackers,
> 
> It sometimes happens that people land patches despite EWS detecting layout test regressions, especially when these seem too unlikely to believe.
> 
> In my experience, EWS has been very stable recently, and if tester bubbles are red, it almost certainly means that the patch is faulty. Even if they don't turn red, but remain yellow for a long time, there is a good chance that the patch introduces flaky failures, so EWS can't make up its mind about exactly which tests regressed.
> 
> If you click on a yellow bubble, that takes you to a page with additional details, where you can see which tests are failing. I'd like to look into improving how this information is presented at some point in the future, yet even now, it shouldn't be too time consuming to check what's going on.
> 
> For reference, we currently have mac and mac-wk2 EWS bots running regression tests in release mode. Other bots only verify that the patch builds, and don't run tests. No bots use debug mode as far as I know, so debug-only build failures and assertions will not be detected. Please run tests locally to catch as many of those as possible.
> 
> - WBR, Alexey Proskuryakov
> 
> _______________________________________________
> webkit-dev mailing list
> webkit-dev at lists.webkit.org
> https://lists.webkit.org/mailman/listinfo/webkit-dev
> 
> _______________________________________________
> webkit-dev mailing list
> webkit-dev at lists.webkit.org
> https://lists.webkit.org/mailman/listinfo/webkit-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.webkit.org/pipermail/webkit-dev/attachments/20140210/99471e1c/attachment.html>