[Webkit-unassigned] [Bug 161242] [GTK][Threaded Compositor] Several flaky tests

[bugzilla-daemon at webkit.org]

https://bugs.webkit.org/show_bug.cgi?id=161242

--- Comment #20 from Carlos Garcia Campos <cgarcia at igalia.com> ---
(In reply to comment #19)
> (In reply to comment #18)
> > (In reply to comment #17)
> > > (In reply to comment #16)
> > > > You are assuming again that there's a single issue.
> > > 
> > > No, clearly there were many different issues, as you've fixed three
> > > different issues so far....
> > > 
> > > > I think the situation now
> > > > is mostly the same to what we had before switching to the threaded
> > > > compositor.
> > > 
> > > Yes, except for the fact that sometimes a bunch of tests pass when they
> > > usually fail. In build #18023 we have 84 unexpected passes; that never
> > > happened before. What's interesting is these tests either always pass or
> > > always fail in a particular run of run-webkit-tests; they are clearly flaky,
> > > but they don't contribute to the flakiness count.
> > 
> > We should probably mark them as pass, to see what's wrong when they fail
> 
> That don't makes much sense to me.
> 
> You can see the failure diff even if they are expected failures. You just
> have to navigate under
> https://build.webkit.org/results/GTK%20Linux%2064-
> bit%20Release%20%28Tests%29/ to find them.
> 
> For example: 
> 
> 1) Check the unexpected passes at end of build #18023
> https://build.webkit.org/builders/GTK%20Linux%2064-
> bit%20Release%20%28Tests%29/builds/18023/steps/layout-test/logs/stdio and
> pick 1
> 
> 2) I pick: imported/mozilla/svg/blend-difference-stacking.html
> 
> 3) Check that on the next build that test failed. Log of the next build:
> https://build.webkit.org/builders/GTK%20Linux%2064-
> bit%20Release%20%28Tests%29/builds/18024/steps/layout-test/logs/stdio <---
> yes it failed
> 
> 4) Go to
> https://build.webkit.org/results/GTK%20Linux%2064-
> bit%20Release%20%28Tests%29/ and find the results for build #18204 ->
> https://build.webkit.org/results/GTK%20Linux%2064-
> bit%20Release%20%28Tests%29/r205335%20%2818024%29/
> 
> 5) Now under that folder navigate to imported/mozilla/svg/ and look for the
> blend-difference-stacking diff
> 
> 6) Here you have it:
> https://build.webkit.org/results/GTK%20Linux%2064-
> bit%20Release%20%28Tests%29/r205335%20%2818024%29/imported/mozilla/svg/blend-
> difference-stacking-diffs.html

hmm, well, not very convenient, but still. I have theory, though. When they unexpectedly pass, they don't actually pass, we just fail to render both the actual and expected files in the same way (for example a fully white image in both cases). That's a problem of the reftests. So, more interesting to see what's failing, which can be also reproduced locally more easily, would be to see what we render when they pass.

-- 
You are receiving this mail because:
You are the assignee for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.webkit.org/pipermail/webkit-unassigned/attachments/20160903/8ec6e2f4/attachment.html>