[Webkit-unassigned] [Bug 206579] [CSS Backgrounds] ASSERTION FAILED: !image->size().isEmpty() on css/css-backgrounds/background-size/vector/zero*ratio-auto-5px.html

[bugzilla-daemon at webkit.org]

https://bugs.webkit.org/show_bug.cgi?id=206579

Carlos Alberto Lopez Perez <clopez at igalia.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |dpino at igalia.com

--- Comment #4 from Carlos Alberto Lopez Perez <clopez at igalia.com> ---
(In reply to Alexey Proskuryakov from comment #3)
> > Well, its also costly to mark tests timing-out as Timeout, and not simply skip them.. but we don't do that, do we?
> 
> We do. Please do, too.
> 

Well, on the GTK and WPE ports (so far) we don't do that. We mark tests as Crash or Timeout if they end always-crashing or always-timing-out.

Maybe we should start doing thinks like you do, it makes little sense that we don't follow the same rules.

Its there a document or wiki page where its documented the policy you follow regarding this?

> A lot of the work people are doing importing WPT tests is overall
> counter-productive, because it makes WebKit regression tests slower and less
> reliable. 

It may be counter-productive from the point of view of getting the layout test step to run as fast as possible and without flaky results since WPT tests are a lot of extra tests to run, but its not counter-productive from the point of view of getting more test coverage.

I have just checked it, and currently of 53340 layout tests, 17314 are WPT tests.. that's around 1/3 of the layout tests.
Maybe we should re-think how we run WPT tests and perhaps run them in a new different step from the layout-test one.

> It is also a huge cost that someone else then has to find and
> triage flaky tests.
> 

I agree, flaky tests are a time sink problem.

> > Also I don't see why it should impact stability or prevent from getting crashing reports from other tests?
> 
> Generating a crash log is a CPU and disk access intensive operation. Since
> we already over-commit on the number of parallel processes, this is not
> trivial. 

Good point, I see how leaving tests expectations as Crash can contribute to cause flaky tests (specially unexpected timeouts).

It still seems not ideal to me to skip crashing tests, but having unexpected flaky tests seems a bigger problem.

So, I will change the expectation for this tests to Skip.

-- 
You are receiving this mail because:
You are the assignee for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-unassigned/attachments/20200227/8acafb57/attachment.htm>