[Webkit-unassigned] [Bug 195233] [ Mac WK2 iOS Debug ] REGRESSION(r233667) Layout Test imported/w3c/web-platform-tests/web-animations/interfaces/DocumentTimeline/constructor.html is a flaky failure

[bugzilla-daemon at webkit.org]

https://bugs.webkit.org/show_bug.cgi?id=195233

--- Comment #2 from Shawn Roberts <sroberts at apple.com> ---
Adding additional repro steps:

Test was flaky through its entire available history on dashboard and searching through Trac I could not find any recent changes. Started Bisection point at 228049 which was the oldest revision I could spade.

Once I started getting 100% failures I switched test parameters to --exit-after-n-failures=25 to limit iterations run.

For Debug failures it failed in r233667 and 233636 was the only version I could spade previous to that.

Once I found the revision it was starting to fail on, Looking on Trac for r233667 it appears test expectations were modified.

Release WK2:

242285 flaky failure
242279 flaky failure
242112 flaky failure
237300 flaky failure
235000 flaky failure
234133 flaky failure
233702 flaky failure
233676 flaky failure
233671 flaky failure
233668 flaky failure
233667 4997 tests ran as expected, 3 didn't failed unexpectedly (text diff)

233666 0 tests ran as expected, 25 didn't
233665 0 tests ran as expected, 25 didn't
233649 0 tests ran as expected, 25 didn't
233595 0 tests ran as expected, 25 didn't
233483 0 tests ran as expected, 25 didn't
233263 0 tests ran as expected, 25 didn't
231525 0 tests ran as expected, 25 didn't
228049 0 tests ran as expected, 25 didn't : Unexpected text-only failures (1) [ Failure ]


Debug WK2:

233678 733 tests ran as expected, 5 didn't (4262 didn't run) failed unexpectedly (text diff)
233636 0 tests ran as expected, 25 didn't Unexpected text-only failures (1) [ Failure ]


Since initial testing found that iOS Simulator Release WK2, and Mac Debug WK1 are now flaky failures.

iOS Simulator:

242640 722 tests ran as expected, 5 didn't (4273 didn't run) Unexpected text-only failures (1)
233667 4997 tests ran as expected, 3 didn't failed unexpectedly (text diff)
233666 0 tests ran as expected, 25 didn't Unexpected text-only failures (1) [ Failure ]


Mac Debug WK1:

233678 395 tests ran as expected, 5 didn't (4600 didn't run): failed unexpectedly (text diff)
233636 0 tests ran as expected, 25 didn't (4975 didn't run): failed unexpectedly (text diff)

-- 
You are receiving this mail because:
You are the assignee for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-unassigned/attachments/20190308/9f1816d7/attachment.html>