[Webkit-unassigned] [Bug 161003] Many of the expected results from imported/w3c are wrong (contain FAIL strings)

[bugzilla-daemon at webkit.org]

https://bugs.webkit.org/show_bug.cgi?id=161003

--- Comment #6 from Alexey Proskuryakov <ap at webkit.org> ---
> Surely we don't want to ever have FAIL in our expected results; 

We actually do, for the reasons outlined in the comments above. Marking a test as Failure is quite detrimental.

Sometimes when dealing with old official test suites (like DOM2), a failure result on a subtest is the correct one, due to a change in newer spec version, so we don't even want to "fix" it. Yet we do want to run the test, to make sure that we don't start crashing, or that we don't accidentally revert to old behavior, and of course to verify other subtests in the test.

> that makes it extremely difficult to track which tests are broken....

The correct way to track bugs is in Bugzilla. I do not think that marking every partially failing test as Failure would work for tracking efforts - there is no priority, no discussion, no CC list etc.

-- 
You are receiving this mail because:
You are the assignee for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.webkit.org/pipermail/webkit-unassigned/attachments/20160820/29965718/attachment.html>