[Webkit-unassigned] [Bug 153707] [GTK] Some W3C DOM tests are failing

[bugzilla-daemon at webkit.org]

https://bugs.webkit.org/show_bug.cgi?id=153707

--- Comment #8 from youenn fablet <youennf at gmail.com> ---
In theory, I would prefer to keep the most compliant expectation as the baseline.
In practice, whenever w3c tests are refreshed, I usually use Mac-wk2 as the baseline and other bot results are put as specific expectations.

I would go with 2 and see whether things are going well when refreshing the tests.
And reverting to 3 or improving webkitpy if something does not go well after refreshing.
I usually dislike option 1 since we cannot catch any regression anymore.


(In reply to Manuel Rego Casasnovas from comment #7)
> (In reply to Ms2ger from comment #6)
> > (In reply to Michael Catanzaro from comment #5)
> > > The W3C tests do not expect touch events to be enabled, seriously? Then we
> > > should move these tests to the expected failures section of our
> > > TestExpectations file, mark them to be skipped, add a comment on top, remove
> > > the bug annotation, and close this bug.
> > 
> > The upstream -expected file marks the touch event subtests as failing, while
> > they pass in gtk.
> 
> -expected.txt files for WPT imported tests usually contain PASS and FAIL
> messages,
> so they can be used as regression tests to detect if any patch
> modifies their behavior.
> 
> Regarding these particular tests it seems that they pass in GTK+
> but fail in Mac.
> 
> So I guess we've differnt options here (not sure what's the preferred one,
> so adding @youenn in CC for feedback):
> 1) Update the generic -expected.txt file to use the GTK+ results (PASS).
>    Then mark the tests as failure in Mac TestExpectations file.
> 2) Update the generic -expected.txt file to use the GTK+ results (PASS).
>    And add a specific Mac -expected.txt file with the results in Mac.
> 3) Keep current -expected file as is. And then add a specific
>    GTK+ -expected.txt file, with the GTK+ results.

-- 
You are receiving this mail because:
You are the assignee for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-unassigned/attachments/20170801/af1d7c04/attachment.html>