[Webkit-unassigned] [Bug 118599] New: Replace MathML pixel tests by reftests

[bugzilla-daemon at webkit.org]

https://bugs.webkit.org/show_bug.cgi?id=118599

           Summary: Replace MathML pixel tests by reftests
           Product: WebKit
           Version: 528+ (Nightly build)
          Platform: All
        OS/Version: All
            Status: UNCONFIRMED
          Severity: Normal
          Priority: P2
         Component: MathML
        AssignedTo: webkit-unassigned at lists.webkit.org
        ReportedBy: fred.wang at free.fr
                CC: cfleizach at apple.com, mrobinson at webkit.org


As indicated in

http://trac.webkit.org/wiki/Writing%20Layout%20Tests%20for%20DumpRenderTree

and as I explained in bug 118053 comment 7, using pixel tests is too fragile especially for MathML where the rendering is not defined precisely (as opposed to e.g. SVG or CSS). Basically, if we start to change anything like fixing bug 115787 or the patch Martin Robinson is working on for stretchy operators, that will just break all the MathML tests. Hence one has to verify the tests by eye and regenerate the references again which is error-prone and a bit in contradiction with the goal of automated regression tests. Moreover it is not always clear what's the tests are verifying (position of scripts? sizes? spacing? font style? everything at the same time?) and currently references seem to only available on GTK, EFL and Mac (so I guess the other platforms just load the pages without verifying against a references?)

I suggest to replace them by reftests. We won't be able to test the subjective fact that a formula is "good-looking" and the test pages may be somewhat less "natural". But at least they become easy to verify (the two pages should just match / not match), they allow to restrict the unit test to only the minimal feature you want to check and ignore unrelated stuff, they won't break so easily and will be more cross-compatible (in particular no need to generate the references for each platform).

-- 
Configure bugmail: https://bugs.webkit.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.