[webkit-dev] Supporting w3c ref tests and changing our convention

Ryosuke Niwa rniwa at webkit.org
Fri Nov 4 12:34:42 PDT 2011


There was a discussion about supporting W3C ref tests at TPAC yesterday.
However, there appears to be some disagreement in how we support them.


CSS WG ref tests contain link elements that specify reference files. e.g.

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
 <head <http://december.com/html/4/element/head.html>>
  <title <http://december.com/html/4/element/title.html>>CSS Reftest
Reference</title <http://december.com/html/4/element/title.html>>
  <link <http://december.com/html/4/element/link.html> rel="author"
title="NAME_OF_AUTHOR" href="mailto:EMAIL OR http://CONTACT_PAGE"/>
<link <http://december.com/html/4/element/link.html> rel="match"
href="green-box-ref.xht" />
  <style <http://december.com/html/4/element/style.html>
type="text/css"><![CDATA[   CSS FOR REFERENCE  ]]></style
 </head <http://december.com/html/4/element/head.html>>
 <body <http://december.com/html/4/element/body.html>>
 </body <http://december.com/html/4/element/body.html>></html

The highlighted line specifies that the rendered result of this page should
be compared with that of green-box-ref.xht. This is very different from our
current method to use filenames such as -match.html, -mismatch.html, etc..
to identify reference files.

In addition, W3C test suites have a build step, which generates
Mozilla-like test manifest files. This file will contain entries that
identifies tests and corresponding reference files. e.g.

== floats-wrap-bfc-outside-001.xht floats-wrap-bfc-outside-001-ref.xht
== floats-wrap-top-below-bfc-001l.xht floats-wrap-top-below-001l-ref.xht
!= floats-wrap-top-below-bfc-001l.xht floats-wrap-top-below-001l-notref.xht
== floats-wrap-top-below-bfc-001r.xht floats-wrap-top-below-001r-ref.xht
!= floats-wrap-top-below-bfc-001r.xht floats-wrap-top-below-001r-notref.xht

All browser vendors who attended the meeting preferred this format.
Microsoft representatives didn't attend the meeting.

*Link element approach*

   - Can reuse same ref. file for multiple tests
   - Can have multiple ref. files for single test
   - Information is self-contained in the test file
   - We may get away with test suite build step

(It turns out that we can't convert W3C ref tests to use WebKit conventions
due to the first two points.)


   - Requires us modifying each port's DRT to support this format
   - Adding link elements itself may affect tests (all W3C tests are
   required to have link elements at the moment)
   - Hard to understand relationship between files. e.g. if we want to
   figure out which tests use ref.html, we must look at all test files
   - Other browser vendors (Firefox and Opera) prefer manifest file approach

*Manifest file approach*

   - Easy to parse, and do not require us modifiying each port's DRT to
   - Easy to tell relationships between files at glance
   - Do not affect tests since manifest files are external to tests
   - Preferred approach of Firefox and Opera


   - Have to maintain information in two places, tests and manifests (W3C
   test suites auto-generates manifest files in their build step)
   - Require build step (new W3C tests only have link elements but not

I strongly prefer WebKit use the manifest file approach and deprecate our
current file-name convention because it won't burden each port, and it's
the preferred method of other browser vendors.  Any opinions?
Ryosuke Niwa
Software Engineer
Google Inc.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20111104/edaf674c/attachment.html>

More information about the webkit-dev mailing list