[Webkit-unassigned] [Bug 66295] [Meta] Support w3c reftests

[bugzilla-daemon at webkit.org]

https://bugs.webkit.org/show_bug.cgi?id=66295





--- Comment #20 from Hayato Ito <hayato at chromium.org>  2011-11-03 10:48:19 PST ---
Okay. Although I prefer pre-parsing of reference links before passing it to DRT, the following is yet another crazy idea.

1. Collect every 'HTML' files without parsing. HTML file might be one of the followings:
  A. HTML file used by *traditional* Layout tests
  B. HTML file used by non-w3c reftests (e.g. a.html)
  C. Expected html file by non-w3c reftests (e.g. a-expected.html)
  D. Expected mismatch html file by non-w3c reftests (e.g. a-expected-mismatch.html)
  E. w3c reftests (c.html)
  F. w3c reference files (e.g. c-ref.html, c-notref.html)
  G. misplaced html files. :)

  In w3c reftests, one html file might be used as both E and F.

In this phase, we don't know how many html files actually represens for 'test'.
Some of them are used only by 'references'.

2. Get rendering results of all HTML files and also get reference links for each tests from DRT.

3. Given rendering results and reference links information, compare and evaluate the rendering results.

The problem is that we cannot store all rendering results in memory. We proceed 2 & 3 in some incremental ways....


(In reply to comment #15)
> I'd rather we restructure the code so that reftests get run the same way as non-reftests. The only difference is *after* we've run the test we need to evaluate whether it's a reftest.
> 
> So basically, we would still I identify reftests via the -expected.html files during the initial crawl of the LayoutTests directory, but then we'd also identify them when we run the test. 
> 
> Ideally, I don't think we need to specially parse the files. We can add DRT infrastructure to print out the appropriate text when it encounters a reftest link element. Then run-webkit-tests can read the printed out line to know which file is the reference. This might be too much work though.
> 
> If that's too complicated, I'd also be OK with just running a regexp over the file before we run it, but I worry that would increase the time it takes to run the test. The part I'm worried about is the extra disk access, but maybe it's OK since the file will then be in the disk cache for when DRT needs to read it.
> 
> In short, I'd like to see someone try the less complicated regexp approach right before we run the test and (locally!) see if it has a significant cost in the time it takes to run the full test suite. If I understand correctly, that wouldn't actually require restructuring the test runner code much since we'd still know before running the test that it's a reftest.

-- 
Configure bugmail: https://bugs.webkit.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.