[webkit-dev] Supporting w3c ref tests and changing our convention

[Dirk Pranke]

On Fri, Nov 4, 2011 at 1:32 PM, Hayato Ito <hayato at chromium.org> wrote:
> Could you clarify why we have to need to modify DRT if we have Link Element
> approach?
> I guess one of the reasons is the performance. It might be better that we
> use DRT to parse and extract reference links from HTML since parsing HTML
> using Python might take unacceptable time and might be inaccurate.
> Is there any other reasons we should modify DRT to support Link Element
> approach?

I agree with Ito-san that I don't think you have to modify each DRT. I
see no reason to think that we couldn't parse the files reliably in
NRWT. It's unclear how much of a perf impact there would be but that's
easy enough to determine - I would expect it to be minimal compared to
the time of actually rendering a page.

That said, supporting a manifest file is clearly fairly easy to do in
NRWT, and presumably easy to add as a build step (e.g., make DRT
depend on it) and may have the added bonus of allowing us to run
various mozilla reference test suites that wouldn't be using the
links, so I'm fine with that.

I think we should voice a concern w/ the W3C that their tests must
follow consistent naming styles (for maintainability); we shouldn't
view the links and the  manifest step as a carte blanche to name tests
and results whatever they want.

Separately, if we are throwing around numbers in the range of >100K
for tests to run, we should consider when we actually want to run them
- i.e., what will the cycle time be if we run them on every change,
etc.? But that can be dealt with when we get there.

-- Dirk