[webkit-dev] Supporting w3c ref tests and changing our convention

[Ryosuke Niwa]

On Fri, Nov 4, 2011 at 1:47 PM, Dirk Pranke <dpranke at chromium.org> wrote:
>
> It's unclear how much of a perf impact there would be but that's
> easy enough to determine - I would expect it to be minimal compared to
> the time of actually rendering a page.
>

Since I expect w3c to end up having hundreds of thousands of tests, I see
any performance implication to be a serious threat.

 That said, supporting a manifest file is clearly fairly easy to do in
> NRWT, and presumably easy to add as a build step (e.g., make DRT
> depend on it) and may have the added bonus of allowing us to run
> various mozilla reference test suites that wouldn't be using the
> links, so I'm fine with that.
>

In fact, we already have a patch to support it on :
https://bugs.webkit.org/show_bug.cgi?id=66837
and https://bugs.webkit.org/show_bug.cgi?id=71567

I think we should voice a concern w/ the W3C that their tests must
> follow consistent naming styles (for maintainability); we shouldn't
> view the links and the  manifest step as a carte blanche to name tests
> and results whatever they want.
>

Yes, I have. More comments on http://www.w3.org/html/wg/wiki/Testing or
http://wiki.csswg.org/test will be helpful.

Separately, if we are throwing around numbers in the range of >100K
> for tests to run, we should consider when we actually want to run them
> - i.e., what will the cycle time be if we run them on every change,
> etc.? But that can be dealt with when we get there.
>

We need separate bots in the long term for sure.

- Ryosuke
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20111104/85f71a34/attachment.html>