[webkit-dev] Layout Tests and the W3C

[Rebecca Hauck]

From: Maciej Stachowiak <mjs at apple.com<mailto:mjs at apple.com>>
Date: Mon, 15 Apr 2013 18:29:43 -0700
To: Dirk Pranke <dpranke at chromium.org<mailto:dpranke at chromium.org>>
Cc: WebKit-Dev <webkit-dev at lists.webkit.org<mailto:webkit-dev at lists.webkit.org>>
Subject: Re: [webkit-dev] Layout Tests and the W3C


On Apr 15, 2013, at 5:31 PM, Dirk Pranke <dpranke at chromium.org<mailto:dpranke at chromium.org>> wrote:

Hi all,

Those of you who are subscribed to blink-dev@ will see that I just sent out a note entitled "Blink, Testing and the W3C", describing stuff I'm working on to get the W3C tests running regularly as part of the layout tests. The W3C has gotten to the point where they have many thousands of ref tests, a lot of which are newer versions of old test suites that we imported once upon a time (meaning that we can replace a lot of pixel tests with ref tests).

That work leverages a bunch of work Rebecca Hauck has been working on that actually started pre-Blink-fork, and in fact currently only exists as a patch posted to bugs.webkit.org<http://bugs.webkit.org/> ( https://bugs.webkit.org/show_bug.cgi?id=111513 ).

I would like to get something resembling the same process working for both WebKit and Blink, since we use basically the same tools. There will probably be some differences, of course, e.g., in how we mirror repos and track issues, but I would hope we'd all agree the more tests we can share (both between WebKit and Blink and with other vendors like FF and IE) the happier everyone will be, right?

We haven't really worked out the processes for how this'll work yet; Rebecca and I were starting to work on it when the fork happened.

Assuming there is still interest, I'll start sketching out how things might work for WebKit and report back. Also, if we actually start to make progress on this, I'd be happy to talk about it at the WebKit committer's meeting if there's interest there as well.

I think it would be great to run W3C reftests as part of WebKit's layout tests. Extra awesome would be to have the setup to import new versions of test suites and/or new test suites regularly. I would welcome such work and I expect others would as well.

Yes! I think it should go in a few phases – 1) work out the mechanics of the import, 2) identify & import the first suite of tests manually (probably the most stable suite – CSS2.1 for example), 3) come up with a strategy for refreshing and importing new tests (could be partly considered in the mechanics of the import script), 4) turn on the script to automatically import/refresh once all the wrinkles from 1-3 are sorted out.

I also think this would make a great contributor's meeting topic, either as a regular session or a hackathon.

Just added this to the wiki (thanks to Alan Stearns – it wasn't cooperating with me when I tried).  I'm happy to do an overview of the issues, what little bit of work I've started on this, and open it up to discussion on how to proceed.  One thing I would hope for is if we are able to request time slots at all – if we can have this in an early-ish slot on Thursday, that'd be great. I won't be able to attend on Friday.

Cheers,
-Rebecca



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20130418/1a044e97/attachment.html>