[webkit-dev] New script to import W3C CSS tests

[Rebecca Hauck]

Hi Dirk  & Everyone,

Dirk, your question about this in IRC the other day was timely indeed.

I've recently written a batch of tests for CSS Regions that I intend to submit to both the W3C and Webkit (and have planned more to come). Rather than push two slightly different versions of the same tests to each repo, and to address how these tests are to be maintained over time, I took a stab automating the import and just submitted a patch [1].

I know this conversation was started a while ago as I attended the WK Contributor's meeting last spring where my colleagues Jacob Goldstein and Alan Stearns led the discussion. I was sort of new onto the scene then and was pretty much a lurker in the meeting, so you probably don't remember me.  I recall that much of the debate was about handling failing tests, approved vs. submitted, duplicates, etc.  Since all I've done is automate the mechanics of the import itself, I have not solved a lot of those issues, but the conversion of tests certainly had its own set of things to consider. I figured I would put this one piece in place, get feedback on it, and hopefully resurrect the discussion on fitting this into the bigger picture.  A great goal would be to have this script run on an automated schedule and to extend it to deal with test results, expected failures, etc.

All of the dirty details are in the ChangeLog of the patch and in the script headers, but I basically tried to cover #8 from last spring's meeting notes [2]. I think the script covers all of those bullets, with that last bullet as "sort of."  I also got a little inspiration from the import-w3c-performance-wg-tests that already exists. I followed a few of their steps, but had to add a few layers to handle the added complexity of the CSS test suites.

Dirk, when you have a chance, take a look. Review it with gusto as you say. And anyone else who's interested, your feedback is welcome.  I fairly certain this will go a few rounds before it's settled. :)

Cheers,
-Rebecca


[1] https://bugs.webkit.org/show_bug.cgi?id=111513
[2] http://trac.webkit.org/wiki/ImportingThirdPartyTests

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20130307/0eebd48a/attachment.html>