[webkit-dev] Can we distinguish imported tests in LayoutTests/css3 ?

[Ojan Vafai]

On Tue, Jun 12, 2012 at 9:50 AM, Darin Adler <darin at apple.com> wrote:

> On Jun 11, 2012, at 7:33 PM, Mike Lawther wrote:
>
> > Does having the 'fast' directory still serve a useful purpose?
>
> Not sure it does.
>
> The original intent was to put more extensive complete tests that were
> slow but had greater coverage outside the “fast” directory, so that you
> could run the majority of the most useful tests in a few seconds.
>
> Times to run tests have increased so much, despite the parallel test
> runner, that it’s no longer a useful distinction.
>
> A new logical organization for tests would be welcome, and it’s a big
> project where I am not sure we have a clear enough plan to start on it
> incrementally. Here are a few considerations:
>
> - Moving test files around could cause some huge churn, with some
> super-slow check-outs for everyone working on the project and many old
> patches potentially needing to be rebased. This is especially true when
> image results are involved.
>
> - Each time we change the tests we need to make sure we haven’t broken new
> tests on various platforms and that we’ve updated the skipped or test
> expectations files properly.
>

If we could get more EWS bots running the tests (currently only Chromium
Linux does), that would make this a lot easier. It would also improve
general development on WebKit. Fewer cases of people being surprised by
broken layout tests when a patch gets committed.

One more item I'd add to this list:
-Replace js-test-post.js with an onload handler in js-test-pre.js so we can
get rid of that file entirely.


> - Calling the entire directory that hosts our regression test suite
> LayoutTests seems wrong; that name was apropos when most of the tests were
> for layout issues, very early in the life of the test machinery.
>
> - Imported test suites ought to be gathered together in a single directory
> then grouped by where they were imported from by having a directory for
> each source.
>
> - We want to continue the practice of keeping copies imported test suites
> as close to the original as possible. In some cases in the past that meant
> preserving incorrect imported tests with expected “failures” that weren’t
> really failures. Often we’d put corrected copies elsewhere in the test tree
> while waiting for or working to get the imported test to fixed at the
> source.
>
> - Keeping the tests that require the HTTP server in a separate directory
> still seems like it might serve a purpose, although there seems to be an
> extra directory level in the test hierarchy there that is not helpful.
>
> - When rearranging tests that share the test shell JavaScript files we’d
> likely need to update relative paths. Might also be nice to put that test
> shell in a more logical location.
>
> - Making JavaScript-only tests runnable without WebKit’s involvement would
> be a welcome feature; great for anyone working on JavaScriptCore. That
> means we’d want to be sure to put those into their own directories. Ideally
> that would eventually be coupled by a project to move the Mozilla test
> suite currently in JavaScriptCore into the main WebKit regression tests.
>
> -- Darin
> _______________________________________________
> webkit-dev mailing list
> webkit-dev at lists.webkit.org
> http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20120612/8546bb3a/attachment.html>