[webkit-dev] Can we distinguish imported tests in LayoutTests/css3 ?
darin at apple.com
Tue Jun 12 09:50:08 PDT 2012
On Jun 11, 2012, at 7:33 PM, Mike Lawther wrote:
> Does having the 'fast' directory still serve a useful purpose?
Not sure it does.
The original intent was to put more extensive complete tests that were slow but had greater coverage outside the “fast” directory, so that you could run the majority of the most useful tests in a few seconds.
Times to run tests have increased so much, despite the parallel test runner, that it’s no longer a useful distinction.
A new logical organization for tests would be welcome, and it’s a big project where I am not sure we have a clear enough plan to start on it incrementally. Here are a few considerations:
- Moving test files around could cause some huge churn, with some super-slow check-outs for everyone working on the project and many old patches potentially needing to be rebased. This is especially true when image results are involved.
- Each time we change the tests we need to make sure we haven’t broken new tests on various platforms and that we’ve updated the skipped or test expectations files properly.
- Calling the entire directory that hosts our regression test suite LayoutTests seems wrong; that name was apropos when most of the tests were for layout issues, very early in the life of the test machinery.
- Imported test suites ought to be gathered together in a single directory then grouped by where they were imported from by having a directory for each source.
- We want to continue the practice of keeping copies imported test suites as close to the original as possible. In some cases in the past that meant preserving incorrect imported tests with expected “failures” that weren’t really failures. Often we’d put corrected copies elsewhere in the test tree while waiting for or working to get the imported test to fixed at the source.
- Keeping the tests that require the HTTP server in a separate directory still seems like it might serve a purpose, although there seems to be an extra directory level in the test hierarchy there that is not helpful.
More information about the webkit-dev