Thanks for the explanation, Matt! For those who haven't been following the evolution of run-webkit- tests as closely as Matt and I have, the point of the changes to run- webkit-tests was to allow the separation of cross-platform and platform-specific tests. Currently there are a lot of tests sitting under LayoutTests that are Mac-only (such as fast/AppleScript), and there are *loads* of test results that are Mac/CG-only due to font metrics (basically all the render tree dumps). This has made things difficult as other platforms have tried to get the regression tests running, so the hope is that we can now start separating the platform- specific tests from the truly cross-platform ones, and ditto for the expected results. A full description of the changes to run-webkit-tests can be found at <http://trac.webkit.org/projects/webkit/wiki/RunWebKitTestsDesign>. On Aug 14, 2007, at 9:52 PM, Matt Lilek wrote:
Pixel and render-tree test results will now be generated in the test's most common corresponding platform/ directory. So creating a new test in fast/backgrounds on Leopard will have the results generated in platform/mac/fast/backgrounds. If that test's results are specific to Leopard, they will manually have to be moved to the mac-leopard directory and a different set of results created on Tiger (or the test could just be made Leopard-only).
The basic question you should ask yourself when deciding where to place a new test is "Does this test apply to all platforms, or just mine?" Tests meant to highlight a bug that occurred on a specific platform can still be made cross-platform tests if the expected behavior is valid on all platforms. And it's OK for a cross-platform test to have platform-specific results -- as I said above, render tree dumps are currently not easily shared between platforms (though this is definitely something we'd like to improve!). -Adam