Hi Darin & Dave & Krzystof & Jean-Claude, During the Qt/KDE platform porting, I've basically run into the same problem, though I also dislike the idea of a fuzz factor.
1) Reorganize the tests so that tests that should work across all platforms are separate from ones that might need different results per platform. (Perhaps we could rationalize the organization and naming of tests in some other ways as well.)
I still think this is the best solution, having -expected-qt.txt, -expected-gdk.txt, -expected-osx.txt, -expected-win.txt, to be able to find out regressions pixel-wise, so that 1px does make a difference.
Also, I think the DumpRenderTree tool is going to need a rename eventually. That was a good name for the original tool back when I first wrote it and it was only about render trees, but now it's more like "test engine" or something along those lines. Similarly, I think the LayoutTests directory might need a rename for similar reasons.
Something like RegressionTester may be better, indeed. My suggestions is to create subdirectories for the specific platforms, where the -expected-<PLATFORM>.txt files & image diffs etc. will go. Maybe it would be even wiser to adjust the "Get-WebKit-From-SVN" scripts to just svn co the specific subdirectory needed. For example it doesn't make sense for you guys to get our -expected-gdk/qt.txt files. And we also don't need for instance the -win.txt files. Please let me know what you think. Niko