[webkit-qt] Tests in Skipped List that Pass with Pixel Differences
lists at roberthogan.net
Sun Apr 18 14:30:12 PDT 2010
There is a definite class of tests in the Qt skipped list that fail only
because of unimportant pixel differences.
I propose to open a master bug for these the purpose of which will be to
act as a staging pad for removing them from the Skipped list.
I would see it working like this:
- Identify a group of tests that only fail due to pixel differences and for
which it is clear pixel differences are not material
- Add a patch against LayoutTests/platform/qt/Skipped with the tests
- Along with the patch add a comment to bugzilla which will act as a a
'manifest' listing each test to be removed and why it is safe to use
platform specific results. It might even be best to keep this manifest as a
file in the platform/qt tree.
- Send a note to the buildbot team requesting them to try the patch.
- Wait for a member of the buildbot team to run the patch against the
buildbot. The buildbot team can then post an updated patch with the changes
to the Skipped list and the platform specific results. They mark it for
- A reviewer comes along, satisifies themselves that platform specific
results are justified for each of the skipped tests and approves.
Keeping all this under a single bugzilla entry might get cumbersome after a
while, when that happens a new one can be created to replace it and linked
from the old one. I think it's important to have one go-to place for this
Thoughts? Do the buildbot team this is something that could work?
More information about the webkit-qt