On Wed, Jul 6, 2011 at 9:00 AM, Adam Roben <aroben@apple.com> wrote:
On Jul 6, 2011, at 11:58 AM, Adam Barth wrote:
I'm not sure we've quite figured that out yet. NRWT supports both Skipped lists and test_expectations.txt, which is a more expressive (but also more complex) version of Skipped lists. IMHO, we should wait for the dust to settle on the transition before changing our practices.
OK. Then I have another question:
What should I do to make the Leopard and SnowLeopard bots green, now that they have switched to NRWT?
Looking at Leopard, there's one jscore-test issue and two run-webkit-test issues: http://build.webkit.org/results/Leopard%20Intel%20Release%20(Tests)/r90460%2... One of the run-webkit-test issues seems to relate to -0. It's possible that's a test harness issue, but it seems more likely that it's a regression in JavaScriptCore. The other is some SVG foreign object issue, which I have less insight into. We can either fix the regressions, update the expected.txt files, or add the test to the Skipped list, as before. On SnowLeopard, there are also two failing tests, but I believe these tests are related to the NRWT transition: http://build.webkit.org/results/SnowLeopard%20Intel%20Release%20(Tests)/r904... The third-party-cookie-relaxing.html test probably needs to be changed because it depends on the persistent state of the cookie jar. Either Eric or I will dig into the test to figure out how to make it more robust. The storagetracker tests also have similar issues. They appear to be flaky with NRWT, which is <https://bugs.webkit.org/show_bug.cgi?id=57799>. These tests are also flaky under ORWT (but less so): https://bugs.webkit.org/show_bug.cgi?id=58835 https://bugs.webkit.org/show_bug.cgi?id=58836 We need to fix both issues, but they didn't seem like issues that should block the change to NRWT. That's a somewhat round-about way of not answering your question. :) Adam