[webkit-dev] Windows Bots Are Green!

[Brent Fulgham]

Hi Everyone,

Over the past few weeks I’ve spent a considerable amount of time reviewing and correcting a number of problems with the Windows testing infrastructure.

We were skipping thousands of tests, including Accessibility, http tests, and large sections of forms, css, and svg tests. Some tests were skipped because features were supposedly incomplete, but in the years since the skip entry was added to the file, the work had been completed and feature worked perfectly well. Some tests were skipped simply because feature flags had never been enabled on Windows.

Happily, all of this is in the past. Now that this work is complete, we have much more test coverage of many areas of WebKit that had been previously ignored on this platform.

1. We are now using native Windows Apache to run our HTTP tests, and have been doing so for the past couple of weeks. This seems to be quite stable, and is allowing us to make sure SSL and other important features work properly on Windows.
2. Accessibility tests are back up and running.
3. Thanks to Ossy, we now run all JSC stress tests on Windows.
4. Nearly all other tests are now running, with a few notable exceptions below.
5. I have switched to running Windows tests in a single process, because I found that running in parallel introduced inconsistent results. (See <https://bugs.webkit.org/show_bug.cgi?id=140914 <https://bugs.webkit.org/show_bug.cgi?id=140914>>).

So now that I’ve completed this task, PLEASE help keep the Windows bots green! :-)

If you are at all interested in the various Windows ports, and are looking for something to do, there are a number of bugs I’ve filed that could use some attention:

1. For some reason, running layout tests in parallel introduces inconsistent behavior and spurious failures. There must be some kind of cross-talk between the different shards. <https://bugs.webkit.org/show_bug.cgi?id=140914 <https://bugs.webkit.org/show_bug.cgi?id=140914>>.
2. There are a number of debug assertions firing that cause Debug test runs to exit early. (see <https://bugs.webkit.org/show_bug.cgi?id=140517 <https://bugs.webkit.org/show_bug.cgi?id=140517>>, 
3. Something weird is going on with the page cache. <https://bugs.webkit.org/show_bug.cgi?id=140190, https://bugs.webkit.org/show_bug.cgi?id=140871 <https://bugs.webkit.org/show_bug.cgi?id=140871>>
4. Accessibility tests have a number of problems:
(a) Several Accessibility tests fail in debug mode because they assert that they are accessing text iterators before layout is complete. The comment with this assertion indicates that this can cause crashes and instability. <https://bugs.webkit.org/show_bug.cgi?id=140867 <https://bugs.webkit.org/show_bug.cgi?id=140867>>
(b) Some accessibility tests are very flaky with regard to digging down into the DOM. Test will pass one run, fail the next. <https://bugs.webkit.org/show_bug.cgi?id=140798 <https://bugs.webkit.org/show_bug.cgi?id=140798>>

Feel free to contact me any time if you want to tackle any of these problems!

Otherwise, please just help keep the Windows bots green. I don’t want to have to go through all of this again! :-)

Best regards,

-Brent


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.webkit.org/pipermail/webkit-dev/attachments/20150220/63144f8a/attachment.html>