[webkit-dev] Switching to new-run-webkit-tests

[Eric Seidel]

Update:

Leopard Release, Gtk and Qt have been successfully transitioned.

The Leopard Debug bot has a stray httpd process (without corresponding
/tmp/WebKit/httpd.pid) and will require manual intervention:
http://build.webkit.org/builders/Leopard%20Intel%20Debug%20%28Tests%29/builds/32086/steps/layout-test/logs/stdio
If someone at Apple could give it a kick, I'd be most grateful.

I'll work on the WebKit2 ports more tomorrow.  I'll need some help
with Windows eventually.

-eric

p.s.  As previously noted, run-webkit-tests is calling
new-run-webkit-tests with --child-processes=1.  So the bots are
running in very very slow mode (about as fast as ORWT was).  We'll
turn on parallel testing with new-run-webkit-tests once we've
transitioned all ports.


On Tue, Jul 5, 2011 at 10:53 PM, Eric Seidel <eric at webkit.org> wrote:
> Update:
>
> Snow Leopard - Successful transition.
> Leopard - Had to roll-back due to a bug in webkitdirs.pm which errors
> in both ORWT and NRWT, but causes NRWT to fail hard.
> https://bugs.webkit.org/show_bug.cgi?id=63973
> Gtk - Bot seems hard-hung (unclear if NRWT related).  Waiting for
> assistance from a maintainer.
> Qt - All tests were crashing before I moved to NRWT
> (http://build.webkit.org/builders/Qt%20Linux%20Release/builds/35024/steps/layout-test/logs/stdio)
> all tests still crash after moving to NRWT
> (http://build.webkit.org/builders/Qt%20Linux%20Release/builds/35032/steps/layout-test/logs/stdio).
>  Waiting for the Qt guys to wake.
> Windows - Have not attempted transition, will likely need some help.
> https://bugs.webkit.org/show_bug.cgi?id=38756
> WebKit2 - Working on remaining blockers, plan to switch over tonight
> or tomorrow.  https://bugs.webkit.org/show_bug.cgi?id=56729
>
> Thanks for your patience in this process.  Please let me know if you
> see any issues you think we may have missed.  We're still watching the
> bots and fixing issues as they appear.
>
> -eric
>
> On Tue, Jul 5, 2011 at 4:03 PM, Eric Seidel <eric at webkit.org> wrote:
>> We've turned NRWT back on for the WebKit1 Snow Leopard bots.  We
>> believe we've solved the http-lock issue and will be monitoring the
>> bots.
>>
>> -eric
>>
>> On Fri, Jul 1, 2011 at 5:54 PM, Adam Barth <abarth at webkit.org> wrote:
>>> Thanks for your patience with the disruptions on the tree today.  The
>>> bots are having some trouble with the HTTP lock that don't manifest
>>> locally or on our test slave.  I've turned NRWT back off while we try
>>> to sort that out.
>>>
>>> Thanks,
>>> Adam
>>>
>>>
>>> On Fri, Jul 1, 2011 at 12:49 AM, Adam Barth <abarth at webkit.org> wrote:
>>>> Sorry, no switch over tonight.  The WebKit2 build is too destroyed to
>>>> have any confidence that we're doing things correctly and the Snow
>>>> Leopard buildbot appears to have an errant HTTP server bound to port
>>>> 8080.  :(
>>>>
>>>> Adam
>>>>
>>>>
>>>> On Thu, Jun 30, 2011 at 11:47 AM, Adam Barth <abarth at webkit.org> wrote:
>>>>> According to https://bugs.webkit.org/show_bug.cgi?id=34984, there are
>>>>> only two remaining blocking issues:
>>>>>
>>>>> 1) Teaching new-run-webkit-tests to understand the difference between
>>>>> WebProcess crashes and other sorts of crashes in WebKit2.
>>>>> 2) Fixing some issues with the Apple Windows port.
>>>>>
>>>>> Once we get (1) fixed (hopefully today), we're going to start moving
>>>>> (non-Windows) ports over to new-run-webkit-tests (hopefully tonight).
>>>>>
>>>>> I suspect more issues will crop up once we actually flip the switch.
>>>>> Please do not hesitate to contact Eric or myself (either via #webkit
>>>>> or via bugs.webkit.org).  We'll try to respond to any issues that
>>>>> arise as quickly as possible.
>>>>>
>>>>> Thanks for your patience.
>>>>>
>>>>> Adam
>>>>>
>>>>>
>>>>> On Mon, Jun 27, 2011 at 4:26 PM, Xan Lopez <xan at gnome.org> wrote:
>>>>>> On Tue, Jun 28, 2011 at 1:17 AM, Eric Seidel <eric at webkit.org> wrote:
>>>>>>> There appear to be 6 remaining blocking issues:
>>>>>>> https://bugs.webkit.org/showdependencytree.cgi?id=34984&hide_resolved=1
>>>>>>>
>>>>>>> We would like to hear from others who have tried new-run-webkit-tests,
>>>>>>> if they have issues which they believe should block migrating to NRWT.
>>>>>>>  (If so, please file and block the master bug!)
>>>>>>
>>>>>> I can see the GTK+ port thing with Xvfb is there already, so not a lot
>>>>>> to add. NWRT is more sensitive to slow tests than the old
>>>>>> infrastructure, so we had to add a bunch of them to the expectations
>>>>>> file; I don't think this is particularly bad. In any case with the
>>>>>> Xvfb patch and my local expectations file things run beautifully and
>>>>>> way faster than before, so looking great from our side!
>>>>>>
>>>>>> Xan
>>>>>>
>>>>>>>
>>>>>>> Thanks.
>>>>>>>
>>>>>>> -eric
>>>>>>> _______________________________________________
>>>>>>> webkit-dev mailing list
>>>>>>> webkit-dev at lists.webkit.org
>>>>>>> http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
>>>>>>>
>>>>>> _______________________________________________
>>>>>> webkit-dev mailing list
>>>>>> webkit-dev at lists.webkit.org
>>>>>> http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
>>>>>>
>>>>>
>>>>
>>> _______________________________________________
>>> webkit-dev mailing list
>>> webkit-dev at lists.webkit.org
>>> http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
>>>
>>
>