[webkit-dev] JSC String re-factorings are hosing the tree.

[Eric Seidel]

This issue has resurfaced again today.  We had 3 long breaks from 3
different people all related to JSC changes in the last 24 hours. :(

FYI, the WinEWS bot is up and running again (this time for good!) as
of 48 hours ago.  I don't think it's quite caught up yet, and it takes
about 60 mins to cycle for big builds.  However, it's at least able to
provide some information for patches which are up for review for an
hour or more.

-eric

On Wed, Apr 21, 2010 at 3:27 PM, Gavin Barraclough
<barraclough at apple.com> wrote:
> Hi Eric,
>
> Many apologies for the redness.  These changes are pretty much complete now,
> so hopefully there shouldn't be any more big file moves like this too soon.
>
> One thing that was hugely useful in minimizing the breakage as much as
> possible while making these changes was the ews bots – these generally
> helped me to get my patches building cleanly on all platforms bar Windows
> before committing.  It is a real shame that an ews bot isn't available for
> Windows, since this would be particularly useful - JSC changes frequently
> break Windows builds due to the .def files.
>
> I believe a big problem that caused the extended periods of redness was the
> slowness of the Windows test queues.  These can lag badly behind the builds,
> making failures here very are easy to miss - having landed a large change,
> and waited to watch the waterfall stay green for an extended period of time,
> it was easy to be under the misapprehension that everything was okay.  Only
> later would I discover windows test had started to fail.  Clearly there is a
> lesson I've learned here, but maybe we can find some more hardware to throw
> at these queues, to help them avoid getting quite so far behind.
>
> cheers,
> G.
>
>
> On Apr 21, 2010, at 1:39 PM, Eric Seidel wrote:
>
>> A large portion of the tree redness in the last 3 days is due to JSC
>> string re-factoring.
>>
>> We need to build some better tools, or find some better method to land
>> these changes w/o hosing the tree.  I'm happy to help with building of
>> said tools if folks have requests/suggestions.
>>
>> Broken in 58001  Fixes: 58003, 58006, 58007, 58008, 58010
>> Time: 55m
>>
>> Broken in 57904   Fixes: 57908, 57911, 57912, 57917
>> Time 1hr 45m
>>
>> Broken in 57829   Attempted fix: 57835, Rolled out in:57853
>> Time: 3h 21m
>>
>> Re-broke in 57879   Fixes: 57883, 57884
>> Time: 3h 3m
>>
>> Getting 57829 landed resulted in nearly a full work-day of tree
>> redness. :(  Also, even once a change is fixed, it will take 15 mins
>> or so for all the bots to cycle green.
>>
>> -eric
>> _______________________________________________
>> webkit-dev mailing list
>> webkit-dev at lists.webkit.org
>> http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
>
>