[webkit-dev] A post-mordem of today's tree redness

[Alexey Proskuryakov]

On 06.04.2010, at 10:06, Adam Barth wrote:

>> 4) It's harder to isolate regressions if these appear and disappear  
>> several times (aforementioned confusion doesn't help either).  
>> Screening bugs about regressions also becomes more error-prone.  
>> This arguments goes both ways though - it's even harder to isolate  
>> regressions if the platform in question had broken build at the time.
>
> Concretely, supposed we hadn't cleaned up the Tiger bot to be green
> recently.  I strongly suspect the regression caused by r57081 would
> have been lost in the thought process that "the Tiger bot is always
> red."  Even though the regression was real and affected every
> platform.  Had we noticed the problem (say) a month later, we would
> have had a devil of a time tracking down the issue as evidenced by the
> effort required to fix the previous ancient Tiger-only failures.


I was talking about regressions not covered by regression tests. When  
there is a bug about something that works in Safari 4, but not in ToT,  
you often need to find out when exactly it broke - and if there were  
several breakage points, it gets harder. Usually not by much, but  
sometimes this can lead you astray, and then it costs a lot.

Again, this is not to say that we should never roll out patches, but  
it's another cost to consider.

- WBR, Alexey Proskuryakov