[webkit-dev] Buildbot early exit

[Adam Barth]

We've had cases of a patch destroying the buildbots by rampaging
through thousands of test failures.  The 20 limit is a fail-stop to
keep the bots rolling.

I agree that try servers would be a better way to handle this case
than lighting the buildbot on fire.  I'd like to extend the EWS to run
tests, which could then make the new baselines available.  Some other
folks have expressed interest in having that work too.  If you're
interested in helping to make that happen, let me know.

Adam


On Mon, Feb 8, 2010 at 6:51 PM, Nikolas Zimmermann
<zimmermann at physik.rwth-aachen.de> wrote:
> Good morning crowd,
>
> can anyone explain the "--exit-after-n-failures=20" default behaviour for
> the build bots?
>
> Background:
> Today I had to update a lot of SVG *expected.txt files including all
> platform specific variations (in LayoutTests/platform/<gtk/qt/win/mac-*>).
> I landed my patch with updated mac results and planned to wait for the build
> bots to show me all differences, so I could update all baselines
> for the individual platforms needing custom layout test results differing
> from the mac baseline.
>
> Though because the bots exit early after 20 test failures it took a long
> time to identify all differences because I could only fix a maximum of
> 20 failures per commit - that's odd. I'm aware it's a rare case to update
> 900+ test results in one shot but I thought it's worth to mention.
>
> I think the best way to resolve this problem is to run layout tests on the
> try bots, so any layout test differences can be identified before comitting.
> I recall some talk about that in the past, what's the status?
>
> Cheers,
> Niko
>
> _______________________________________________
> webkit-dev mailing list
> webkit-dev at lists.webkit.org
> http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
>