[webkit-dev] run-bindings-tests

[Ryosuke Niwa]

On Thu, Sep 8, 2011 at 12:19 PM, Oliver Hunt <oliver at apple.com> wrote:
>
> The other problem is "people check in without looking at the bots".  I do
> try to watch the bots, but the time between me landing and the bots actually
> going red can literally be hours.  Of course i'm away from irc/email
> whatever when I land a patch at 4pm and it turns the bots red at 11pm.
>

I agree. Some bots take 3-4 hours to run (especially if you touch Node.h,
platform.h, etc... that cause full or near-full rebuild of WebCore), and
it's not realistic for us to expect contributors to keep watching at the
waterfall until all bots go green. Sure, they should be ultimately
responsible for keeping them green and should be checking them back but the
fact most of bots always have some tests failing make this process
needlessly harder as well.

On Thu, Sep 8, 2011 at 12:27 PM, Adam Barth <abarth at webkit.org> wrote:
>
> Maybe we should teach run-webkit-tests how to run all the various
> testing scripts?  That way there's just one command to learn and run.
> run-bindings-tests takes 3.6s on my machine, so I suspect the extra
> time involved isn't noticeable.
>

On Thu, Sep 8, 2011 at 12:29 PM, Eric Seidel <eseidel at google.com> wrote:

> If the objection against run-bindings-tests is that they're not part
> of some larger test script which developers can run locally, it's very
> easy to add a wrapper script which runs all known testing harnesses.
>
> The test tests which currently run on the bots include:
> run-webkit-tests (minutes)
> run-javascriptcore-tests (45s)
> test-webkitpy (32s)
> test-webkitperl (2.0s)
> run-binding-tests (2.4s)
> run-api-tests (failed on my machine, so I can't tell you how long)
>
> There are a couple other scripts which run on specific platforms, but
> that's the core set.


I like this idea!  But I think having each kind of test as a separate step
on buildbot so this wrapper script is either used only locally or else we
need to teach buildslave/buildbot how to talk to this script.

- Ryosuke
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20110908/f1784247/attachment.html>