[webkit-dev] Coming Soon: new-run-webkit-tests

[Eric Seidel]

Yes, I think we should keep Chromium's default low timeout.  Having
such a low timeout allows new-run-webkit-tests to easily run all flaky
tests every time, even ones which occasionally timeout.

We just need to mark slow tests as SLOW in
LayoutTests/platform/mac/test_expectations.txt

On Fri, Apr 9, 2010 at 5:14 PM, Adam Barth <abarth at webkit.org> wrote:
> The new tool had a much tighter timeout than the old tool.  We'll
> probably have to increase that.  You can also mark a test as SLOW in
> test_expectations.txt, and then it will be given a longer timeout.  It
> might be worth having a shorter timeout for developers so we know when
> we write slow tests.  :)
>
> Adam
>
>
> On Fri, Apr 9, 2010 at 5:03 PM, Yuzo Fujishima <yuzo at google.com> wrote:
>> Sorry, (mostly) false positives.
>> I've synced to r57383 and all but 3 failures are gone.
>> I don't see /tmp/layout-test-results. Where are the errors logged?
>> The following is the console output.
>> $ ./WebKitTools/Scripts/new-run-webkit-tests
>> stopping DumpRenderTree timed out, killing it
>> killed
>>   websocket/tests/frame-lengths.html -> unexpected test timed out
>>   tables/mozilla/other/slashlogo.html -> unexpected test timed out
>>   http/tests/cache/subresource-expiration.html -> unexpected test timed out
>>   tables/mozilla/other/slashlogo.html -> unexpected test timed out
>>   http/tests/cache/subresource-expiration.html -> unexpected test timed out
>>   websocket/tests/frame-lengths.html -> unexpected test timed out
>>
>> 12834 tests ran as expected, 3 didn't:
>> Regressions: Unexpected tests timed out : (3)
>>   http/tests/cache/subresource-expiration.html = TIMEOUT
>>   tables/mozilla/other/slashlogo.html = TIMEOUT
>>   websocket/tests/frame-lengths.html = TIMEOUT
>> Yuzo
>> On Sat, Apr 10, 2010 at 8:18 AM, Eric Seidel <eric at webkit.org> wrote:
>>>
>>> Sounds like a bug. You should expect 0 failures on leopard or snow
>>> leopard.  Please file a bug with a log of the failures and I will fix.
>>>
>>> On Apr 9, 2010 4:11 PM, "Yuzo Fujishima" <yuzo at google.com> wrote:
>>>
>>> Hi, Eric,
>>> In my development environment, new-run-webkit-tests reports 149 errors
>>> (out of 12692 tests) while run-webkit-tests none. Is this expected?
>>> Are all of the errors what you mean by "Exposes more flaky tests due to
>>> running tests in a non-deterministic order" ?
>>> Yuzo
>>>
>>> On Sat, Apr 10, 2010 at 7:35 AM, Eric Seidel <eric at webkit.org> wrote:
>>>>
>>>> >
>>>> > Thanks to the tireless efforts of Dirk Pranke, Ojan Vafai, and
>>>> > numerous other engineers, WebKi...
>>>>
>>>> _______________________________________________
>>>> webkit-dev mailing list
>>>> webkit-dev at lists.webkit.org
>>>> http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
>>>
>>
>>
>> _______________________________________________
>> webkit-dev mailing list
>> webkit-dev at lists.webkit.org
>> http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
>>
>>
>