[Webkit-unassigned] [Bug 105636] run-webkit-tests summary seems incomplete

[bugzilla-daemon at webkit.org]

https://bugs.webkit.org/show_bug.cgi?id=105636


Zan Dobersek <zandobersek at gmail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |zandobersek at gmail.com




--- Comment #1 from Zan Dobersek <zandobersek at gmail.com>  2012-12-31 12:55:47 PST ---
(In reply to comment #0)
> From http://build.chromium.org/p/chromium.webkit/builders/WebKit%20Linux/builds/3240/steps/webkit_tests/logs/stdio: 
> 
> => Results: 33706/34160 tests passed (98.7%)
> 
> => Tests to be fixed (894):
>       6 crashes                  ( 0.7%)
>      27 timeouts                 ( 3.0%)
>       4 missing results          ( 0.4%)
>      98 image-only failures      (11.0%)
> 
> => Tests that will only be fixed if they crash (WONTFIX) (6753):
> 
> 
> The numbers under "Tests to be fixed" don't add up to 100%. Are we leaving out TEXT/IMAGE+TEXT failures? Do we leave out flaky passes there?

The numbers listed here are only for failures represented by the EXPECTATIONS_ORDER variable in test_expectations.py:

EXPECTATION_ORDER = (PASS, CRASH, TIMEOUT, MISSING, FAIL, IMAGE, SKIP)

So yeah, there are text, image+text and audio expectations that are being left out. Also, the skipped tests are not numbered here, but are counted in the total (which is 894 here).

-- 
Configure bugmail: https://bugs.webkit.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.