[Webkit-unassigned] [Bug 54855] New: Let NRWT print a detailed summary of expected results

[bugzilla-daemon at webkit.org]

https://bugs.webkit.org/show_bug.cgi?id=54855

           Summary: Let NRWT print a detailed summary of expected results
           Product: WebKit
           Version: 528+ (Nightly build)
          Platform: PC
        OS/Version: Mac OS X 10.5
            Status: UNCONFIRMED
          Severity: Normal
          Priority: P2
         Component: Tools / Tests
        AssignedTo: webkit-unassigned at lists.webkit.org
        ReportedBy: wangxianzhu at google.com


I've found that a detailed summary of expected results is useful to me to understand the situation of current expectations.

I propose an option '--print=detailed-expected' which behaves like '--print=expected' but gives detailed summary of each combination of expectations of each type.

An example output is as follows:

Found:  23149 tests
Expect: 13564 passes   (13500 now,   64 wontfix)
Expect:   821 failures (  756 now,   65 wontfix)
            1 failures (    1 now,    0 wontfix) CRASH
          252 failures (  207 now,   45 wontfix) FAIL
           94 failures (   94 now,    0 wontfix) IMAGE
           71 failures (   61 now,   10 wontfix) IMAGE+TEXT
            1 failures (    1 now,    0 wontfix) MISSING
          390 failures (  381 now,    9 wontfix) TEXT
           12 failures (   11 now,    1 wontfix) TIMEOUT
Expect:   150 flaky    (  149 now,    1 wontfix)
            4 flaky    (    4 now,    0 wontfix) FAIL CRASH
            3 flaky    (    3 now,    0 wontfix) FAIL TIMEOUT
            3 flaky    (    3 now,    0 wontfix) IMAGE IMAGE+TEXT
            2 flaky    (    2 now,    0 wontfix) IMAGE+TEXT CRASH
           17 flaky    (   17 now,    0 wontfix) PASS CRASH
            6 flaky    (    5 now,    1 wontfix) PASS FAIL
            1 flaky    (    1 now,    0 wontfix) PASS FAIL CRASH
            1 flaky    (    1 now,    0 wontfix) PASS FAIL TIMEOUT
            1 flaky    (    1 now,    0 wontfix) PASS FAIL TIMEOUT CRASH
           21 flaky    (   21 now,    0 wontfix) PASS IMAGE
            2 flaky    (    2 now,    0 wontfix) PASS IMAGE IMAGE+TEXT
            5 flaky    (    5 now,    0 wontfix) PASS IMAGE+TEXT
            1 flaky    (    1 now,    0 wontfix) PASS MISSING
           38 flaky    (   38 now,    0 wontfix) PASS TEXT
            2 flaky    (    2 now,    0 wontfix) PASS TEXT CRASH
            7 flaky    (    7 now,    0 wontfix) PASS TEXT TIMEOUT
            2 flaky    (    2 now,    0 wontfix) PASS TEXT TIMEOUT CRASH
           21 flaky    (   21 now,    0 wontfix) PASS TIMEOUT
            1 flaky    (    1 now,    0 wontfix) PASS TIMEOUT CRASH
            3 flaky    (    3 now,    0 wontfix) TEXT CRASH
            1 flaky    (    1 now,    0 wontfix) TEXT IMAGE IMAGE+TEXT
            3 flaky    (    3 now,    0 wontfix) TEXT IMAGE+TEXT
            2 flaky    (    2 now,    0 wontfix) TEXT MISSING
            3 flaky    (    3 now,    0 wontfix) TEXT TIMEOUT
Expect:  8614 skipped  (  400 now, 8214 wontfix)
          175 skipped  (   41 now,  134 wontfix) FAIL
            1 skipped  (    0 now,    1 wontfix) FAIL IMAGE
           53 skipped  (    1 now,   52 wontfix) FAIL TIMEOUT
            2 skipped  (    0 now,    2 wontfix) IMAGE+TEXT
         1663 skipped  (    2 now, 1661 wontfix) PASS
          395 skipped  (  212 now,  183 wontfix) PASS FAIL
          267 skipped  (    2 now,  265 wontfix) PASS FAIL TIMEOUT
            5 skipped  (    5 now,    0 wontfix) PASS TIMEOUT
            1 skipped  (    1 now,    0 wontfix) PASS TIMEOUT CRASH
         5740 skipped  (   14 now, 5726 wontfix) TEXT
            2 skipped  (    1 now,    1 wontfix) TEXT TIMEOUT
            1 skipped  (    1 now,    0 wontfix) TEXT TIMEOUT CRASH
          308 skipped  (  120 now,  188 wontfix) TIMEOUT
            1 skipped  (    0 now,    1 wontfix) TIMEOUT CRASH

-- 
Configure bugmail: https://bugs.webkit.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.