[Webkit-unassigned] [Bug 105531] [WTR] Wrong totals in WebKitTestRunner.

[bugzilla-daemon at webkit.org]

https://bugs.webkit.org/show_bug.cgi?id=105531





--- Comment #6 from Dirk Pranke <dpranke at chromium.org>  2013-01-14 17:46:21 PST ---
Okay, sorry for the delay, I finally got time to look at this.

(In reply to comment #0)
> When using  --debug-rwt-logging parameter total are not correct.
> Example in attachment.
> 
> I see there two problems:
> -    We got 229 test ran as expected, but in totals it’s 228.

Here we have the problem of data being sliced and diced in different ways.

The "229" number comes from finding 232 tests and skipping 3 of them.

The "228" number comes from finding 228 tests and having 4 of them fail. In this case tests that are skipped are counted as having passed.

> -    We got 2 missing results – this shouldn’t be counted as expected if it's not expected.
>

I don't know what version you ran this against (so what the contents of your TestExpectations files were), but the lack of a "failed unexpectedly" in the log file probably means that you had the two tests marked as Missing, and, so, they were expected.

So, I don't see any actually incorrect computations here.

As I mentioned somewhere (email?) the problem with these routines is that there's lots of different ways one might want to slice and dice these statistics, and it's hard to come up with meaningful and clear buckets without being really, really verbose. That said, I'm definitely open to changes to the wording or how we are in fact bucketing things if that will help someone.

-- 
Configure bugmail: https://bugs.webkit.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.