[Webkit-unassigned] [Bug 99800] New: Dromaeo should report individual test result

[bugzilla-daemon at webkit.org]

https://bugs.webkit.org/show_bug.cgi?id=99800

           Summary: Dromaeo should report individual test result
           Product: WebKit
           Version: 528+ (Nightly build)
          Platform: Unspecified
        OS/Version: Unspecified
            Status: NEW
          Severity: Normal
          Priority: P2
         Component: Tools / Tests
        AssignedTo: webkit-unassigned at lists.webkit.org
        ReportedBy: rniwa at webkit.org
                CC: eric at webkit.org, mjs at apple.com, zoltan at webkit.org,
                    morrita at google.com


Right now, Dromaeo repeats runs within subtests and report the aggregation of results from subtests. This alienates Dromaoe tests from other performance tests and prevents us from making the test more configurable for the bug 97510.

We can mitigate the problem by reporting individual result for each subtest, and then summing each record to compute the aggregated result for each run.

Say we have subtests t_1, t_2, ... t_n for some test t. Right now, Dromaeo measures 5 runs for t_1, t_2, ... t_n, and computes statistics (stdev, etc...) for each subtest t_i with i=1, ..., n. At the end of the day, we'll get n results, each of which contains mean, median, min, max, & stdev. I'd like to modify Dromaeo slightly so that it'll also report individual values it obtained in each run as "times". So for each t_i, we'll have times(t_i), which is a tuple with 5 values; e.g. (1, 2, 3, 4, 5). In total, we'll have 5n values for t. Then, I can compute the sum of times(t_i)[j] over i=1...n; this value represents a run of the aggregated test t. This is going to make Dromaeo tests behave like any other performance tests that report multiple runs.

-- 
Configure bugmail: https://bugs.webkit.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.