[Webkit-unassigned] [Bug 103824] clean up test result summarization / aggregating code in nrwt

[bugzilla-daemon at webkit.org]

https://bugs.webkit.org/show_bug.cgi?id=103824


Dirk Pranke <dpranke at chromium.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |eric at webkit.org




--- Comment #1 from Dirk Pranke <dpranke at chromium.org>  2012-12-01 17:31:21 PST ---
The general model I have in mind can be thought of functionally or in a dataflow manner ...

1) get a list of test paths (e.g., ['fast/html', 'fast/forms']

2) convert those into a list of test names

3) convert those into a list of TestInputs

4) run the tests, each test produces a list of TestFailures plus other data (like test run time) which is collected into a TestResult for each test

5) the list of TestResults is collected into a TestPass class (what is currently called a ResultSummary), or some similar name; the concept is "making a pass over the tests" rather than "this test passed", but I don't like the confusion. 

6) if we had any unexpected failures and were not told to not retry failures, we do a second pass over those, producing a second TestPass objects (currently what is called the retry_summary)

7) from those two TestPass objects we can merge the results and summarize them (in the routine currently called summarize_results that returns a flat JSON/POD object that can be stored as full_results.json)

8) the results of 6 and 7 are collected together (along with the single exit code containing the number of unexpected failures) into a RunDetails object and handed back to callers, who now will have a full description of everything that happened during the test run.

I'm not wild about any of these names and am open to suggestions. Note that all of this basically describes the existing flow and objects; I'm not changing anything much, just cleaning it up.

-- 
Configure bugmail: https://bugs.webkit.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.