[webkit-dev] Test Expectations: A Pipe Dream

[Dimitri Glazkov]

(re-sending to webkit-dev, since this idea was just mentioned on #webkit)

If wishes were fishes, checking layout tests expectations into WebKit
would work like so:

* Authors of layout tests never need to submit test expectations with
their patches

* All new patches are flagged by a Layout Test Expectations Resolver
(LaTER from here on).

* LaTER is a set of EWS bots (one for each platform). It takes the new
patch, applies/builds/runs tests/generates expectations.

* Newly generated expectations are compared against existing
expectations. A minimal set of changes for each platform is created.

* This set of generated expectations is merged with the changes in the
original patch, and attached to bug as a new patch, obsoleting the
original patch. LaTER knows not to chew on this patch again.

* The rest of the process (review/cq/land) is the same as it was before.

Thoughts? Comments? More crazy ideas?

:DG<