[webkit-efl] Changing the test expectations lookup sequence
Thiago Marcos P. Santos
tmpsantos at gmail.com
Thu Mar 7 02:10:23 PST 2013
Currently on the EFL WK2 port, the lookup sequence for test
expectations is "efl-wk2 -> wk2 -> efl -> generic".
The "efl" part of this sequence is shared between EFL WebKit1 and EFL WebKit2.
I'm currently facing some challenges when gardening because of:
1. Currently we have some tests skipped (~90) on the "wk2"
expectations that have different expectations on "efl". It gives you
the impression that the test is running (because is marked as [
Failure ] when in fact is being skipped because "wk2" has precedence.
The flakiness dashboard also gets confused about it and gives a false
2. Removing or adding a file to the shared expectation requires
testing on both ports, which IMO is a heavy burden.
3. I'm trying to remove from the flaky list some tests that are now
considered stable (no failures on the past 100 runs). Because of the
shared expectations, I will have to analyse the results of one extra
bot, even though I have no intentions of doing gardening for WebKit1.
I would like to suggest changing the lookup sequence to "efl-wk2 ->
wk2 -> generic", so we can make EFL WK2 port completely independent.
More information about the webkit-efl