[webkit-efl] Changing the test expectations lookup sequence

[Pozdnyakov, Mikhail]

Hi Thiago,

Makes sense to me. Think it's good.

BR,
Mikhail

________________________________________
From: webkit-efl-bounces at lists.webkit.org [webkit-efl-bounces at lists.webkit.org] on behalf of Thiago Marcos P. Santos [tmpsantos at gmail.com]
Sent: Thursday, March 07, 2013 12:10 PM
To: webkit-efl at lists.webkit.org
Subject: [webkit-efl] Changing the test expectations lookup sequence

Hi EWebkitters,

Currently on the EFL WK2 port, the lookup sequence for test
expectations is "efl-wk2 -> wk2 -> efl -> generic".

The "efl" part of this sequence is shared between EFL WebKit1 and EFL WebKit2.

I'm currently facing some challenges when gardening because of:

1. Currently we have some tests skipped (~90) on the "wk2"
expectations that have different expectations on "efl". It gives you
the impression that the test is running (because is marked as [
Failure ] when in fact is being skipped because "wk2" has precedence.
The flakiness dashboard also gets confused about it and gives a false
positive.

2. Removing or adding a file to the shared expectation requires
testing on both ports, which IMO is a heavy burden.

3. I'm trying to remove from the flaky list some tests that are now
considered stable (no failures on the past 100 runs). Because of the
shared expectations, I will have to analyse the results of one extra
bot, even though I have no intentions of doing gardening for WebKit1.

I would like to suggest changing the lookup sequence to "efl-wk2 ->
wk2 -> generic", so we can make EFL WK2 port completely independent.

Cheers,
_______________________________________________
webkit-efl mailing list
webkit-efl at lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-efl
---------------------------------------------------------------------
Intel Finland Oy
Registered Address: PL 281, 00181 Helsinki 
Business Identity Code: 0357606 - 4 
Domiciled in Helsinki 

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.