> On Wed, Sep 12, 2018 at 8:49 PM youenn fablet <youennf(a)gmail.com> wrote:
>
>>
>> Since an experimental feature flag should not remain off for a long time,
>>> a per-experimental-feature file listing the tests to which activate the
>>> feature could be considered.
>>>
>>>
>>> That clashes with Ryosuke's point that moving tests shouldn't change
>>> behaviour.
>>>
>>
>> I think it is better to discover the issue at the moment tests are moved.
>> When moving/creating such tests in WPT, the potential issue will only
>> arise after the export/import cycle. Chances are that the issue will get
>> unnnoticed.
>>
>
> No, the move here is like moving it for debugging purposes. The test
> failing to function just because it got moved out of a test directory is
> bad.
>
> We already have some hard-coded list of paths under which WTR/DRT behave
> differently, and it bites me every time I write tests in those directories,
> or write a test outside of that directory expecting the test to work.
>
> I would strongly object to directory/file-based feature enabling/disabling.
>
> The larger problems are:
>>>
>>> - We don't want experimental features to be on in shipping browsers. It
>>> might be ok for them to be always on in tests, but then MiniBrowser would
>>> be different from WKTR. At the moment they are always on in tests, and use
>>> their default value in MiniBrowser (and shipping).
>>>
>>
>> When implementing a new feature, the first thing we usually do is having
>> an experimental flag turned off by default. At some point, we turn on the
>> experimental flag for tests. When ready, the flag is turned on by default.
>>
>> It is nice to keep the cost of implementing this workflow as low as
>> possible.
>> Having to tweak layout tests does not go in that direction.
>> If this workflow was fully implementable by just updating
>> WebPreferences.yaml, that would be great.
>> That could also allow running mini browser in a mode where the same flags
>> would be turned on as WKTR.
>>
>
> I would share the same concern Dean raised. The fact WTR/DRT behave
> differently from trunk is bad. We end up accidentally enabling or disabling
> features on trunk because WTR/DRT don't test what's enabled/disabled on
> trunk. The cost of that risk, to me, is a lot higher than the marginal cost
> of having to add a single line directive in each test as one develops a
> feature.
>
> - R. Niwa
>
>