[Webkit-unassigned] [Bug 222885] Formalize rules for what is an Experimental Feature

[bugzilla-daemon at webkit.org]

https://bugs.webkit.org/show_bug.cgi?id=222885

Sam Sneddon [:gsnedders] <gsnedders at apple.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |gsnedders at apple.com

--- Comment #19 from Sam Sneddon [:gsnedders] <gsnedders at apple.com> ---
https://chromium.googlesource.com/chromium/src/+/HEAD/third_party/blink/renderer/platform/RuntimeEnabledFeatures.md is the Blink guidelines for this.

In short, they have four basic categories:

 * status: <missing> - anything goes.

 * status: "test" - it must be in a sufficient state to permit internal testing. For example, enabling it should not be known to easily cause crashes, leak memory, or otherwise significantly effect the reliability of bots. Consideration should also be given to the potential for loss of test coverage of shipping behavior.

 * status: "experimental" - it should be far enough along to permit testing by early adopter web developers. Many chromium enthusiasts run with --enable-experimental-web-platform-features, and so promoting a feature to experimental status can be a good way to get early warning of any stability or compatibility problems. If such problems are discovered (e.g. major websites being seriously broken when the feature is enabled), the feature should be demoted back to no status or status: "test" to avoid creating undue problems for such users.

 * status: "stable" - it must be complete and ready for use by all chrome users.

Personally, I quite like this, as it gives a clear indication as to how usable we expect the browser to be with the feature enabled.

Roughly speaking, we currently have three statuses effectively:

 * off-by-default - anything goes

 * enabled in TestOptions::defaults - equivalent of Blink's status: "test"

 * on-by-default - equivalent of Blink's status: "stable"

What we don't have is anything equivalent to their status: "experimental", and as an outsider it’s very hard to distinguish features where they’re off-by-default because (e.g.) just the interfaces have been added with no/little implementation from those where they’re off-by-default because we’d rather they remain off until the next release branches (due to lack of confidence or whatever reason) but are otherwise believed to be complete.

Note that this does have some degree of ramification for https://wpt.fyi's default view, where STP is currently run (at the WebKit team’s request) with only features enabled by default, whereas Chrome Dev is run with --enable-experimental-web-platform-features and Firefox Nightly is run with all features enabled in CI, hence STP is at a disadvantage to the others, but currently there’s no easy route to determine what a reasonable set of features to enable would be (and leads to a bigger loss of coverage for Safari, given Chrome and Firefox both have their beta releases with only their on-by-default settings but Safari beta results aren’t available).

On the whole, my preference would be:

 * Have a single runtime-enabled features file (this means changing status should just be a -1/+1 change of the status line).

 * Adopt the four categories from Blink, defining them in some sensible but largely analogous way, perhaps splitting the status: <missing> into a further status: "debug only" for things like ITP disablement.

 * Expose those four categories through the WebKit API.

At that point, applications embedding WebKit can split the list up in their UI in a more meaningful chunks, and potentially add "enable all experimental features" and "enable all testable features" options too.

-- 
You are receiving this mail because:
You are the assignee for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-unassigned/attachments/20210323/72477a32/attachment-0001.htm>