[Webkit-unassigned] [Bug 222885] Formalize rules for what is an Experimental Feature

[bugzilla-daemon at webkit.org]

https://bugs.webkit.org/show_bug.cgi?id=222885

--- Comment #22 from Ryosuke Niwa <rniwa at webkit.org> ---
(In reply to Maciej Stachowiak from comment #21)
> (In reply to Sam Sneddon [:gsnedders] from comment #19)
> > https://chromium.googlesource.com/chromium/src/+/HEAD/third_party/blink/
> > renderer/platform/RuntimeEnabledFeatures.md is the Blink guidelines for this.
> > 
> > In short, they have four basic categories:
> > 
> >  * status: <missing> - anything goes.
> > 
> >  * status: "test" - it must be in a sufficient state to permit internal
> > testing. For example, enabling it should not be known to easily cause
> > crashes, leak memory, or otherwise significantly effect the reliability of
> > bots. Consideration should also be given to the potential for loss of test
> > coverage of shipping behavior.
> > 
> >  * status: "experimental" - it should be far enough along to permit testing
> > by early adopter web developers. Many chromium enthusiasts run with
> > --enable-experimental-web-platform-features, and so promoting a feature to
> > experimental status can be a good way to get early warning of any stability
> > or compatibility problems. If such problems are discovered (e.g. major
> > websites being seriously broken when the feature is enabled), the feature
> > should be demoted back to no status or status: "test" to avoid creating
> > undue problems for such users.
> > 
> >  * status: "stable" - it must be complete and ready for use by all chrome
> > users.
> > 
> > Personally, I quite like this, as it gives a clear indication as to how
> > usable we expect the browser to be with the feature enabled.
> > 
> > Roughly speaking, we currently have three statuses effectively:
> > 
> >  * off-by-default - anything goes
> > 
> >  * enabled in TestOptions::defaults - equivalent of Blink's status: "test"
> > 
> >  * on-by-default - equivalent of Blink's status: "stable"
> > 
> > What we don't have is anything equivalent to their status: "experimental",
> > and as an outsider it’s very hard to distinguish features where they’re
> > off-by-default because (e.g.) just the interfaces have been added with
> > no/little implementation from those where they’re off-by-default because
> > we’d rather they remain off until the next release branches (due to lack of
> > confidence or whatever reason) but are otherwise believed to be complete.
> > 
> > Note that this does have some degree of ramification for https://wpt.fyi's
> > default view, where STP is currently run (at the WebKit team’s request) with
> > only features enabled by default, whereas Chrome Dev is run with
> > --enable-experimental-web-platform-features and Firefox Nightly is run with
> > all features enabled in CI, hence STP is at a disadvantage to the others,
> > but currently there’s no easy route to determine what a reasonable set of
> > features to enable would be (and leads to a bigger loss of coverage for
> > Safari, given Chrome and Firefox both have their beta releases with only
> > their on-by-default settings but Safari beta results aren’t available).
> > 
> > On the whole, my preference would be:
> > 
> >  * Have a single runtime-enabled features file (this means changing status
> > should just be a -1/+1 change of the status line).
> > 
> >  * Adopt the four categories from Blink, defining them in some sensible but
> > largely analogous way, perhaps splitting the status: <missing> into a
> > further status: "debug only" for things like ITP disablement.
> > 
> >  * Expose those four categories through the WebKit API.
> > 
> > At that point, applications embedding WebKit can split the list up in their
> > UI in a more meaningful chunks, and potentially add "enable all experimental
> > features" and "enable all testable features" options too.
> 
> I like this proposal a lot!
> 
> One proposed friendly amendment: let's disallow missing status (build
> failure if you try to add a feature with no status) and add meaningful
> statuses for flags that are not test, experimental, or stable. Here's a few
> I can think of:
> * unstable - meant to be an actual feature, which would eventually evolve to
> "test" status and beyond, but not yet functional and usable.
> * internal debug - flags like "Disable Accelerated Compositing" that we
> think browser devs might need to use to diagnose problems, but that we don't
> think are reasonable useful for web developers
> * developer debug - debugging flags that we think are reasonable for web
> developers to try to diagnose problems (perhaps the WebRTC flags, currently
> in their own submenu, would fall in this category)
> 
> Maybe pulling on this thread there'd be too many statuses to represent every
> one, but missing status can be a mistake that's easy to make and hard to
> spot (hard to notice lack of something), so I think it would be better to
> have statuses that can apply to any kind of flag we'd like to have.

So the list of categories we're thinking is:
"unstable" - feature in active/stale development
"internal debug" - for WebKit engineers
"developer debug" - toggling flags for web developers
"tests" - enabled by default in DRT/WTR
"experimental" - features enable by default in STP, etc... but not yet stable
"stable" - enabled by default & ready for prime time

I think "prototype" or "exploratory" might be better instead of "unstable". I also feel like the distinction between "experimental" vs "developer debug" might be a bit confusing for web developers.

Maybe we need to explicitly call it some of the flags are for feature enablement and others are for debugging purposes so something like:
"Internal Debug Options"
"Developer Debug Options"
"Exploratory Features"
"Test Enabled Features"
"Experimentally Enabled Features"
"Stably Enabled Features"

-- 
You are receiving this mail because:
You are the assignee for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-unassigned/attachments/20210323/12c2f58f/attachment-0001.htm>