[webkit-dev] Settings and Testing (Settings, RuntimeEnabledFeatures, WebPreferences)

[Joseph Pecoraro]

Toggling settings in tests is a confusing area, with multiple (sometimes conflicting) ways to modify settings.

After cleaning things up a bit in r211006 I wanted to document my understanding of the patterns I see right now. Much of this was new to me so others may find it useful.

Different Types of Settings:

• RuntimeEnabledFeatures (WebCore)
- Should be for features that have Web Exposed APIs that will only be exposed if the feature is enabled
- Should be for features that must be toggled before a page has loaded, since after a page load may be too late for EnabledAtRuntime bindings

• Settings (WebCore)
- All other settings that change engine behavior
- Okay to toggle at any time and WebCore often respects the setting immediately

• WebPreferences (WebKit, WebKit2)
- API level Toggles for the above settings
- Toggles for WebKit level behavior (e.g. Tabbing between links behavior)
- Ports may have different default values for features

Policies for Settings in LayoutTests:

• TestRunners (WebKitTestRunner / DumpRenderTree)
- Should set up a consistent environment before each test
	- Ports have different defaults, so having TestRunner make things consistent seems reasonable
	- WTR: TestController::resetPreferencesToConsistentValues
	- DRT: resetWebPreferencesToConsistentValues
- Should enable Experimental features for tests
	- Since we allow users to enable these features they should be enabled and get maximum testing
	- WTR: WKPreferencesEnableAllExperimentalFeatures
	- DRT: enableExperimentalFeatures

• If a test wants to change a Setting:
- Use: internals.settings.setFoo(value);
- Settings in Settings.in are autogenerated, others are added to InternalSettings as needed
- InternalSettings handles resetting modifications before the next test

• If a test wants to change a RuntimeEnabledFeature:
- No set pattern.
- Some tests use internals.settings but this seems inappropriate, since the page has already loaded
- Some tests use the special comment syntax parsed by TestRunners; this makes sense, but would not be good for imported tests

• If a test wants to change a WebPreference / WebKit level setting:
- Use: testRunner.someMethod(value);
- Avoid: testRunner.overridePreference(preferenceKey, value);
- TestRunners should reset these back to a consistent value before the next test

Let me know if you think anything is inaccurate or if you have ideas to simplify / improve.

- Joe
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.webkit.org/pipermail/webkit-dev/attachments/20170120/724a178c/attachment.html>