[Webkit-unassigned] [Bug 186045] New: webkit-test-runner: Add support for the reftest-wait attribute

[bugzilla-daemon at webkit.org]

https://bugs.webkit.org/show_bug.cgi?id=186045

            Bug ID: 186045
           Summary: webkit-test-runner: Add support for the reftest-wait
                    attribute
           Product: WebKit
           Version: WebKit Nightly Build
          Hardware: Unspecified
               URL: https://web-platform-tests.org/writing-tests/reftests.
                    html
                OS: Unspecified
            Status: NEW
          Severity: Normal
          Priority: P2
         Component: Tools / Tests
          Assignee: webkit-unassigned at lists.webkit.org
          Reporter: fred.wang at free.fr
                CC: fred.wang at free.fr, jfernandez at igalia.com,
                    lforschler at apple.com, Ms2ger at igalia.com,
                    rego at igalia.com, rwlbuis at gmail.com, youennf at gmail.com

>From the WPT doc, reftest-wait is a mechanism similar to testRunner's waitUntilDone / notifyDone():

"Controlling When Comparison Occurs

By default reftest screenshots are taken after the load event has fired, and web fonts (if any) are loaded. In some cases it is necessary to delay the screenshot later than this, for example because some DOM manipulation is required to set up the desired test conditions. To enable this, the test may have a class="reftest-wait" attribute specified on the root element. This will cause the screenshot to be delayed until the load event has fired and the reftest-wait class has been removed from the root element. Note that in neither case is exact timing of the screenshot guaranteed: it is only guaranteed to be after those events."

The reftest-wait attributed is already used by the following tests imported into WebKit:

./FileAPI/url/url_xmlhttprequest_img.html
./css/css-grid/abspos/absolute-positioning-changing-containing-block-001.html
./css/css-grid/abspos/grid-item-absolute-positioning-dynamic-001.html
./css/css-grid/abspos/grid-positioned-item-dynamic-change-001.html
./css/css-ui/text-overflow-005.html
./css/css-ui/text-overflow-017.html
./css/css-ui/text-overflow-021.html
./css/mediaqueries/min-width-tables-001.html
./css/selectors/focus-within-001.html
./css/selectors/focus-within-002.html
./css/selectors/focus-within-003.html
./css/selectors/focus-within-004.html
./css/selectors/focus-within-005.html
./css/selectors/focus-within-006-expected.html
./css/selectors/focus-within-006.html
./css/selectors/focus-within-007.html
./css/selectors/focus-within-008.html
./css/selectors/focus-within-010.html
./css/selectors/focus-within-shadow-001.html
./css/selectors/focus-within-shadow-002.html
./css/selectors/focus-within-shadow-003.html
./css/selectors/focus-within-shadow-004.html
./css/selectors/focus-within-shadow-005.html
./css/selectors/focus-within-shadow-006.html
./css/selectors/selector-placeholder-shown-type-change-001.html
./css/selectors/selector-placeholder-shown-type-change-002.html
./css/selectors/selector-placeholder-shown-type-change-003.html
./css/selectors/selector-read-write-type-change-002.html
./css/selectors/selector-required-type-change-002.html
./mathml/relations/html5-tree/href-click-1.html
./mathml/relations/html5-tree/href-click-2.html

Probably, most of them pass because the expected final rendering happens fast enough. However, this is a bit fragile and might lead to flacky tests. Attached is a patch that moves WPT's reftest_wait_0 test so that one can run it with Tools/Scripts/run-webkit-tests reftest_wait_0.html. As indicated in the test description, it is expected to fail since: 

- reftest_wait_0.html is a doc with a red background but after 2 seconds the background color is changed to green (The end of the test is controlled via the reftest-wait attribute).
- reftest_wait_0-expected.html is a doc with a red background.

However, because webkit-test-runner does not support reftest-wait, the screenshot of reftest_wait_0.html is actually taken before the background color is turned into green.

-- 
You are receiving this mail because:
You are the assignee for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-unassigned/attachments/20180529/74823f2d/attachment.html>