[webkit-dev] Another WPT bite
youennf at gmail.com
Fri May 12 12:31:00 PDT 2017
Filed https://github.com/w3c/web-platform-tests/issues/5909 and
Le ven. 12 mai 2017 à 12:08, Rick Byers <rbyers at chromium.org> a écrit :
> On Fri, May 12, 2017 at 2:43 PM, youenn fablet <youennf at gmail.com> wrote:
>> Le ven. 12 mai 2017 à 11:07, Alexey Proskuryakov <ap at webkit.org> a
>> écrit :
>>> 9 мая 2017 г., в 11:27, Simon Fraser <simon.fraser at apple.com>
>>> Another consideration here is "would my test be useful for other browser
>>> vendors". I don't think the answer is a unanimous "yes", so I think we
>>> should only use WPT for tests that will think are worth sharing.
>>> Since imported WPT tests are very flaky, and are not necessarily written
>>> to defend against important regressions, investigating issues with them is
>>> relatively lower priority than investigating issues observed with WebKit
>>> tests. So I would recommend not mixing tests for WebKit regressions with
>>> WPT tests - if your test eventually ends up in LayoutTests/imported, it
>>> will become a lot less effective.
>> WPT tests are flaky in WebKit because WPT stability bots do not run yet
>> Safari, and most tests are written in non-WebKit environment.
>> Often, it is due to the fact that tests are not passing and flakiness is
>> only seen with failing assertions.
>> From my experience with fetch API tests, I disagree that broken WPT tests
>> are lower priority.
>> I think it will change as more WebKit contributors will author WPT tests.
>> I agree that tests doing subtle WebKit-specific regression checks are not
>> good candidates for WPT upstream.
>> When the test is all about conformance with a spec, it is a very good
>>> Using the complicated harness has a similar consequence - if you use any
>>> WPT goodies like templates or server side scripts, the cost of
>>> investigating issues on the test increases, and makes the test less
>> It is true that WPT put some emphasis on easing authoring of tests.
>> I guess there is a learning curve here in WPT test debugging.
>> If you have a file with 20 tests, it is harder to debug.
>> It is also increasing the chances for flakiness/timeouts.
>> Maybe we could send that feedback.
>> WPT infra could also be improved: more verbose debug-only output,
>> enabling running selected subtest only...
>> testharness.js is actively maintained and is continuously improving.
>> Since we have specific requirements as you are describing, we should push
> Concretely, please file issues with the "infra" label
> <https://github.com/w3c/web-platform-tests/issues?q=is%3Aissue+is%3Aopen+label%3Ainfra> to
> track upstream WPT infrastructure bugs and feature requests. Google has an
> ongoing contract with Bocoup (Bob and Mike cc'd) who are improving the
> infrastructure every day. Of course there's an opportunity to make it even
> better fast with more funding from additional organizations who would
> benefit :-)
>> I don't know if there is any way to adopt WPT that won't make testing
>>> less effective. WPT tests may be useful in very rare cases, where we
>>> actively want to communicate certain subtle behavior details to other
>>> vendors - but even so, I imagine that other vendors may not put high
>>> priority on those, for the same reasons.
>> My own experience is that WPT tests are actually very valuable, at least
>> when we are talking about interoperability/spec conformance.
>> I also see WPT as an efficient way to author tests.
>>> - Alexey
>>> webkit-dev mailing list
>>> webkit-dev at lists.webkit.org
>> webkit-dev mailing list
>> webkit-dev at lists.webkit.org
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the webkit-dev