[webkit-dev] Upstreaming from LayoutTests to web-platform-tests, coordinating Blink+WebKit

[Alexey Proskuryakov]

(re-sending from a correct address)

> 17 нояб. 2017 г., в 9:18, youenn fablet <youennf at gmail.com <mailto:youennf at gmail.com>> написал(а):
> 
> Chris recently noticed that some heavily used files (testharness*) were cacheable through Apache but not WPT.
> This is now fixed and should improve WPT performances.

This is part of <https://bugs.webkit.org/show_bug.cgi?id=178277 <https://bugs.webkit.org/show_bug.cgi?id=178277>>, another part is that the server is 10x slower than Apache.

I just tested on my MacBook Pro, and WPT tests took 23% of time while being only 9% of the total count. Taking in mind that WebKit own tests have higher value due to the way we choose what to test (see below), that's not a great story in my opinion.

One other thing that we discussed before was the operational complexity of running WPT tests. We frequently need to share tests with people who don't work on WebKit directly, but have the need to edit and run our tests. Inability to drag and drop a local copy into a Safari window is a deterrent to addressing problems caught by the tests. I think that the response we got was that tests will continue to require a server to run.

Let me explain why I think that WebKit tests are often more valuable as regression tests than WPT tests are. We add tests as we fix bugs, so we know that the tests are generally for problems that have a high impact on users and developers - that's because someone actually discovered the problem, and someone prioritized it highly enough to fix. We also know that our tests cover code that is error prone, which is why we had bugs in the first place. Of course, anything can be broken, but certain things are less likely to. Compliance tests written for specs are also valuable, but at some point we need to prioritize which tests to investigate and even to run.

- Alexey

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20171117/8a9f6b27/attachment.html>