[webkit-dev] Another WPT bite

[youenn fablet]

>
>
> Besides other issues mentioned, testharness tends to result in more
> verbose tests compared to js-test, at least for simple cases.
>

For synchronous tests, I am not sure there is any big difference one way or
the other.
With asynchronous tests, it might be true, but using
testharness.js/promise_test usually improves things there.
 I personally find it easier to not wrap code-to-be-tested into quotes.

Another concern is the lack of verbose output which reduces the ability to
debug failing tests.
This can be partially fixed by authoring tests with that issue in mind.
For instance, having a big promise_test to handle the asynchronous aspect
of a test and nested test() inside it.

The thing I specifically asked Youenn to ask is, whether we should
> place a test inside LayoutTests/wpt like LayoutTests/http/tests when
> we want to write a test using testharness.js which requires some sort
> of network code.
>
> Since people have had some opinions about directory structures in the past.
>
>
> It seems like we need a few different directories, here are my opinions on
> them:
>
> (1) Imported web platform tests that don't need a server
>     Currently LayoutTests/imported/w3c/web-platform-tests, which seems
> fine.
>

All WPT tests are expected to run behind the WPT server.
That is the way tests are authored and tested elsewhere.
If we have an issue with that, it is best to bring that and fix that
directly in WPT.
I encountered several times small issues due to file:// based origins which
makes me think defaulting to http is a safe choice.

One concern is efficiency. We should study that and improve on that.
Another concern is the ease of running tests for developers: drag&dropping
tests into a browser instead of running a server.
We can partially accommodate this by rewriting
testharness.js/testharnessreport.js urls.
A significant and growing amount of wpt tests will not behave as expected
(other resources loaded, origins, need for specific headers, need for
https...)

(2) Imported web platform tests that do need a server
>     Probably should be under LayoutTests/imported/w3c/ somewhere, or maybe
> under http/tests/ per point (4)
>

I don't think this will work, web-platform-tests is organized in terms of
features.
There is no clear separation between file based compatible tests and http
based tests like in WebKit.

(3) Custom testharness.js tests that don't need a server
>     Probably these should just go in their normal topic-specific
> directories and should not need a special directory
>

Right.
The only case where it might make sense to put such tests in a specific
WPT-enabled directory is if the plan is to upstream these tests at some
point.
Such tests could be added in imported/w3c/web-platform-tests directly but
this requires coordination with resyncing tests at the moment.
In a not-too-far-future, I hope such tests would directly be authored in
imported/w3c/web-platform-tests.


> (4) Custom testharness.js tests that do need a server
>     Can these just be a subdirectory of http/tests/? We have websocket and
> ssl/tls tests in there too. Would be nice to not need a separate directory
> for networking tests that to use a particular test framework.
>

I do not have strong feelings there, http/wpt might make sense if it is
found easier to understand to everybody.
I'll update the patch accordingly and will land it sometimes this week if
there is no additional feedback.

Thanks!
   y
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20170509/dbbfac7f/attachment.html>