[webkit-dev] Another WPT bite

[Maciej Stachowiak]

> On May 8, 2017, at 11:15 PM, Ryosuke Niwa <rniwa at webkit.org> wrote:
> 
> On Mon, May 8, 2017 at 11:01 PM, Brady Eidson <beidson at apple.com <mailto:beidson at apple.com>> wrote:
>  <x-redundant-cluster-toggle://0> <x-redundant-cluster-toggle://0> <x-redundant-cluster-toggle://0>On May 8, 2017, at 10:44 PM, Ryosuke Niwa < <x-redundant-cluster-toggle://0>rniwa at webkit.org <mailto:rniwa at webkit.org>> wrote:
>>> On Mon, May 8, 2017 at 10:17 PM, Brady Eidson <beidson at apple.com <mailto:beidson at apple.com>> wrote:
>>>> But now talking about testharness.js directly, I object on the grounds of "a
>>>> file:// regression test is dirt easy to hack on and work with, whereas
>>>> anything that requires me to have an httpd running is a PITA"
>>> I think whether we use file:// or http:// is orthogonal point to using
>>> testharness.js.  Many of the tests Chris and I have written using
>>> testharness.js are checked into regular LayoutTests/ directories, and
>>> they work just fine.
>> Yes, I misunderstood this in Youenn's original message. Good to know!
>>>> I just object to making it the "recommended way" of writing tests.
>>> Would you equally object to making js-test.js / js-test-pre.js the
>>> recommended way of writing tests?
>> Yes.
>>> If not, why?
>> N/A
>>> What we're suggesting is to give preferential treatments to
>>> testharness.js over js-test.js / js-test-pre.js when you were already
>>> planning to write a test with the latter two scripts.
>> "It's okay to write your test however you'd like. If you were considering using js-test, maybe you should consider using testharness instead."
>> Is that's what's being proposed?
> 
>> 
>>> On May 8, 2017, at 10:44 PM, Ryosuke Niwa <rniwa at webkit.org <mailto:rniwa at webkit.org>> wrote:
>>> 
>>> On Mon, May 8, 2017 at 10:17 PM, Brady Eidson <beidson at apple.com <mailto:beidson at apple.com>> wrote:
>>> 
>>>> But now talking about testharness.js directly, I object on the grounds of "a
>>>> file:// regression test is dirt easy to hack on and work with, whereas
>>>> anything that requires me to have an httpd running is a PITA"
>>> 
>>> I think whether we use file:// or http:// is orthogonal point to using
>>> testharness.js.  Many of the tests Chris and I have written using
>>> testharness.js are checked into regular LayoutTests/ directories, and
>>> they work just fine.
>> 
>> Yes, I misunderstood this in Youenn's original message. Good to know!
>>> 
>>>> I just object to making it the "recommended way" of writing tests.
>>> 
>>> Would you equally object to making js-test.js / js-test-pre.js the
>>> recommended way of writing tests?
>> 
>> Yes.
>> 
>>> If not, why?
>> 
>> N/A
>> 
>>> What we're suggesting is to give preferential treatments to
>>> testharness.js over js-test.js / js-test-pre.js when you were already
>>> planning to write a test with the latter two scripts.
>> 
>> "It's okay to write your test however you'd like. If you were considering using js-test, maybe you should consider using testharness instead."
>> 
>> Is that's what's being proposed?

Besides other issues mentioned, testharness tends to result in more verbose tests compared to js-test, at least for simple cases.

> 
> The thing I specifically asked Youenn to ask is, whether we should
> place a test inside LayoutTests/wpt like LayoutTests/http/tests when
> we want to write a test using testharness.js which requires some sort
> of network code.
> 
> Since people have had some opinions about directory structures in the past.

It seems like we need a few different directories, here are my opinions on them:

(1) Imported web platform tests that don't need a server
    Currently LayoutTests/imported/w3c/web-platform-tests, which seems fine.
(2) Imported web platform tests that do need a server
    Probably should be under LayoutTests/imported/w3c/ somewhere, or maybe under http/tests/ per point (4)
(3) Custom testharness.js tests that don't need a server 
    Probably these should just go in their normal topic-specific directories and should not need a special directory
(4) Custom testharness.js tests that do need a server
    Can these just be a subdirectory of http/tests/? We have websocket and ssl/tls tests in there too. Would be nice to not need a separate directory for networking tests that to use a particular test framework.

Regards,
Maciej

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20170509/5cdac357/attachment-0001.html>