[webkit-dev] Another WPT bite

[Ryosuke Niwa]

On Mon, May 8, 2017 at 10:17 PM, Brady Eidson <beidson at apple.com> wrote:
> On May 8, 2017, at 9:31 PM, youenn fablet <youennf at gmail.com> wrote:
>
> Hi all,
>
> Discussing with some WebKittens, testharness.js is more and more used in
> WebKit.
> Is it time to make testharness.js the recommended way of writing
> LayoutTests?
>
>
> Setting aside the pros or cons of testharness.js itself, I disagree with the
> principle of "1 single way to write all regression tests"
>
> In the past 11 years I've heard from multiple members of the team commenting
> on the benefits of different people writing regression tests in their own
> styles using their own techniques. We're capable of covering more edge cases
> when we don't have enforced uniformity. And I agree wholeheartedly.

Agreed.

> But now talking about testharness.js directly, I object on the grounds of "a
> file:// regression test is dirt easy to hack on and work with, whereas
> anything that requires me to have an httpd running is a PITA"

I think whether we use file:// or http:// is orthogonal point to using
testharness.js.  Many of the tests Chris and I have written using
testharness.js are checked into regular LayoutTests/ directories, and
they work just fine.

The imported web platform tests use WPT today because that's how
upstream works. But we're looking at ways to reduce the number of
tests that need to be ran under WPT for imported tests as well.

> I just object to making it the "recommended way" of writing tests.

Would you equally object to making js-test.js / js-test-pre.js the
recommended way of writing tests? If not, why?

What we're suggesting is to give preferential treatments to
testharness.js over js-test.js / js-test-pre.js when you were already
planning to write a test with the latter two scripts.

- R. Niwa