[webkit-dev] Tests that separate JS and HTML

Ojan Vafai ojan at chromium.org
Mon Aug 16 12:44:39 PDT 2010

I agree that dealing with the script to generate tests and having the actual
test content be in a different file is a significant maintenance overhead.
But I also think that having standard testing code across many tests reduces
the amount of effort it takes to understand a test.

We discussed this at
https://lists.webkit.org/pipermail/webkit-dev/2010-July/013673.html. I think
Maciej's solution sounds ideal, although I'm not convinced it's worth
including doctype for news tests. If we use the load event, then we can drop
the post script entirely and then it's just about including a single script
line at the top. FWIW, this is how the dump-as-markup.js tests and the
proposed perf tests work (https://bugs.webkit.org/show_bug.cgi?id=43997).


On Fri, Aug 13, 2010 at 3:17 AM, Alexey Proskuryakov <ap at webkit.org> wrote:

> On multiple occasions, I've been noticing that separating JavaScript for
> tests into its own file has been causing me pain. There are several
> disadvantages that I know of:
> 1. One doesn't see test content when a tool (such as run-webkit-tests)
> sends them to a failing test. If I want to see it in browser, I need to
> copy/paste script src, or manually edit the address in address bar.
> 2. The proliferation of script-tests prompts some contributors to
> unnecessarily complicate their tests to fit the format. Sometimes, the DOM
> that could just be in the HTML content is dynamically created. Other times,
> complicated schemes for async tests are used, instead of directly calling
> notifyDone().
> 3. These tests cannot be attached to Bugzilla bugs in a working form (e.g.
> if one wants to file a bug against another browser).
> 4. The test does not live next to its expected results, so one has to keep
> two folders open when working on the test.
> On the other hand, I can't think of any clear advantages. Some potential
> advantages could be:
> 1. Ease of boilerplate modification. I don't see why we would want to
> modify boilerplate in existing tests - it's more likely that such changes
> would break them than improve them.
> 2. Ease of test writing. Maybe people who use script tests often would
> object, but for me, a step that I must do any time I start working on a test
> - at a very specific moment - is a problem. Usually, I start with a simple
> test that I keep on desktop (or in Bugzilla), and only move it to
> LayoutTests later.
> 3. Higher quality of boilerplate. The boilerplate is quite straightforward,
> all the logic is in external files.
> 4. Tests become more concise. This is true, but I don't think it has
> practical benefits - it's quite easy to focus on test code even if
> everything is in one file.
> Of course, there is the special case of fast/js tests, which we (I think)
> still hope to run without a WebView one day. For that, keeping JS separate
> is obviously desirable.
> We have lots of imported tests in WebKit regression test suite, as well as
> tests employing various custom runners developed by WebKit contributors. So
> to a degree, the manner in which a test is written will always remain a
> matter of personal preference, we can't and shouldn't standardize this.
> However, as I was personally seeing more script-tests not just being added,
> but living their active lives, I started feeling frustration towards them.
> When reviewing patches, I recommend against using tests that have their main
> logic in files other than the main HTML.
> - WBR, Alexey Proskuryakov
> _______________________________________________
> webkit-dev mailing list
> webkit-dev at lists.webkit.org
> http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20100816/2725d15a/attachment.html>

More information about the webkit-dev mailing list