[Webkit-unassigned] [Bug 127095] run-webkit-tests should support assert-only js tests w/o expected.txt files

[bugzilla-daemon at webkit.org]

https://bugs.webkit.org/show_bug.cgi?id=127095





--- Comment #4 from youenn fablet <youennf at gmail.com>  2014-01-18 15:09:27 PST ---
> > To handle the case  of testharness.js based tests that would only partially run, the situation is less clear.
> > For existing webkit tests, we already have -expected.txt files.
> > For new webkit tests, a parameter could be added to identify the number of expected tests.
> > For existing W3C tests, I do not know what can be done on that. Manual checking may not be the best solution anyway. And more tests (even if partially run) seems better than fewer tests.
> 
> Checking in -expected.txt doesn't mean we can import less tests.  Also, we can import tests with FAIL expectations as well.  That's how we've been importing tests.

Agreed on FAIL expectations. -expected files should be kept for sure.

Checking in -expected.txt files does not guarantee that all tests are actually run though.
To guarantee that, one needs to carefully read each test.
This obviously takes time, hence less imported tests at the end of the day.

When importing tests, I would prefer adding the number of expected tests directly in the HTML file compared to counting the number of PASS in the expected file.
First, tests are more often read than expected files, giving more chances to catch errors in the number of tests.
Second, this may be pushed to the W3C repository.

Would you still object to the approach if the checking of the number of tests is made mandatory?

-- 
Configure bugmail: https://bugs.webkit.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.