[webkit-dev] Importing W3C tests to webkit
jacobg at adobe.com
Wed May 23 13:39:07 PDT 2012
Dirk, my apologies, I was on travel the week you replied and missed your
message. I found it and will review / update now.
On 5/23/12 1:25 PM, "Dirk Pranke" <dpranke at chromium.org> wrote:
>On Wed, May 23, 2012 at 11:56 AM, Ryosuke Niwa <rniwa at webkit.org> wrote:
>> As I have said in the past, we should just import all tests, and treat
>> non-text, non-ref tests as pixel tests. If we wanted to reduce the
>> pixel tests we import, then we should submit those patches to W3C
>> directly submitting them to WebKit.
>> In general, I don't buy the argument that adding more pixel tests incurs
>> more cost than the benefit we get from importing the tests. If that
>> case, we can just get rid of the existing pixel tests we have.
>Not so: if the tests we had had 100% coverage, then importing more
>tests would buy us nothing, but getting rid of the existing tests
>would be quite unfortunate. Clearly adding tests incurs some costs and
>probably provides some benefit; the question is when does the cost
>exceed the benefit?
>As I am not against importing more tests per se, I think this only
>makes sense to evaluate on a case-by-case basis.
>> The only sane argument I've heard so far to gate pixel tests is that the
>> correctness of such tests need to be manually inspected, which requires
>> lot of manual labor and is very error prone.
>I'm assuming the above includes the ongoing maintenance cost of
>keeping pixel tests up to date, as well as the cost at the initial
>There is also the fact that the more tests we have, the more tests we
>have to run, and increasing cycle time by itself is a cost to
>developer productivity. Of course, it's also potentially the case that
>we have to update tests from time to time, although this doesn't
>Jacob, I gave you a bunch of feedback not long after you published the
>initial writeup, but it doesn't look like any of that has been
More information about the webkit-dev