[webkit-dev] can we stop using Skipped files?

[Ojan Vafai]

On Sun, Jun 10, 2012 at 4:54 AM, Balazs Kelemen <kbalazs at webkit.org> wrote:
>
>   So the unit tests are superfluous.  In particular, if I had to pick
>> between only having unit tests or only having regression tests, I might
>> pick unit tests.  But if I already have regression tests then I'm unlikely
>> to want to incur technical debt to build unit tests, particularly since
>> unit tests requiring changing the infrastructure to make the code more
>> testable, which then leads to the problems listed above.
>>
>
>  There are many code paths are used rarely. In practice, we were having
> regressions frequently when people modified the code. Since the codebase
> has been unittested, the rate of regressions has gone down considerably.
> The time you spend dealing with tests is considerably less than the time
> you spend rolling patches in an out as you encounter different edge cases
> that different configurations/flags hit.
>
>
>
> A quick note to unittests. I think it's easy to define a hard limit for
> unittests, which is that: if I want to add a feature, or some customizing
> option for a particular port, it should be less effort to write the
> unittest than to write the actual code. I heard from my colleges a few
> times that it's not always the case with nrwt. I can imagine that it's not
> trivial to setup the unittest system for a module that has not been
> unittested so far but I think it should rather be the job of those who are
> actively working on the test harness, not of those who just need some work
> to be done for their port.
>

While this is a nice ideal to strive for, I don't think this ever plays out
for testing on any project, e.g. it is very frequently harder to write
tests for my WebCore changes than to make the change itself. Certainly
anything we can do to make testing easier is better, but I don't see NRWT
as more difficult to test than any other code in the WebKit project.

WebKit has a policy of every change requiring tests. I don't see why
tooling should be any different. It's unfortunate that NRWT started with 0
tests, so there are still (very few now!) parts that aren't tested. It's
hard to test those parts if that's what your modifying. However, it's
*especially* for the cases of port-specific code that need testing. Those
are exactly the codepaths that break from lack of testing.

Untested code is inherently harder to maintain in my experience. Most of
the time, committing untested code is just implanting technical debt that
someone will have to pay later.

There's noone whose fulltime job is it to maintain WebKit tooling. It's a
shared effort supported by everyone in the project.

It's long been a part of WebKit culture as long as I've been involved in
the project that sometimes you get stuck reworking code you didn't write in
order to make your change because your change just barely pushes the
existing messy code over the edge of too messy. It's true that it makes
your simple change much harder, but it's better for the project overall,
even if occasionally a patch doesn't get checked in because the associated
work was more than the contributor was willing to do.

Ojan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20120610/e515243f/attachment.html>