[webkit-dev] Testing changes to CodeGenerator*.pm
mjs at apple.com
Thu Apr 29 11:05:38 PDT 2010
On Apr 29, 2010, at 10:54 AM, Adam Barth wrote:
>> It would be great to have a tool that generates a diff of derived
>> for inspection, but making it into a test for everyone to maintain
>> like unnecessary burden. I certainly would feel bad about having to
>> a test that verifies source file content instead of behavior.
> You should feel free to develop a better testing harness. This one
> certainly isn't best conceivable tool, but it's better than what we
> had previously, which was essentially the C++ compiler.
> The maintenance is super easy. I've been doing a lot of development
> work on the code generator in the past few days, and it amounts to
> typing a single command:
> ./WebKitTools/Scripts/run-bindings-tests --reset-results
> The harness has been super useful in working on the code generator
> because the tests run in a few seconds. That lets me iterate on the
> script much more quickly compared to rebuilding the world every time I
> want to try a tweak.
It also strikes me as odd to do testing by doing exact comparison of
the generated source. But I can also see side benefits. I think the
real issue here may be one of naming. If the use model is that you
fully regenerate the results every time you make a change to the
bindings generator, then it's not really a regression test. The
purpose is not to catch unintentional changes but rather to be able to
observe changes to generated code, and new kinds of generated code,
while working on a change and when reviewing code. Perhaps the tool
should have a name that reflects that, instead of implying that the
purpose is to catch bugs accidentally introduced by changes. It
doesn't seem like an efficient or effective way to do the latter.
More information about the webkit-dev