[webkit-dev] Testing changes to CodeGenerator*.pm

[Maciej Stachowiak]

On Apr 29, 2010, at 10:54 AM, Adam Barth wrote:

>
>> It would be great to have a tool that generates a diff of derived  
>> sources
>> for inspection, but making it into a test for everyone to maintain  
>> feels
>> like unnecessary burden. I certainly would feel bad about having to  
>> maintain
>> a test that verifies source file content instead of behavior.
>
> You should feel free to develop a better testing harness.  This one
> certainly isn't best conceivable tool, but it's better than what we
> had previously, which was essentially the C++ compiler.
>
> The maintenance is super easy.  I've been doing a lot of development
> work on the code generator in the past few days, and it amounts to
> typing a single command:
>
> ./WebKitTools/Scripts/run-bindings-tests --reset-results
>
> The harness has been super useful in working on the code generator
> because the tests run in a few seconds.  That lets me iterate on the
> script much more quickly compared to rebuilding the world every time I
> want to try a tweak.

It also strikes me as odd to do testing by doing exact comparison of  
the generated source. But I can also see side benefits. I think the  
real issue here may be one of naming. If the use model is that you  
fully regenerate the results every time you make a change to the  
bindings generator, then it's not really a regression test. The  
purpose is not to catch unintentional changes but rather to be able to  
observe changes to generated code, and new kinds of generated code,  
while working on a change and when reviewing code. Perhaps the tool  
should have a name that reflects that, instead of implying that the  
purpose is to catch bugs accidentally introduced by changes. It  
doesn't seem like an efficient or effective way to do the latter.

Regards,
Maciej