[webkit-dev] Testing changes to CodeGenerator*.pm

[Adam Barth]

On Thu, Apr 29, 2010 at 10:27 AM, Jeremy Orlow <jorlow at chromium.org> wrote:
> On Thu, Apr 29, 2010 at 5:44 PM, Alexey Proskuryakov <ap at webkit.org> wrote:
>> On 26.04.2010, at 22:06, Adam Barth wrote:
>>> If you make changes to CodeGenerator*.pm, please test your
>>> change using the following command:
>>>
>>> ~/git/webkit$ ./WebKitTools/Scripts/run-bindings-tests
>>
>>
>> I don't understand the utility of such testing. When one edits
>> CodeGenerator*.pm, that's generally to change its output, so
>> run-bindings-tests tests will of course fail.
>>
>> It's great to test end-to-end behavior, and unit tests can also also
>> useful sometimes, but why test that source code stays byte to byte
>> identical?
>
> When you make a change to the code generator, you should make
> a corresponding change to the generated test code.  This allows the reviewer
> to see how the change in CodeGenerator*.pm affects the generated code and
> documents the change in SVN.  The hope is that it'll be obvious if your
> change is going to have unintended consequences on the generated code.
>  We've found this useful when working on the V8 bindings.

Yeah, without seeing the consequence on the generated code, it can be
very difficult to review changes to the code generator.  Imagine if
you were only able to review patches to WebCore by looking at the
LayoutTests.

The tests have already let us find and fix a handful of bugs and we've
only had them for three days.

Adam