[webkit-dev] run-bindings-tests

[Adam Barth]

2011/9/8 Oliver Hunt <oliver at apple.com>:
> On Sep 8, 2011, at 7:21 PM, Alexey Proskuryakov wrote:
>> 08.09.2011, в 12:25, Darin Adler написал(а):
>>> I find the bindings tests quite helpful. Because the perl script is so hard to read, it’s the changes in bindings script test results that I look at when reviewing changes to the bindings scripts. The fact that the results are checked in helps me review patches.
>>
>> OK, then they are valuable indeed.
>>
>> However, I still feel that there is a disconnect between the desired effect (provide a diff in a patch for review) and the implementation (tests that can pass or fail). This also puts the burden of maintaining the results on people who needn't care about them - for example, Oliver's patch clearly didn't need someone look over generated code changes.
>
> I think the argument is that it _did_ need someone -> the reviewer didn't have any "nice" way to see the difference in output that would have been visible had i included updates to the expected output.
>
> My problem with the test is that it isn't run as part of run-webkit-tests (which is what we say you must run), the test output is fairly awful, and the test script doesn't support --help, or --reset (it turns out it does have an equivalent to --reset, but why use a different argument in one tool from what we use in the main one?)

Those all sound like very fixable issues.  I'm sorry I picked the
wrong flag name.  I was trying to copy the name of the flag used by
run-webkit-tests, but I must have screwed it up somehow.

If you'd be willing to file bugs about the improvements you'd like to
see in the test, I'll be happy to make them.  I'm not quite up to the
task of making run-webkit-tests run all the various tests, but
hopefully someone will volunteer to make that happen.

Adam


>> I'm not sure what the better solution would be though. Perhaps a bot could provide a diff of DerivedSources for any patch that touches code generators, but I'm not volunteering to implement one :-)