[Webkit-unassigned] [Bug 185783] test262/Runner.pm: add unit tests

[bugzilla-daemon at webkit.org]

https://bugs.webkit.org/show_bug.cgi?id=185783

--- Comment #37 from Leo Balter <leonardo.balter at gmail.com> ---


> I do not think tests are the appropriate way to enforce this compatibility. End-to-end buildbot tests would, but this may not be feasible. You will need an English explanation somewhere in either the test runner or unit tests (or both) to explain that the formatting of the output must never change without changing buildbot parsing code. Otherwise, the tests themselves only assert the output was compatible with the output buildbot expected as of the some date. Lack of such English could lead to someone changing the output format AND the tests only to cause Buildbot breakage once a WebKit.org admin pushes a change to Builbot.

> If we are still interested in pursuing tests for the output then can we at least mock out the calls to JSC so that we do not need to have a dependency on it?

In that way, I would need a definition of the desirable mocking. The current patch has some mocking tests - as a small subset clones from Test262 tests. The tests for the buildbot mocks the output itself. I wonder if we could tackle this progressively, considering the density and all friction found here in this issue. The initial idea was to bring a starter for the unit tests, it seems we are going beyond now and this could only cause more friction.

> > > If the former, then I suggest we make more use of webkitdirs.pm as it will simplify the code and make it more consistent with our other Perl code. 
> 
> > I sent a patch for this already and it got rejected. We also considered it
> > would be a unwelcome burden having to tell everyone to keep consistency for
> > it with such an old version of Perl as it is not directly necessary for this
> > script.


> Do you have a bug # for this?

https://bugs.webkit.org/show_bug.cgi?id=185316

-- 
You are receiving this mail because:
You are on the CC list for the bug.
You are the assignee for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-unassigned/attachments/20180529/e5af6fd2/attachment.html>