[Webkit-unassigned] [Bug 185783] test262/Runner.pm: add unit tests

[bugzilla-daemon at webkit.org]

https://bugs.webkit.org/show_bug.cgi?id=185783

--- Comment #35 from Leo Balter <leonardo.balter at gmail.com> ---
As Valerie mentioned, I'm taking over this patch as she's in a vacation time.

> It is still beneficial to add Buildbot unit tests to catch parsing regressions in how we "surface those errors" in the Buildbot UI.

The unit tests for the buildbot contains only tests for how the buildbot will interpret the results, isolating from anything else, including the results from the test262-runner. 
It contains a static provided expectation text and asserts how the buildbot script read it.

This is for me an important reason to maintain tests for the runner that asserts the output will be compatible with the buildbot, but I'm not sure why they need to be in the same place. I can do it anyway if you prefer.

> You wrote "had", the past tense of "have". Was this intentional and you are expressing that there is no longer a strong motivation to be stand alone and we can use more of webkitdirs.pm? Or was this accidental and the tests262-runner still has a strong motivation to be stand alone?

We still have a motivation but Valerie did some throughout workarounds to minimize the incompatibility.

This has been discussed through other patches and I'm afraid there are a few details I can't disclose here. Valerie already mentioned we need to keep it compatible with Perl 5.8.8 and the webkitdirs.pm is not, so it's hard to rely on it.

> If the former, then I suggest we make more use of webkitdirs.pm as it will simplify the code and make it more consistent with our other Perl code. 

I sent a patch for this already and it got rejected. We also considered it would be a unwelcome burden having to tell everyone to keep consistency for it with such an old version of Perl as it is not directly necessary for this script.

> If the tests262-runner is still motivated to be stand alone then how did you come to the decision to incorporate the tests for it into the same directory hierarchy as the unit tests for all of our shared Perl code and use test-webkitperl to run them. I mean, if there is a strong motivation for the test262-runner to be stand alone then why isn't there a strong motivation to have the unit test runner for the test262-runner be stand alone?

The decision was made to keep consistency over the other unit tests. I believe some of this also received feedback in this same patch issue.

This was finally decided through patches discussions and internal discussions with Filip Pizlo and Michael Saboff. Do you think this is worth going all the way back as a time effort considering where we the tests for the runner should be placed? I honestly don't mind of any place, but I feel I can't decide myself.

-- 
You are receiving this mail because:
You are on the CC list for the bug.
You are the assignee for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-unassigned/attachments/20180529/7cf2ac10/attachment.html>