[webkit-dev] A question about new-run-webkit-tests

[Maciej Stachowiak]

On Mar 18, 2011, at 2:22 PM, Dirk Pranke wrote:

> In short, what Adam just said :)
> 
> In long(-er),
> 
> Your annoyance is quite understandable. I won't go into the reasons
> for the delay, but the major technical reason has been fixed, finally.
> These are the issues that I am aware of that remain:
> 
> * There are GTK bots that run NRWT as well as the Chromium bots
> * I have been in conversations a bit w/ wms and lforschler to get the
> apple mac port using it
> * There are a couple of known minor bugs with the mac port
> * There are some known minor features missing
> * There are known issues with the apple win port not working (this is
> probably the biggest issue keeping things from working, but it
> shouldn't be too hard to surmount).
> 
> I hope to make rapid progress on all of these fronts, and pull
> together all of the others issues into a single fairly easily
> trackable spot.
> 
> I would like to be able to either say we're fully cut over or at least
> give a good update at the meeting in April.

Here's some of the issues I consider critical to address before the project can switch over (besides the additional test failures and flakiness):

"new-run-webkit-tests output is different from (and worse than) original run-webkit-tests"
<https://bugs.webkit.org/show_bug.cgi?id=37736>

"new-run-webkit-tests does not report stderr output"
<https://bugs.webkit.org/show_bug.cgi?id=37739>

"new-run-webkit-tests should upload crash logs"
<https://bugs.webkit.org/show_bug.cgi?id=55907>

"new-run-webkit-tests low timeout is too sensitive to runaway processes or test crashes"
<https://bugs.webkit.org/show_bug.cgi?id=37738>

"new-run-webkit-tests doesn't support WebKit2"
<https://bugs.webkit.org/show_bug.cgi?id=56729>

"new-run-webkit-tests fails on Lion seed"
<https://bugs.webkit.org/show_bug.cgi?id=56730>

"new-run-webkit-tests doesn't support sample-on-timeout"
<https://bugs.webkit.org/show_bug.cgi?id=56731>


Regards,
Maciej