[webkit-dev] can we stop using Skipped files?

[Filip Pizlo]

On Jun 8, 2012, at 9:52 AM, Ojan Vafai wrote:

> On Fri, Jun 8, 2012 at 9:25 AM, Filip Pizlo <fpizlo at apple.com> wrote:
> 
> On Jun 8, 2012, at 9:16 AM, Balazs Kelemen wrote:
> 
> > On 06/08/2012 05:21 PM, Filip Pizlo wrote:
> >> On Jun 8, 2012, at 4:38 AM, Balazs Kelemen<kbalazs at webkit.org>  wrote:
> >>
> >>> On 06/08/2012 09:46 AM, Osztrogonac Csaba wrote:
> >>>> Hi,
> >>>>
> >>>> Dirk Pranke írta:
> >>>>> I believe most if not all of the ports have started using either
> >>>>> TestExpectations files or a combination of TestExpectations files
> >>>>> (except for the Apple Win port).
> >>>>>
> >>>>> Can we explicitly switch to the TestExpectations files at this point
> >>>>> and drop support for Skipped files on the other ports (and perhaps
> >>>>> disable old-run-webkit-tests for all but apple win)?
> >>>> Until NRWT can't handle cascaded TestExpectations - https://bugs.webkit.org/show_bug.cgi?id=65834,
> >>>> Qt port can't drop supporting Skipped files. We have many tests skipped in qt-5.0, qt-5.0-wk1,
> >>>> qt-5.0-wk2, wk2 Skipped lists. We can't migrate all of them to the only one TestExpectations.
> >>>>
> >>>> And I disagree with disabling ORWT at all. Qt port still support using ORWT locally.
> >>>> It is better for gardening than NRWT. NRWT regularly has problems with generating
> >>>> new results for a given platform dir (qt,qt-5.0,qt-5.0-wk1,...), it doesn't support
> >>>> the good --skipped=only option . If folks don't want to use it, just not use, but
> >>>> disabling for everyone by fiat isn't a friendly thing.
> >>> 1. These are real weaknesses of nrwt, we should fix them. If gardening is better with orwt (i doubt that is the case, but I don't do gardening regularly), we should improve nrwt, i.e. reimplement features from orwt.
> >> I applaud your enthusiasm.
> >>
> >>> 2. I believe basically everybody agrees that we should drop orwt, except you Ossy. Maybe I'm wrong. So, is there anybody still want to have support for orwt? If so, why?
> >> I'm with Ossy on this.
> >>
> >> Getting rid of ORWT would be a show stopper for me.
> >>
> >> This talk of fixing bugs in NRWT is really great but until such time as those bugs are fixed, let's keep ORWT.
> >>
> >
> > Understandable. Let me make it clear: I don't prefer one over the other. I believe it's contra-productive that some people/bots use this, and others use that. It adds overhead to bot maintainance, it's bad for developer experience, and it blocks the evolution of the one and only tool - because some people still make efforts on fixing/improving the other instead.
> 
> I think one issue that ought to be fixed with NRWT is the level of infrastructural complexity that it has, and the extent to which its algorithmic design (say, load balancing technique) is ossified through the use of entirely unnecessary object oriented pattern overhead.
> 
> In short, NRWT is overengineered.
> 
> As such, there will be those of us, which prefer solutions that are not overengineered, and who thus end up hacking ORWT when we need to get work done.
> 
> Do you really do quick local hacks to ORWT? Or are you talking about adding actual features to ORWT?

Both.

> 
> While NRWT is more code than ORWT, it's also thoroughly unittested. It's a little harder to dive into, but it's much easier to maintain IMO.

It's a lot harder to dive into, a lot more cumbersome to improve, and not any easier to maintain.

The fact that it is unittested is part of the problem.  It restricts what you can do to the interfaces used internally in the code, which makes larger changes much harder to pull off.  It also steepens the learning curve.

Look, the test harness should be a thing that helps us get work done, not an end unto itself.  The harness should err on the side of understandability, hackability, and, in short, simplicity.

> 
> The complexity in the codebase is partially because it was written before webkitpy existed and then ported to work on top of webkitpy. So there is still cleanup that could happen to make it less complex. But the specific complexity you're talking about for the parallelism was necessary to make NRWT use multiple processes, which itself was necessary because of Python multithreading bugs.

I look forward to seeing what such a cleanup would look like.

Python feels like the wrong technology to use for multithreading.  I'm curious if, in the future, next time we rewrite RWT (and I do believe that there will be such a "next time"), we could pick a language that allows this more naturally.  It's sad that most languages have bugs in either the library code for starting and managing processes (Java, I'm glaring at you!) or bugs in threads (as you say, Python, and in my experience, Ruby and Perl, and probably others as well).

-F
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20120608/33a6910f/attachment.html>