[webkit-dev] can we stop using Skipped files?

[Dirk Pranke]

Hi Ossy,

Thanks for your reply ...

On Fri, Jun 8, 2012 at 12:46 AM, Osztrogonac Csaba <oszi at inf.u-szeged.hu> wrote:
> Hi,
>
> Dirk Pranke írta:
>
>> I believe most if not all of the ports have started using either
>> TestExpectations files or a combination of TestExpectations files
>> (except for the Apple Win port).
>>
>> Can we explicitly switch to the TestExpectations files at this point
>> and drop support for Skipped files on the other ports (and perhaps
>> disable old-run-webkit-tests for all but apple win)?
>
>
> Until NRWT can't handle cascaded TestExpectations -
> https://bugs.webkit.org/show_bug.cgi?id=65834,
> Qt port can't drop supporting Skipped files. We have many tests skipped in
> qt-5.0, qt-5.0-wk1,
> qt-5.0-wk2, wk2 Skipped lists. We can't migrate all of them to the only one
> TestExpectations.
>

Sorry, you're right, I should have mentioned that we would need to
support cascaded files before we can do anything. As Ojan mentioned, I
am actively working on that.

One thing that would be helpful would be to how each port would like
to support using cascading files and modifiers. Assuming each of you
is free to do as you like, you can pick from several paths, e.g. you
could use one file, or one file per operating system, or one file per
os plus one file per version plus maybe a webkit-2 specific file
(roughly what most ports do today, I think), or something else?


> And I disagree with disabling ORWT at all. Qt port still support using ORWT
> locally.
> It is better for gardening than NRWT. NRWT regularly has problems with
> generating
> new results for a given platform dir (qt,qt-5.0,qt-5.0-wk1,...), it doesn't
> support
> the good --skipped=only option .

I am not aware of these issues ... can you explain further, or file
bugs? In particular, I'm not sure I fully understand how all of the
different Qt version/platform combinations work, so maybe there are
issues there that can be improved?

If ORWT is easier for you to use than NRWT, I would love to understand
why, so that hopefully I can improve NRWT for you as well.

> If folks don't want to use it, just not use, but disabling for everyone by fiat isn't a friendly thing.

I'm sorry, I was under the impression that no one (apart from Apple
Win) was using ORWT any more, and I thought we had reached feature
parity between the two tools (or at least that NRWT supported
everything that ORWT did, obviously ORWT doesn't support reftests or a
bunch of other things that NRWT does).

I'm not about to remove a tool that people are still using :). That
said, having two tools that are both widely used and do essentially
the same thing is unfortunate, so it would be good to converge
where/when/if we can, I think.

-- Dirk