[webkit-qt] A quick look at LayoutTests (more specifically, http/tests/ and media/)

[Árvai Zoltán]

Hi Jesus,

I opened tickets for this topic:
  https://bugs.webkit.org/show_bug.cgi?id=71911
  https://bugs.webkit.org/show_bug.cgi?id=71914

I will deal with this issue, soon.

br,
Zoltan

2011-11-08 14:17, Jesus Sanchez-Palencia wrote:
> Hi folks,
>
> Today I spent some time taking a look at the status of our Layout 
> Tests. After taking a look at the Skipped lists I decided to give it a 
> try to two largely skipped folders: http/tests/  and media/ .
>
> I used:   "run-webkit-tests --no-launch-safari --no-sample-on-timeout 
> --no-new-test-results --results-directory layout-test-results 
> --use-remote-links-to-tests --release --qt --force FOLDER", which was 
> copied from the Szeged bots.
>
> I ran the tests for WebKit1 (6582a115083311f52297cde636c82c96c97f9f29) 
> with Qt 4.8 (6582a115083311f52297cde636c82c96c97f9f29) and for WebKit2 
> with Qt5 (6df6e99be09731991a1689cf1a71c601f4814865). For WK2 I ran it 
> on 2 different machines, running different Linux distros.
>
> Perhaps I'm missing something or I did something wrong but, without 
> _any_ fix to WebKit nor Qt, and for my surprise, I manage to get a 
> reasonable amount of "Expected to fail, but passed" tests:
>
> - media/ WK2: ~110 of "unexpected-passing" tests.
> - http/tests/ WK2: ~120 of "unexpected-passing" tests.
> - media/ WK1: 68 "unexpected-passing" tests.
> - http/tests/ WK1: 45 "unexpected-passing" tests.
>
> Which, according to Ossy, are Skipped tests that were marked as fail, 
> but that can be passing now.
> Here you can find the list of unexpected-passing tests for WK2: 
> http://pastebin.com/zBGHcYa3
> Here you can find the list of unexpected-passing tests for WK1: 
> http://pastebin.com/ZyMpK2T1
>
> Since I don't have access to a proper buildbot environment, I wonder 
> if it wouldn't be possible for the Szeged folks to take a look at this 
> list and try to unskip these tests. Perhaps just running the same 
> command as I did might spot a certain amount of tests that could be 
> unskipped right now. I would be surprise if there isn't at least 10 
> tests among these ~200 that are passing already, not to say more...
>
> Moreover, I believe this is a good way to easily spot tests that could 
> be passing; Just running the tests with --force to a certain folder 
> and then cross-checking the output list marked as "Expected to fail, 
> but passed". I was told this is the "Skipped list reducer week at 
> Szeged", so I hope you guys can use this somehow.
>
> Cheers,
> jesus
>
>
>
>
>
>
> _______________________________________________
> webkit-qt mailing list
> webkit-qt at lists.webkit.org
> http://lists.webkit.org/mailman/listinfo.cgi/webkit-qt

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-qt/attachments/20111109/c78053e2/attachment.html>