[wpe-webkit] Webkit tests in WPE

[Joshua Watt]

On 7/17/19 10:15 AM, Michael Catanzaro wrote:
> On Wed, Jul 17, 2019 at 10:10 AM, Michael Catanzaro 
> <mcatanzaro at igalia.com> wrote:
>> Another problem I've seen before
>
> Also: timezone, some of the layout tests expect your system clock is 
> set to Pacific time, because of course they do. :)
>
> All issues that cause the tests to not work reliably across different 
> systems are bugs, of course. If you can find a way to make the tests 
> work more reliably, whether by adding dependencies to the jhbuild, 
> rewriting tests to avoid timezone problems, or however else, then of 
> course those fixes are appreciated. I think you've already noticed 
> this, but as a reminder, be sure to use the WPE WebKit component on 
> Bugzilla to make sure we see your patches and issues.

Thanks for the help. I was finally able to get a test run to finish by 
disabling coredumps on my system which prevented it from becoming 
unusable for hours while it tried to process all the crashes I was 
getting. In the end, I had 1845 tests fail. I dug around in the test 
results and tried to see if there was one or more common failure modes 
and found a few that seem to happen frequently. If anyone has any ideas 
on what these might be I'd appreciated it:

Several tests seem to have additional debugging output (or perhaps are 
trying to render error messages?) when either much less or no output is 
expected. For example:

--- /projects/wpe/WebKit/WebKitBuild/Release/layout-test-results/fast/block/crash-while-folding-anonymous-blocks-expected.txt
+++ /projects/wpe/WebKit/WebKitBuild/Release/layout-test-results/fast/block/crash-while-folding-anonymous-blocks-actual.txt
@@ -1 +1,15 @@
-
+layer at (0,0) size 800x600
+  RenderView at (0,0) size 800x600
+layer at (0,0) size 800x169
+  RenderBlock {html} at (0,0) size 800x169
+    RenderBody {body} at (8,16) size 784x137
+      RenderBlock {parsererror} at (16,0) size 752x137 [bgcolor=#FFDDDD] [border: (2px solid #CC7777)]
+        RenderBlock {h3} at (18,20) size 716x23
+          RenderText {#text} at (0,0) size 324x21
+            text run at (0,0) width 324: "This page contains the following errors:"
+        RenderBlock {div} at (18,61) size 716x15
+          RenderText {#text} at (0,0) size 385x14
+            text run at (0,0) width 385: "error on line 1 at column 10: AttValue: \" or ' expected"
+        RenderBlock {h3} at (18,94) size 716x23
+          RenderText {#text} at (0,0) size 429x21
+            text run at (0,0) width 429: "Below is a rendering of the page up to the first error."

There are many variations of that one with different error messages. 
There are also some where the look mostly correct, but the case of the 
tags is different (e.g. "html" when "HTML" was expected) and the pixels 
sizes are a little off.

I also had some fail with this strange message:

CONSOLE MESSAGE: line 8: ReferenceError: Can't find variable: fetch_tests_from_worker

I had a lot of tests timeout, almost always with messages like:

CONSOLE MESSAGE: line 17: TypeError: null is not an object (evaluating 'div.style')
FAIL: Timed out waiting for notifyDone to be called


Any help in sorting these out would be much appreciated.

> Michael
>
>