[Webkit-unassigned] [Bug 160057] New: Benchmark test content-animation timeouts always

[bugzilla-daemon at webkit.org]

https://bugs.webkit.org/show_bug.cgi?id=160057

            Bug ID: 160057
           Summary: Benchmark test content-animation timeouts always
    Classification: Unclassified
           Product: WebKit
           Version: WebKit Nightly Build
          Hardware: Unspecified
                OS: Unspecified
            Status: NEW
          Severity: Normal
          Priority: P2
         Component: Tools / Tests
          Assignee: webkit-unassigned at lists.webkit.org
          Reporter: clopez at igalia.com
                CC: bugs-noreply at webkitgtk.org, lforschler at apple.com,
                    rniwa at webkit.org, sam at webkit.org,
                    simon.fraser at apple.com, slewis at apple.com

The benchmark test content-animation added in https://trac.webkit.org/changeset/199805 timeouts.

The GTK+ performance bots is running all the benchmark tests plans, and this is one is constantly timing out. Check: https://build.webkit.org/builders/GTK%20Linux%2064-bit%20Release%20%28Perf%29 It uses the "--allplans" command line switch that runs all *.plan one after other.

I tried running it on the GTK+ platform:

$ Tools/Scripts/run-benchmark --platform gtk --browser minibrowser --plan content-animation

and it timeouts

I tried running it also on Mac platform:

$ Tools/Scripts/run-benchmark --platform osx --browser safari --plan content-animation

and it also timeouts.

... looking at the changelog of r199805 it says:

"""
When run as a benchmark, test completion involves a couple of bounces between the test and MobileSafari.
Test completion is initiated by a setTimeout() in the test, at which point location.hash is set to "#done".
MobileSafari detects that, and asynchronously dumps performance data. Once this is complete, MobileSafari
changes the URL has to "#submit", which triggers the XHR to the test relay that indicates the test is complete.
"""

So, if I understand things correctly, this tests depends on some black-magic that only happens on MobileSafari to work?

This is very unfortunate.

I think we should either:

Option 1) Fix this test so it can run with any browser.
Option 2) Add a way to skip test plans by platform/browser or make some tests to only run on some very specific platform/browser.

Opinions?

-- 
You are receiving this mail because:
You are the assignee for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.webkit.org/pipermail/webkit-unassigned/attachments/20160721/643a4329/attachment.html>