[webkit-dev] Some text about the B3 compiler
rniwa at webkit.org
Tue Feb 2 10:58:10 PST 2016
On Tue, Feb 2, 2016 at 10:42 AM, Carlos Alberto Lopez Perez
<clopez at igalia.com> wrote:
> On 02/02/16 00:27,
- R. Niwa
>> On Mon, Feb 1, 2016 at 7:32 AM, Carlos Alberto Lopez Perez
>> <clopez at igalia.com> wrote:
>>> On 31/01/16 05:16, Filip Pizlo wrote:
>>>> As my original message said, I was wondering if there was any support
>>>> doesn’t support that because it doesn’t know what a browser is.
>>>> Some tests, like Kraken, Octane, JetStream, and Speedometer, either
>>>> require a browser to run (like JetStream and Speedometer) or have
>>>> significantly different behavior in the browser than in their
>>>> command-line harnesses (like Kraken and Octane). If you did have a
>>>> bot that ran these tests in some GTK+ or EFL browser, you’d probably
>>>> catch bugs that testing the JSC shell cannot catch.
>>> Some questions regarding this JSC in-browser benchmarks:
>>> How does the Apple port runs this? Do you use some script that is
>>> currently available on the WebKit source tree? Are the buildbots running
>>> this tests public?
>> We've been running those benchmarks with
> I see.
> But this script seems focused on comparing the performance between
> different browsers (safari vs chrome vs firefox) rather than in testing
> and comparing the performance between different revisions of WebKit.
Not at all. It simply supports running benchmark in other browsers.
> Do you think it makes any difference (from the point of view of
> detecting failures, not from the performance PoV) to run this tests in a
> full-fledged browser like Safari rather than in WebKitTestRunner ?
Yes. There are many browser features that can significantly impact the
real world performance.
> We already have a performance test bot running tests inside WTR.
> And I see that the current set of tests executed on this bot already
> includes Speedometer, and that JetStream and Sunspider are skipped on
> So I see some options going forward:
> - Fix the JetStream and Sunspider tests so they can be run as part of
> the current run-perf-tests script that the performance bots execute.
We should use run-benchmark instead since run-benchmark spits out the
JSON file that's compatible with run-pref-tests.
> - Implement support on the script run-benchmark to run the tests inside
> WTR, and create a new step running this script that will be executed on
> the test bots.
I don't see a point in doing this. Why is it desirable to run these
benchmarks inside WebKitTestRunner?
> - Deploy a new bot that runs run-perf-tests on a full-fledged browser
> like Safari or Epiphany.
We should just do this.
> I wonder what you think is the best option or if there is some option
> not viable.
> From my PoV, the option #1 has the advantage of reusing the current
> infrastructure that collects and draws performance data at
We have an internal instance of the same dashboard to which we're
reporting results of run-benchmark script.
- R. Niwa
More information about the webkit-dev