[webkit-dev] Some text about the B3 compiler

[Carlos Alberto Lopez Perez]

On 02/02/16 00:27, Ryosuke Niwa wrote:
> On Mon, Feb 1, 2016 at 7:32 AM, Carlos Alberto Lopez Perez
> <clopez at igalia.com> wrote:
>> On 31/01/16 05:16, Filip Pizlo wrote:
>>> As my original message said, I was wondering if there was any support
>>> for running some JavaScript tests *in browser*.  run-jsc-stress-tests
>>> doesn’t support that because it doesn’t know what a browser is.
>>>
>>> Some tests, like Kraken, Octane, JetStream, and Speedometer, either
>>> require a browser to run (like JetStream and Speedometer) or have
>>> significantly different behavior in the browser than in their
>>> command-line harnesses (like Kraken and Octane).  If you did have a
>>> bot that ran these tests in some GTK+ or EFL browser, you’d probably
>>> catch bugs that testing the JSC shell cannot catch.
>>>
>>> -Filip
>>
>> Some questions regarding this JSC in-browser benchmarks:
>>
>> How does the Apple port runs this? Do you use some script that is
>> currently available on the WebKit source tree? Are the buildbots running
>> this tests public?
> 
> We've been running those benchmarks with
> http://trac.webkit.org/browser/trunk/Tools/Scripts/run-benchmark
> 

I see.

But this script seems focused on comparing the performance between
different browsers (safari vs chrome vs firefox) rather than in testing
and comparing the performance between different revisions of WebKit.

Do you think it makes any difference (from the point of view of
detecting failures, not from the performance PoV) to run this tests in a
full-fledged browser like Safari rather than in WebKitTestRunner ?

We already have a performance test bot running tests inside WTR.
And I see that the current set of tests executed on this bot already
includes Speedometer, and that JetStream and Sunspider are skipped on
PerformanceTests/Skipped.

So I see some options going forward:

 - Fix the JetStream and Sunspider tests so they can be run as part of
the current run-perf-tests script that the performance bots execute.

 - Implement support on the script run-benchmark to run the tests inside
WTR, and create a new step running this script that will be executed on
the test bots.

 - Deploy a new bot that runs run-perf-tests on a full-fledged browser
like Safari or Epiphany.

I wonder what you think is the best option or if there is some option
not viable.

From my PoV, the option #1 has the advantage of reusing the current
infrastructure that collects and draws performance data at
https://perf.webkit.org

> It might be useful for Mac and other ports to have bots available on
> build.webkit.org as well.
> 

Indeed.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 883 bytes
Desc: OpenPGP digital signature
URL: <https://lists.webkit.org/pipermail/webkit-dev/attachments/20160202/bcc78820/attachment.sig>