[webkit-dev] Some text about the B3 compiler

[Filip Pizlo]

> On Feb 2, 2016, at 4:56 PM, Carlos Alberto Lopez Perez <clopez at igalia.com> wrote:
> 
> On 02/02/16 19:58, Ryosuke Niwa wrote:
>> On Tue, Feb 2, 2016 at 10:42 AM, Carlos Alberto Lopez Perez
>>> But this script seems focused on comparing the performance between
>>> different browsers (safari vs chrome vs firefox) rather than in testing
>>> and comparing the performance between different revisions of WebKit.
>> 
>> Not at all.  It simply supports running benchmark in other browsers.
>> 
>>> Do you think it makes any difference (from the point of view of
>>> detecting failures, not from the performance PoV) to run this tests in a
>>> full-fledged browser like Safari rather than in WebKitTestRunner ?
>> 
>> Yes. There are many browser features that can significantly impact the
>> real world performance.
>> 
> 
> I'm specifically not asking about performance, but about correctness.
> 
> This discussion was started because Filip said that running JS tests on
> a browser catches many failures that are not cached when running the
> tests from the terminal.
> 
> So, I'm wondering if running the JS tests on WTR or Safari makes any
> difference when catching failures.

I suspect that the browser will catch more failures.

> 
>>> We already have a performance test bot running tests inside WTR.
>>> And I see that the current set of tests executed on this bot already
>>> includes Speedometer, and that JetStream and Sunspider are skipped on
>>> PerformanceTests/Skipped.
>>> 
>>> So I see some options going forward:
>>> 
>>> - Fix the JetStream and Sunspider tests so they can be run as part of
>>> the current run-perf-tests script that the performance bots execute.
>> 
>> We should use run-benchmark instead since run-benchmark spits out the
>> JSON file that's compatible with run-pref-tests.
>> 
> 
> I'm a bit lost here. Are you planning to deprecate run-perf-tests with
> run-benchmark? What is wrong with run-perf-tests?
> 
>>> - Implement support on the script run-benchmark to run the tests inside
>>> WTR, and create a new step running this script that will be executed on
>>> the test bots.
>> 
>> I don't see a point in doing this.   Why is it desirable to run these
>> benchmarks inside WebKitTestRunner?
>> 
> 
> Less dependencies: WTR (or the MiniBrowser) is something that is
> currently built by the bots on each build.
> If we want to use Epiphany (for example) for the performance tests, is
> another thing we have to take care of building before each run. Not a
> big deal, but I wonder if is really worth.
> 
>>> - Deploy a new bot that runs run-perf-tests on a full-fledged browser
>>> like Safari or Epiphany.
>> 
>> We should just do this.
>> 
>>> I wonder what you think is the best option or if there is some option
>>> not viable.
>>> 
>>> From my PoV, the option #1 has the advantage of reusing the current
>>> infrastructure that collects and draws performance data at
>>> https://perf.webkit.org
>> 
>> We have an internal instance of the same dashboard to which we're
>> reporting results of run-benchmark script.
>> 
> 
> What about making this public? We will happily contribute with a
> GTK+/Linux buildbot for it.
> 
> 
> Regards.
> 
> _______________________________________________
> webkit-dev mailing list
> webkit-dev at lists.webkit.org
> https://lists.webkit.org/mailman/listinfo/webkit-dev