[Webkit-unassigned] [Bug 155736] REGRESSION(r194294): string-base64 is 12% slower on MacBookPro

bugzilla-daemon at webkit.org bugzilla-daemon at webkit.org
Tue Mar 22 10:21:36 PDT 2016


https://bugs.webkit.org/show_bug.cgi?id=155736

--- Comment #1 from Mark Lam <mark.lam at apple.com> ---
I re-ran benchmarks on release build of r194293 vs r194294 on my MBP.  Here are the results:

Warning: could not identify checkout location for base64
Warning: could not identify checkout location for new64
260/260                                             
Generating benchmark report at /Volumes/Data/bench/rollout/base64_new64_SunSpider_Galeon_20160322_1013_report.txt
And raw data at /Volumes/Data/bench/rollout/base64_new64_SunSpider_Galeon_20160322_1013.json

Benchmark report for SunSpider on Galeon (MacBookPro11,3).

VMs tested:
"base64" at /Volumes/Data/bench/rollout/base64/jsc
"new64" at /Volumes/Data/bench/rollout/new64/jsc

Collected 4 samples per benchmark/VM, with 4 VM invocations per benchmark. Emitted a call to gc() between
sample measurements. Used 1 benchmark iteration per VM invocation for warm-up. Used the jsc-specific
preciseTime() function to get microsecond-level timing. Reporting benchmark execution times with 95%
confidence intervals in milliseconds.

                                    base64                    new64                                       

3d-cube                         4.7800+-0.0966     ?      5.2605+-1.4835        ? might be 1.1005x slower
3d-morph                        5.5892+-0.5420     ?      5.7764+-0.4662        ? might be 1.0335x slower
3d-raytrace                     5.2646+-0.0894     ?      5.6928+-0.8894        ? might be 1.0813x slower
access-binary-trees             2.3339+-0.3187            2.2958+-0.2749          might be 1.0166x faster
access-fannkuch                 5.9893+-0.3535            5.9674+-0.4265        
access-nbody                    2.6328+-0.0956     ?      2.7456+-0.3342        ? might be 1.0429x slower
access-nsieve                   3.1768+-0.1737            3.1740+-0.2197        
bitops-3bit-bits-in-byte        1.3389+-0.3069            1.2425+-0.1182          might be 1.0776x faster
bitops-bits-in-byte             3.4930+-0.6663            3.2468+-0.1380          might be 1.0758x faster
bitops-bitwise-and              2.0868+-0.0770            2.0593+-0.0483          might be 1.0134x faster
bitops-nsieve-bits              3.0917+-0.1299     ?      3.2731+-0.6106        ? might be 1.0587x slower
controlflow-recursive           2.3829+-0.1113     ?      2.4360+-0.2150        ? might be 1.0223x slower
crypto-aes                      4.3293+-0.1578     ?      4.5211+-0.7130        ? might be 1.0443x slower
crypto-md5                      2.5753+-0.0933     ?      2.5928+-0.1195        ?
crypto-sha1                     2.5049+-0.2026            2.4958+-0.2567        
date-format-tofte               7.3954+-0.2431     ?      7.7652+-1.2213        ? might be 1.0500x slower
date-format-xparb               5.3278+-0.8509            5.1008+-0.6973          might be 1.0445x faster
math-cordic                     3.1001+-0.3570     ?      3.1540+-0.2388        ? might be 1.0174x slower
math-partial-sums               5.1777+-0.4468            4.9267+-0.0397          might be 1.0509x faster
math-spectral-norm              2.0241+-0.1320     ?      2.0867+-0.1109        ? might be 1.0309x slower
regexp-dna                      6.2115+-0.1409     ?      6.2258+-0.1760        ?
string-base64                   4.3695+-0.4214     ?      4.6223+-0.2653        ? might be 1.0578x slower
string-fasta                    6.0775+-0.0788            5.9137+-0.1054          might be 1.0277x faster
string-tagcloud                 9.0709+-1.5893            8.4225+-0.5688          might be 1.0770x faster
string-unpack-code             19.2618+-1.3245     ?     19.8475+-1.0650        ? might be 1.0304x slower
string-validate-input           4.3025+-0.1291     ?      4.5067+-0.1637        ? might be 1.0475x slower

<arithmetic>                    4.7649+-0.0496     ?      4.8212+-0.0703        ? might be 1.0118x slower

Re-trying with LongSpider:

Warning: could not identify checkout location for base64
Warning: could not identify checkout location for new64
236/236                                              
Generating benchmark report at /Volumes/Data/bench/rollout/base64_new64_LongSpider_Galeon_20160322_1017_report.txt
And raw data at /Volumes/Data/bench/rollout/base64_new64_LongSpider_Galeon_20160322_1017.json

Benchmark report for LongSpider on Galeon (MacBookPro11,3).

VMs tested:
"base64" at /Volumes/Data/bench/rollout/base64/jsc
"new64" at /Volumes/Data/bench/rollout/new64/jsc

Collected 4 samples per benchmark/VM, with 4 VM invocations per benchmark. Emitted a call to gc() between
sample measurements. Used 1 benchmark iteration per VM invocation for warm-up. Used the jsc-specific
preciseTime() function to get microsecond-level timing. Reporting benchmark execution times with 95%
confidence intervals in milliseconds.

                                    base64                    new64                                       

3d-cube                       854.7654+-22.0568         849.9207+-7.4580        
3d-morph                     1550.3444+-8.5131         1541.2095+-6.7544        
3d-raytrace                   635.8143+-28.8163    ?    653.1796+-7.0825        ? might be 1.0273x slower
access-binary-trees           816.2689+-16.5012         813.3437+-13.3188       
access-fannkuch               294.5638+-14.4800    ?    294.9229+-11.6754       ?
access-nbody                  526.5207+-7.4685          523.8672+-7.1641        
access-nsieve                 373.6856+-24.2397         371.1917+-13.3476       
bitops-3bit-bits-in-byte       35.9461+-1.4671           35.7699+-1.0434        
bitops-bits-in-byte            79.0490+-2.2097     ?     82.4352+-6.4560        ? might be 1.0428x slower
bitops-nsieve-bits            415.1551+-6.1897     ?    422.8268+-10.8842       ? might be 1.0185x slower
controlflow-recursive         461.2465+-4.2071     ?    462.5665+-7.1707        ?
crypto-aes                    585.8578+-7.8133          580.9565+-7.9344        
crypto-md5                    559.8392+-4.2570          512.3588+-53.8504         might be 1.0927x faster
crypto-sha1                   701.6909+-16.3813         695.6992+-9.4330        
date-format-tofte             581.0825+-21.1241    ?    585.3035+-4.0404        ?
date-format-xparb             673.7775+-10.7687    ?    681.3560+-23.8795       ? might be 1.0112x slower
hash-map                      152.4008+-6.8498          152.1372+-2.6185        
math-cordic                   498.7078+-4.4980     ?    501.6669+-16.8703       ?
math-partial-sums             513.0204+-2.0620     ?    529.1973+-52.0186       ? might be 1.0315x slower
math-spectral-norm            567.6910+-6.1517     ?    579.0108+-13.7440       ? might be 1.0199x slower
string-base64                 375.1602+-3.3395     ?    378.3577+-13.4191       ?
string-fasta                  370.6249+-8.8332     ?    373.4370+-5.8656        ?
string-tagcloud               186.6574+-7.1007          184.3021+-4.8098          might be 1.0128x faster

<geometric>                   408.1578+-1.7735     ?    408.6597+-4.3770        ? might be 1.0012x slower


I also tried running on a MacPro.  Regardless of which machine, I am not seeing the regression.

-- 
You are receiving this mail because:
You are the assignee for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.webkit.org/pipermail/webkit-unassigned/attachments/20160322/cc5bc9c8/attachment-0001.html>


More information about the webkit-unassigned mailing list