[webkit-changes] [WebKit/WebKit] 5c9edc: Add a category-breakdown to compare-results

[Keith Miller]

  Branch: refs/heads/main
  Home:   https://github.com/WebKit/WebKit
  Commit: 5c9edc56375937ca181745a48c0a7e5fa2a95b7a
      https://github.com/WebKit/WebKit/commit/5c9edc56375937ca181745a48c0a7e5fa2a95b7a
  Author: Keith Miller <keith_miller at apple.com>
  Date:   2024-08-05 (Mon, 05 Aug 2024)

  Changed paths:
    M Tools/Scripts/compare-results

  Log Message:
  -----------
  Add a category-breakdown to compare-results
https://bugs.webkit.org/show_bug.cgi?id=277646
rdar://133235581

Reviewed by Yusuke Suzuki.

This provides a breakdown per benchamrk category (e.g. Worst, Startup, Average in JetStream2).
This looks like:

-------------------------------------------------------------------------------------------------------
| subtest |     ms       |     ms       |  b / a   | pValue (significance using False Discovery Rate) |
-------------------------------------------------------------------------------------------------------
| Average |19.187656     |19.209764     |1.001152  | 0.992457                                         |
| First   |32.916071     |32.796429     |0.996365  | 0.969416                                         |
| MainRun |2385.500000   |2386.600000   |1.000461  | 0.914809                                         |
| Runtime |629.700000    |632.100000    |1.003811  | 0.984437                                         |
| Startup |8.140000      |3.000000      |0.368550  | 0.041433                                         |
| Stdlib  |1224.500000   |1221.100000   |0.997223  | 0.271944                                         |
| Worst   |22.904464     |22.745536     |0.993061  | 0.947732                                         |
-------------------------------------------------------------------------------------------------------

for JetStream2. I only added it to JetStream2/3 since those benchmarks have a lot of different categories.

I also changed the script so it will log `--detailed-breakdown` and `--breakdown` if both are passed
rather than just `--detailed-breakdown`.

* Tools/Scripts/compare-results:
(detailedJetStream2Breakdown):
(categoryJetStream2Breakdown):
(detailedJetStream3Breakdown):
(getOptions):
(main):

Canonical link: https://commits.webkit.org/281867@main



To unsubscribe from these emails, change your notification settings at https://github.com/WebKit/WebKit/settings/notifications