[webkit-changes] [WebKit/WebKit] 653b53: [run-benchmarks] The browser benchmark runner harn...

[hysu]

  Branch: refs/heads/main
  Home:   https://github.com/WebKit/WebKit
  Commit: 653b530b9ffd3f4440fc6636b2da3c011633215f
      https://github.com/WebKit/WebKit/commit/653b530b9ffd3f4440fc6636b2da3c011633215f
  Author: Hysu Xiong <hysu at apple.com>
  Date:   2023-06-06 (Tue, 06 Jun 2023)

  Changed paths:
    M Tools/Scripts/webkitpy/benchmark_runner/benchmark_runner.py
    M Tools/Scripts/webkitpy/benchmark_runner/browser_driver/browser_driver.py
    M Tools/Scripts/webkitpy/benchmark_runner/data/patches/webserver/MotionMark.patch
    M Tools/Scripts/webkitpy/benchmark_runner/data/plans/motionmark1.2.1.plan
    M Tools/Scripts/webkitpy/benchmark_runner/webserver_benchmark_runner.py

  Log Message:
  -----------
  [run-benchmarks] The browser benchmark runner harness should be able to specify MotionMark subtests to run
https://bugs.webkit.org/show_bug.cgi?id=257663
rdar://110187208

Reviewed by Dewei Zhu.

This patch adds support for MotionMark subtests in run-benchmarks, similar to the support already included for JetStream2.

Subtests can be listed with the `--list-subtests` flag, and come in the form {SUITE}/{TEST}. MotionMark's default run (without any subtests specified) contains just the MotionMark suite in order, but the test plan includes all the other debug suites and tests as well for running separately.

To run subtests, use the --subtests flag and specify a list of subtests (as listed in --list-subtests), separated by spaces. The runner will run only those subtests in each iteration, and report on only those results.

Also, some more under-the hood improvements:
 - Added a new field to the plan, 'subtest_entry_point', that changes the entry point of the test if --subtests is specified.
 - Changes to webserver/motionmark.patch include not auto-loading the benchmark if `developer.html` is loaded in (because the debug runner already starts tests automatically if encoded), and extending the debug runner to accept multiple suites and tests as arguments.

* Tools/Scripts/webkitpy/benchmark_runner/benchmark_runner.py:
(BenchmarkRunner._run_benchmark):
* Tools/Scripts/webkitpy/benchmark_runner/browser_driver/browser_driver.py:
(BrowserDriver.profile): Fixed a bug where using the --profile flag would crash the benchmark harness, and added logging to note when the placeholder context is being called.
* Tools/Scripts/webkitpy/benchmark_runner/data/patches/webserver/MotionMark.patch:
* Tools/Scripts/webkitpy/benchmark_runner/data/plans/motionmark1.2.1.plan:
* Tools/Scripts/webkitpy/benchmark_runner/webserver_benchmark_runner.py:
(WebServerBenchmarkRunner):

Canonical link: https://commits.webkit.org/264906@main