[webkit-changes] [WebKit/WebKit] 0ac614: Run-benchmark should be able to checkout out a fix...

[dewei-zhu]

  Branch: refs/heads/main
  Home:   https://github.com/WebKit/WebKit
  Commit: 0ac614b6822ca9eccec5f40b7d00bf746f1f08a9
      https://github.com/WebKit/WebKit/commit/0ac614b6822ca9eccec5f40b7d00bf746f1f08a9
  Author: Dewei Zhu <dewei_zhu at apple.com>
  Date:   2022-09-02 (Fri, 02 Sep 2022)

  Changed paths:
    M Tools/Scripts/webkitpy/benchmark_runner/benchmark_builder.py
    M Tools/Scripts/webkitpy/benchmark_runner/data/plans/jetstream2.1.plan
    A Tools/Scripts/webkitpy/benchmark_runner/data/plans/motionmark1.2.1.plan
    A Tools/Scripts/webkitpy/benchmark_runner/github_downloader.py
    A Tools/Scripts/webkitpy/benchmark_runner/plan_file_optimizer.py

  Log Message:
  -----------
  Run-benchmark should be able to checkout out a fixed version of benchmark from Git/GitHub
https://bugs.webkit.org/show_bug.cgi?id=244704
rdar://99465558

Reviewed by Stephanie Lewis.

Add 'local_git_archive' and 'github_source' schema to plan file so that we can download
a subdirectory from a Git repository or from GitHub.
Add 'github_subtree' schema to plan file as a cached and compressed response from GitHub
tree API. This can avoid hitting rate limit in GitHub API.
Write a module to support downloading subdirectories from GitHub.
Fix JetStream2.1 plan file to checkout the right version of JetStream.
Added MotionMark1.2.1 plan file.

* Tools/Scripts/webkitpy/benchmark_runner/benchmark_builder.py: Added support for 'local_git_archive'
and 'github_source'.
(BenchmarkBuilder):
(BenchmarkBuilder.__enter__):
(BenchmarkBuilder._download_from_github):
(BenchmarkBuilder._local_git_archive_eligible):
(BenchmarkBuilder._prepare_content_from_local_git_archive):
* Tools/Scripts/webkitpy/benchmark_runner/data/plans/jetstream2.1.plan:
* Tools/Scripts/webkitpy/benchmark_runner/data/plans/motionmark1.2.1.plan: Added.
* Tools/Scripts/webkitpy/benchmark_runner/github_downloader.py: Added this module for downloading a
subdirectory from GitHub using GitHub API and GitHub raw content endpoint.
(GithubDownloadTask):
(GithubDownloadTask.__init__):
(GithubDownloadTask.execute):
(GithubDownloadTask.subtree):
(retry_on_exception):
(retry_on_exception.decorator):
(retry_on_exception.decorator.wrapped):
(GithubDownloader):
(GithubDownloader.__init__):
(GithubDownloader.is_directory):
(GithubDownloader.is_file):
(GithubDownloader.is_symlink):
(GithubDownloader.session):
(GithubDownloader.download_content):
(GithubDownloader.subtree_for_path_with_commit):
(GithubDownloader._download_subtree_multiprocessing):
(GithubDownloader.walk):
(GithubDownloader._walk):
(GithubDownloader._subtree):
(GithubDownloader._commit):
(GithubDownloader._raw_content_url):
(GithubDownloader._download_raw_content):
(GithubDownloader._resolve_symlink):
(GithubDownloader._request):
* Tools/Scripts/webkitpy/benchmark_runner/plan_file_optimizer.py: Added this script
to cache the response from GitHub tree API for a given commit of a benchmark. This
will avoid hit any GitHub API, thus avoid hitting GitHub API rate limit.
(optimize_plan_file):
(optimize_plan_file.drop_keys):
(main):

Canonical link: https://commits.webkit.org/254124@main