[webkit-changes] [WebKit/WebKit] 6f6791: Parallelize Root Building in Performance Dashboard.

[dewei-zhu]

  Branch: refs/heads/main
  Home:   https://github.com/WebKit/WebKit
  Commit: 6f6791dcd7c496b1ac4e517d1633b5cee9e9ab70
      https://github.com/WebKit/WebKit/commit/6f6791dcd7c496b1ac4e517d1633b5cee9e9ab70
  Author: Dewei Zhu <dewei_zhu at apple.com>
  Date:   2024-07-03 (Wed, 03 Jul 2024)

  Changed paths:
    M Websites/perf.webkit.org/public/api/build-requests.php
    M Websites/perf.webkit.org/public/api/test-groups.php
    M Websites/perf.webkit.org/public/v3/models/build-request.js
    M Websites/perf.webkit.org/server-tests/api-test-groups.js
    M Websites/perf.webkit.org/server-tests/tools-buildbot-triggerable-tests.js
    M Websites/perf.webkit.org/server-tests/tools-sync-buildbot-integration-tests.js
    M Websites/perf.webkit.org/tools/js/buildbot-syncer.js
    M Websites/perf.webkit.org/tools/js/buildbot-triggerable.js

  Log Message:
  -----------
  Parallelize Root Building in Performance Dashboard.
https://bugs.webkit.org/show_bug.cgi?id=276102
rdar://100645242

Reviewed by Ryosuke Niwa.

Reduce root building time by running root building tasks in parallel. As part of this change, a build syncer
is no longer 1-to-1 mapping to a test group, thus the 'buildSyncer' variable is removed.
Since root building tasks from one test group can run in parallel, failure from any of them will fail all
test type requests. Whereas previously, any requests after a failed one including build type requests will be
marked as failed.
To avoid delaying the time a build failure notification is sent, '/api/test-groups/ready-for-notification'
now ignores the state of build type requests and only checks if all test type requests are in either 'failed'
or 'completed'.

Fix a bug that all build type requests with the same build configuration can unnecessarily be scheduled
in one syncing iteration. This is because all of them are pending at the beginning of the sync iteration
and are not eligible for root reuse. This change fixes this issue and ensures only one of them is
scheduled, and the rest will wait for reusing the build product.

* Websites/perf.webkit.org/public/api/build-requests.php: Make one root building failure only cause
test type requests to fail as now build type requests can run in parallel.
* Websites/perf.webkit.org/public/api/test-groups.php: Modify '/api/test-groups/ready-for-notification'
to only check test type requests.
* Websites/perf.webkit.org/public/v3/models/build-request.js:
(BuildRequest.prototype.async findBuildRequestWithSameRoots): Add an optional argument to contain request
id that is scheduled but is not updated yet.
* Websites/perf.webkit.org/server-tests/api-test-groups.js: Add a new unit test for
'/api/test-groups/ready-for-notification' behavior change.
* Websites/perf.webkit.org/server-tests/tools-buildbot-triggerable-tests.js: Fix existing unit tests
and add new tests.
* Websites/perf.webkit.org/server-tests/tools-sync-buildbot-integration-tests.js: Fix existing unit tests
and add new tests.
(configWithOneTesterTwoBuilders):
* Websites/perf.webkit.org/tools/js/buildbot-syncer.js:
(BuildbotSyncer.prototype.scheduleRequestInGroupIfAvailable): Add an optional arugment to specify if request
needs to be scheduled on idle worker (a worker without running or pending tasks) and make the build type
request no longer block the worker from being used by other test groups.
* Websites/perf.webkit.org/tools/js/buildbot-triggerable.js: Make build type requests can be scheduled in
parallel by removing the assumption that a test group only have one build syncer.
Add the logic to always prefer idle worker while scheduling.

Canonical link: https://commits.webkit.org/280656@main



To unsubscribe from these emails, change your notification settings at https://github.com/WebKit/WebKit/settings/notifications