[webkit-changes] [WebKit/WebKit] ca4e82: [run-webkit-tests] Tests which fail and then crash...

[Jonathan Bedard]

  Branch: refs/heads/main
  Home:   https://github.com/WebKit/WebKit
  Commit: ca4e82e8dd0a7ea4c7729cc2785457318acd96d5
      https://github.com/WebKit/WebKit/commit/ca4e82e8dd0a7ea4c7729cc2785457318acd96d5
  Author: Jonathan Bedard <jbedard at apple.com>
  Date:   2024-05-23 (Thu, 23 May 2024)

  Changed paths:
    M Tools/Scripts/webkitpy/layout_tests/models/test_run_results.py
    M Tools/Scripts/webkitpy/layout_tests/run_webkit_tests_integrationtest.py
    M Tools/Scripts/webkitpy/port/test.py

  Log Message:
  -----------
  [run-webkit-tests] Tests which fail and then crash on retry aren't flakey
https://bugs.webkit.org/show_bug.cgi?id=265066
rdar://118578976

Reviewed by Aakash Jain.

* Tools/Scripts/webkitpy/layout_tests/models/test_run_results.py:
(summarize_results): Tests that have unexpected behavior on retry that differs from the original
failure aren't "flaky", they are still regressions and should be treated that way.
* Tools/Scripts/webkitpy/layout_tests/run_webkit_tests_integrationtest.py:
(RunTest.serial_test_basic): Resolve the Fixme.
(RunTest.serial_test_run_singly_actually_runs_tests): Ditto.
(RunTest.test_retrying_multiple_failures):
(RunTest.test_retrying_multiple_failures_expected):
* Tools/Scripts/webkitpy/port/test.py: Add support for tests whose behavior changes on retry.

Canonical link: https://commits.webkit.org/279220@main



To unsubscribe from these emails, change your notification settings at https://github.com/WebKit/WebKit/settings/notifications