[webkit-changes] [WebKit/WebKit] 440943: REGRESSION(257654 at main): [ews-build.webkit.org][GT...

[Carlos Alberto Lopez Perez]

  Branch: refs/heads/main
  Home:   https://github.com/WebKit/WebKit
  Commit: 440943ba5b7ea75da370ea5a3d0325262a055127
      https://github.com/WebKit/WebKit/commit/440943ba5b7ea75da370ea5a3d0325262a055127
  Author: Carlos Alberto Lopez Perez <clopez at igalia.com>
  Date:   2023-03-03 (Fri, 03 Mar 2023)

  Changed paths:
    M Tools/CISupport/ews-build/steps.py
    M Tools/CISupport/ews-build/steps_unittest.py

  Log Message:
  -----------
  REGRESSION(257654 at main): [ews-build.webkit.org][GTK][WPE] RunWebKitTestsRedTree sub-classes are overwriting the 'first_run' properties on the retry steps.
https://bugs.webkit.org/show_bug.cgi?id=253048

Reviewed by Jonathan Bedard.

257654 at main changed the way the class RunWebKitTests() works: instead of specializing the function
buildstep.BuildStep.commandComplete() the function buildstep.BuildStep.runCommand() is specialized.

Buildbot first calls runCommand() and then calls commandComplete()

The problem is with the subclasses of RunWebKitTests() which were specializing a commandComplete()
function: now instead of executing the generic buildstep.BuildStep.runCommand() they execute the
specialized RunWebKitTests.runCommand() which was the previous RunWebKitTests.commandComplete()
so this has the undesired effect that the properties defined at RunWebKitTests().runCommand()
('first_run_flakies', 'first_run_failures' and 'first_results_exceed_failure_limit') get
redefined (overwritten) multiple times with the values generated when running the command
of each one of the sub-classes.

This is wrong and can cause unexpected issues later when analyzing the results in
AnalyzeLayoutTestsResultsRedTree() because the values of the properties 'first_run_flakies'
and 'first_run_failures' are not the ones from the first run but the ones from the last
retry step executed.

Fix it by specializing runCommand() instead of commandComplete() on the subclasses like
RunWebKitTests() class does now. Add also a few unit tests.

* Tools/CISupport/ews-build/steps.py:
(RunWebKitTestsRepeatFailuresRedTree):
(RunWebKitTestsRepeatFailuresRedTree.runCommand):
(RunWebKitTestsRepeatFailuresWithoutChangeRedTree):
(RunWebKitTestsRepeatFailuresWithoutChangeRedTree.runCommand):
(RunWebKitTestsRepeatFailuresRedTree.commandComplete): Deleted.
(RunWebKitTestsRepeatFailuresWithoutChangeRedTree.commandComplete): Deleted.
* Tools/CISupport/ews-build/steps_unittest.py:
(MockLayoutTestFailures):
(MockLayoutTestFailures.__init__):
(TestRunWebKitTestsRedTree.configureStep):
(TestRunWebKitTestsRedTree.test_success):
(TestRunWebKitTestsRedTree.test_set_properties_when_executed_scope_this_class):
(TestRunWebKitTestsRepeatFailuresRedTree.configureStep):
(TestRunWebKitTestsRepeatFailuresRedTree.test_success):
(TestRunWebKitTestsRepeatFailuresRedTree.test_set_properties_when_executed_scope_this_class):
(TestRunWebKitTestsRepeatFailuresWithoutChangeRedTree.configureStep):
(TestRunWebKitTestsRepeatFailuresWithoutChangeRedTree.test_success):
(TestRunWebKitTestsRepeatFailuresWithoutChangeRedTree.test_step_with_change_did_timeout):
(TestRunWebKitTestsRepeatFailuresWithoutChangeRedTree.test_set_properties_when_executed_scope_this_class):

Canonical link: https://commits.webkit.org/261167@main