[webkit-changes] [WebKit/WebKit] 566fb3: [Tools][GTK][WPE] MVT test suite: integrate run-mv...

[Carlos Alberto Lopez Perez]

  Branch: refs/heads/main
  Home:   https://github.com/WebKit/WebKit
  Commit: 566fb370f6b383f85b715686e67a39098bf8dc73
      https://github.com/WebKit/WebKit/commit/566fb370f6b383f85b715686e67a39098bf8dc73
  Author: Carlos Alberto Lopez Perez <clopez at igalia.com>
  Date:   2025-01-14 (Tue, 14 Jan 2025)

  Changed paths:
    M Tools/Scripts/libraries/resultsdbpy/resultsdbpy/__init__.py
    M Tools/Scripts/run-minibrowser
    M Tools/Scripts/run-mvt-tests
    A Tools/Scripts/run-webdriver
    M Tools/Scripts/webkitpy/__init__.py
    M Tools/Scripts/webkitpy/port/base.py
    M Tools/Scripts/webkitpy/port/glib.py
    M Tools/Scripts/webkitpy/port/gtk.py
    M Tools/Scripts/webkitpy/port/wpe.py
    M Tools/Scripts/webkitpy/webdriver_tests/webdriver_driver_wpe.py
    M Tools/Scripts/webkitpy/webdriver_tests/webdriver_test_runner_selenium.py
    A Tools/gtk/MVT_TestExpectations_Release.json
    A Tools/wpe/MVT_TestExpectations_Release.json

  Log Message:
  -----------
  [Tools][GTK][WPE] MVT test suite: integrate run-mvt-tests with webkitpy facilities and support test expectations
https://bugs.webkit.org/show_bug.cgi?id=284916

Reviewed by Adrian Perez de Castro.

This patch improves the script run-mvt-tests with the following features:

 - Support running with WebKitGTK.
 - For WPE support running both with MiniBrowser and Cog.
 - Support running without Flatpak (system libs) as well as with JHBuild.
 - Automatic enabling of headless mode via command-line switch
 - Allow to run the whole MVT tests automatically (all test suites).
   This is the default now.
 - Support for test expectations in order to mark expected failures, including
   support for auto-generating new expectations based on the test run results.
 - Improved logging (with options to enable debug messages or to run quiet only
   logging unexpected failures).
 - Better integration with webkitpy facilities (by using the new run-webdriver
   script that integrates with the webkitpy port driver to set the environment
   for the webdriver and browser rather than manually having to set environment
   variables inside the script).
 - Improved handling of corner-cases where the WebDriver hangs.
 - Automatic cleaning of leftover process after the test run.
 - Automatic randomization of the WebDriver port used for tests, making easier
   to trigger several test runs on the same host at the same time.

On top of that, this also fixes the following:

 - run-minibrowser: was not correctly setting the exit code on failure.
 - create a new run-webdriver helper script to start the webdriver from
   a given platform/port built at WebKitBuild.
 - autoinstalled selenium: upgrade to a newer version of selenium and properly
   install the new websocket-client subdependency with the auto-install logic.
   Note: This new version of Selenium is required to create the driver with a
   custom path and arguments for the webdriver.

* Tools/Scripts/libraries/resultsdbpy/resultsdbpy/__init__.py:
* Tools/Scripts/run-minibrowser:
* Tools/Scripts/run-mvt-tests:
(parse_args):
(fatal): Deleted.
(options): Deleted.
(suite_has_started): Deleted.
(suite_has_finished): Deleted.
(analyze_results): Deleted.
(print_tests_failed): Deleted.
(WPEWebDriver): Deleted.
(WPEWebDriver.__init__): Deleted.
(WPEWebDriver.start): Deleted.
(WPEWebDriver.stop): Deleted.
(WPEWebDriver._isRunning): Deleted.
(WPEWebDriver._isFlatpakSDK): Deleted.
(WPEWebDriver._isJHBuild): Deleted.
(WPEWebDriver._run): Deleted.
(WPEWebDriver._run.dict_flat): Deleted.
(main): Deleted.
* Tools/Scripts/run-webdriver: Added.
(system_has_ipv6):
(main):
* Tools/Scripts/webkitpy/__init__.py:
* Tools/Scripts/webkitpy/port/base.py:
(Port.run_webdriver):
* Tools/Scripts/webkitpy/port/glib.py:
(GLibPort._get_crash_log):
(GLibPort):
(GLibPort.setup_environ_for_webdriver):
(GLibPort.run_webdriver):
* Tools/Scripts/webkitpy/port/gtk.py:
(GtkPort):
* Tools/Scripts/webkitpy/port/wpe.py:
(WPEPort):
(WPEPort.setup_environ_for_webdriver):
* Tools/Scripts/webkitpy/webdriver_tests/webdriver_driver_wpe.py:
(WebDriverWPE.browser_env):
* Tools/Scripts/webkitpy/webdriver_tests/webdriver_test_runner_selenium.py:
* Tools/gtk/MVT_TestExpectations_Release.json: Added.
* Tools/wpe/MVT_TestExpectations_Release.json: Added.

Canonical link: https://commits.webkit.org/288842@main



To unsubscribe from these emails, change your notification settings at https://github.com/WebKit/WebKit/settings/notifications