[Webkit-unassigned] [Bug 227744] New: [GTK] Add a new GTK layout tester bot to build.webkit.org that runs with --skip-failing-tests switch

[bugzilla-daemon at webkit.org]

https://bugs.webkit.org/show_bug.cgi?id=227744

            Bug ID: 227744
           Summary: [GTK] Add a new GTK layout tester bot to
                    build.webkit.org that runs with --skip-failing-tests
                    switch
           Product: WebKit
           Version: WebKit Nightly Build
          Hardware: Unspecified
                OS: Unspecified
            Status: NEW
          Severity: Normal
          Priority: P2
         Component: Tools / Tests
          Assignee: webkit-unassigned at lists.webkit.org
          Reporter: clopez at igalia.com
                CC: aakash_jain at apple.com, bugs-noreply at webkitgtk.org,
                    dpino at igalia.com

We are trying to have a layout tests runner for the GTK port in the EWS, but we are experimenting there random flakiness and failures that are not reproducible on the GTK layout tester of build.webkit.org

I think this is related with the different way of running the layout tests:
- On build.webkit.org all known tests are scheduled for running (even those that are expected to crash or timeout).
- On the EWS, the switch --skip-failing-tests is passed and that changes the order in which the tests are run because the test that are expected to fail are not scheduled for running.

This different order of scheduling the test in the EWS I think it may cause different results sometimes or can be a source of undetected flakiness.

So I think is a good idea to add a new tester for the GTK port for build.webkit.org that runs with this flag like in the EWS.

I believe that will help us to easily discover and make easier doing gardening of this tests that fail or are flaky due to this different order of scheduling the tests because thanks to this bot we will get access to info about this runs in https://results.webkit.org and wktesthunter <https://github.com/clopez/webkit-testhunter>

-- 
You are receiving this mail because:
You are the assignee for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-unassigned/attachments/20210707/76b59f80/attachment.htm>