[Webkit-unassigned] [Bug 93434] New: [NRWT] REGRESSION(r124967): New tests without expected results handled as failures

[bugzilla-daemon at webkit.org]

https://bugs.webkit.org/show_bug.cgi?id=93434

           Summary: [NRWT] REGRESSION(r124967): New tests without expected
                    results handled as failures
           Product: WebKit
           Version: 528+ (Nightly build)
          Platform: All
        OS/Version: All
            Status: NEW
          Keywords: NRWT
          Severity: Normal
          Priority: P2
         Component: Tools / Tests
        AssignedTo: webkit-unassigned at lists.webkit.org
        ReportedBy: ossy at webkit.org


Before r124967 new tests without expected files were handled as new tests,
not as failures. Which means only orange buildbot, not red buildbot. But
after r124967, new tests make the bot red, which is not the expected behaviour.

Let's see the NRWT's output:

before r124967:
----------------
Unexpected flakiness: text diff mismatch (4)
  fast/forms/range/slider-mouse-events.html = TEXT PASS
  fast/forms/range/slider-onchange-event.html = TEXT PASS
  http/tests/security/contentSecurityPolicy/object-src-url-allowed.html = TEXT PASS
  http/tests/security/cross-frame-access-put.html = TEXT PASS


Regressions: Unexpected no expected results found : (2)
  fast/table/table-row-outline-paint.html = MISSING
  svg/as-image/animated-svg-repaints-completely-in-hidpi.html = MISSING


and the buildbot reports:  4 flakes 2 missing results 

after r124967:
---------------
Unexpected flakiness: text failures (4)
  fast/forms/range/slider-mouse-events.html = TEXT PASS
  fast/forms/range/slider-onchange-event.html = TEXT PASS
  http/tests/security/contentSecurityPolicy/object-src-url-allowed.html = TEXT PASS
  http/tests/security/cross-frame-access-put.html = TEXT PASS


Regressions: Unexpected missing results : (2)
  fast/table/table-row-outline-paint.html = MISSING
  svg/as-image/animated-svg-repaints-completely-in-hidpi.html = MISSING


and the buildbot reports:  2 failures 4 flakes


It seems it is a similar bug as https://bugs.webkit.org/show_bug.cgi?id=93346

-- 
Configure bugmail: https://bugs.webkit.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.