[Webkit-unassigned] [Bug 76243] New: [chromium] Reftests expected to be image mismatch still generate image mismatch when ran by NRWT --no-pixel-tests.

[bugzilla-daemon at webkit.org]

https://bugs.webkit.org/show_bug.cgi?id=76243

           Summary: [chromium] Reftests expected to be image mismatch
                    still generate image mismatch when ran by NRWT
                    --no-pixel-tests.
           Product: WebKit
           Version: 528+ (Nightly build)
          Platform: Unspecified
        OS/Version: Unspecified
            Status: NEW
          Severity: Normal
          Priority: P2
         Component: Tools / Tests
        AssignedTo: webkit-unassigned at lists.webkit.org
        ReportedBy: zhenghao at chromium.org
                CC: tony at chromium.org


For example, fast/harness/sample-fail-mismatch-reftest.html
When ran by 'NRWT fast/harness/sample-fail-mismatch-reftest.html', all is fine: All 1 tests ran as expected.
When ran by 'NRWT --no-pixel-tests fast/harness/sample-fail-mismatch-reftest.html', output:
Regressions: Unexpected image mismatch : (1)
  fast/harness/sample-fail-mismatch-reftest.html = IMAGE

In TestExpectations.matches_an_expected_result, it removes pixel expectation when pixel tests is disabled.
        if not pixel_tests_are_enabled:
            expected_results = remove_pixel_failures(expected_results)
However, in test_failures.determine_result_type, reftest failure is treated as image failure.
        elif is_image_failure or is_reftest_failure:
            return test_expectations.IMAGE

-- 
Configure bugmail: https://bugs.webkit.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.