[webkit-help] How to make all run-webkit-tests test pass? Or is it possible?

[Jeff]

Thank you Benjamin, Alexey

This time, "I use run-webkit-tests -2" and following are my test this 
time. There are not so much fail, but is it normal?

 From results.html, I can find:
Tests that failed text/pixel/audio diff (5)
Tests that had stderr output (5)
Flaky tests (failed the first run and passed on retry) (5)
Tests expected to fail but passed (20)
Maybe I just mis-understanding the results.html?

BEGIN OF TEST RESULT
=========================
Zheng-no-Mac-Pro:WebKit_git zhengxu$ Tools/Scripts/run-webkit-tests -2
Using port 'mac-mountainlion-wk2'
Test configuration: <mountainlion, x86_64, debug>
Placing test results in 
/Users/zhengxu/03_workspace/webkit/WebKit_git/WebKitBuild/Debug/layout-test-results
Baseline search path: mac-wk2 -> wk2 -> mac -> generic
Using Debug build
Pixel tests disabled
Regular timeout: 80000, slow test timeout: 400000
Command line: 
/Users/zhengxu/03_workspace/webkit/WebKit_git/WebKitBuild/Debug/WebKitTestRunner 
-

--lint-test-files warnings:
LayoutTests/platform/mac/TestExpectations:1236 More specific entry for 
fast/block/float/overhanging-tall-block.html on line 
LayoutTests/platform/mac/TestExpectations:1236 overrides line 
LayoutTests/platform/mac/TestExpectations:1102. 
fast/block/float/overhanging-tall-block.html

Found 34997 tests; running 32001, skipping 2996.
Running 18 WebKitTestRunners in parallel.

[19/32001] compositing/backgrounds/fixed-background-on-descendant.html 
passed unexpectedly
[20/32001] compositing/background-color/background-color-alpha.html 
passed unexpectedly
   compositing/backgrounds/fixed-backgrounds.html -> ref test hashes 
didn't match but diff passed
[47/32001] compositing/backgrounds/fixed-backgrounds.html passed 
unexpectedly
compositing/background-color/background-color-change-to-text.html -> ref 
test hashes didn't match but diff passed
[47/32001] 
compositing/background-color/background-color-change-to-text.html passed 
unexpectedly
[68/32001] compositing/background-color/background-color-composite.html 
passed unexpectedly
[79/32001] compositing/background-color/background-color-container.html 
passed unexpectedly
[106/32001] 
compositing/background-color/background-color-content-clip.html passed 
unexpectedly
[138/32001] 
compositing/background-color/background-color-padding-clip.html passed 
unexpectedly
[165/32001] compositing/background-color/background-color-simple.html 
passed unexpectedly
[186/32001] 
compositing/background-color/background-color-text-change.html passed 
unexpectedly
[276/32001] compositing/images/positioned-image-content-rect.html passed 
unexpectedly
[594/32001] 
compositing/geometry/fixed-position-composited-page-scale-smaller-than-viewport.html 
passed unexpectedly
[694/32001] 
http/tests/cache/willsendrequest-returns-null-for-memory-cache-load.html 
passed unexpectedly
   css3/filters/custom/custom-filter-blend-modes.html -> ref test hashes 
didn't match but diff passed
   css3/unicode-bidi-isolate-aharon-failing.html -> ref test hashes 
didn't match but diff passed
[938/32001] css3/unicode-bidi-isolate-aharon-failing.html passed 
unexpectedly
[1081/32001] compositing/video/video-object-fit.html failed unexpectedly 
(text diff)
   css3/filters/custom/composited/custom-filter-blend-modes.html -> ref 
test hashes didn't match but diff passed
[2770/32001] css3/filters/custom/effect-custom-transform-parameters.html 
passed unexpectedly
[5468/32001] editing/style/make-text-writing-direction-inline-mac.html 
failed unexpectedly (text diff)
[5577/32001] editing/style/make-text-writing-direction-inline-win.html 
failed unexpectedly (text diff)
   fast/css/sticky/sticky-left-percentage.html -> ref test hashes didn't 
match but diff passed
[9087/32001] http/tests/inspector/inspect-element.html failed 
unexpectedly (text diff)
[9694/32001] fast/dom/Range/range-expand.html failed unexpectedly (text 
diff)
[12142/32001] 
fast/layers/no-clipping-overflow-hidden-added-after-transition.html 
passed unexpectedly
[14235/32001] fast/events/scrollbar-double-click.html failed 
unexpectedly (text diff)
[15010/32001] fast/repaint/region-painting-in-composited-view.html 
passed unexpectedly
   fast/regions/fixed-inside-named-flow-zIndex.html -> ref test hashes 
didn't match but diff passed
   fast/regions/fixed-pos-elem-in-region.html -> ref test hashes didn't 
match but diff passed
[15560/32001] fast/regions/fixed-pos-elem-in-region.html passed unexpectedly
[15619/32001] fast/regions/fixed-pos-region-in-nested-flow.html failed 
unexpectedly (reference mismatch)
[15671/32001] fast/text/international/cjk-segmentation.html failed 
unexpectedly (text diff)
[19221/32001] inspector/storage-panel-dom-storage-update.html failed 
unexpectedly (text diff)
[20326/32001] jquery/manipulation.html failed unexpectedly (text diff)
   svg/repaint/buffered-rendering-static-image.html -> ref test hashes 
didn't match but diff passed
[27261/32001] svg/repaint/buffered-rendering-static-image.html passed 
unexpectedly
[29599/32001] http/tests/navigation/back-twice-without-commit.html 
passed unexpectedly

Retrying 10 unexpected failure(s) ...

Running 1 WebKitTestRunner.

[3/10] editing/style/make-text-writing-direction-inline-mac.html failed 
unexpectedly (text diff)
[4/10] editing/style/make-text-writing-direction-inline-win.html failed 
unexpectedly (text diff)
[5/10] fast/dom/Range/range-expand.html failed unexpectedly (text diff)
[7/10] fast/regions/fixed-pos-region-in-nested-flow.html failed 
unexpectedly (reference mismatch)
[8/10] fast/text/international/cjk-segmentation.html failed unexpectedly 
(text diff)

31971 tests ran as expected, 30 didn't:


Expected to timeout, but passed: (1)
   http/tests/navigation/back-twice-without-commit.html

Expected to fail, but passed: (20)
   compositing/background-color/background-color-alpha.html
compositing/background-color/background-color-change-to-text.html
   compositing/background-color/background-color-composite.html
   compositing/background-color/background-color-container.html
   compositing/background-color/background-color-content-clip.html
   compositing/background-color/background-color-padding-clip.html
   compositing/background-color/background-color-simple.html
   compositing/background-color/background-color-text-change.html
   compositing/backgrounds/fixed-background-on-descendant.html
   compositing/backgrounds/fixed-backgrounds.html
compositing/geometry/fixed-position-composited-page-scale-smaller-than-viewport.html
   compositing/images/positioned-image-content-rect.html
   css3/filters/custom/effect-custom-transform-parameters.html
   css3/unicode-bidi-isolate-aharon-failing.html
fast/layers/no-clipping-overflow-hidden-added-after-transition.html
   fast/regions/fixed-pos-elem-in-region.html
   fast/repaint/region-painting-in-composited-view.html
http/tests/cache/willsendrequest-returns-null-for-memory-cache-load.html
   svg/repaint/buffered-rendering-static-image.html
   tables/mozilla_expected_failures/bugs/bug178855.xml


Unexpected flakiness: text-only failures (5)
   compositing/video/video-object-fit.html [ Failure Pass ]
   fast/events/scrollbar-double-click.html [ Failure Pass ]
   http/tests/inspector/inspect-element.html [ Failure Pass ]
   inspector/storage-panel-dom-storage-update.html [ Failure Pass ]
   jquery/manipulation.html [ Failure Pass ]


Regressions: Unexpected text-only failures (4)
   editing/style/make-text-writing-direction-inline-mac.html [ Failure ]
   editing/style/make-text-writing-direction-inline-win.html [ Failure ]
   fast/dom/Range/range-expand.html [ Failure ]
   fast/text/international/cjk-segmentation.html [ Failure ]

Regressions: Unexpected image-only failures (1)
   fast/regions/fixed-pos-region-in-nested-flow.html [ ImageOnlyFailure ]

Failed to execute Tools/Scripts/new-run-webkit-tests at 
Tools/Scripts/run-webkit-tests line 100.
=========================
END OF TEST RESULT

(2013/09/04 7:08), Benjamin Poulain wrote:
> On 9/3/13 6:43 AM, 徐征 wrote:
>> I am not familiar with layout test of webkit. I created some patch 
>> but hope it is easy to find any degression via layout test.
>>
>> Everytime, I create some patch, I just run 
>> |Tools/Script/run-webkit-tests and all the time there are many 
>> fail(more than 100).|
>> ||I think maybe I just use wrong option for this script.
>> |It there anyway to pass all the test in run-webkit-tests?
>> Or, if it is normal that not all test are passed, is there any way to 
>> confirm degression efficient?
>>
>> I am using OSX10.8.4.
>> And I find in http://www.webkit.org/quality/testing.html, it is 
>> written that "Regression tests must be run on Leopard", should I run 
>> test on Leopard?|
> The testing webpage is wrong, I'll fix it.
>
> The common ways to run the tests are "run-webkit-tests -2" and 
> "run-webkit-test".
>
> OS X 10.8.4 is the best platform to run the tests, you should be able 
> to run them without problems.
> What kind of errors do you get?
>
> Benjamin

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-help/attachments/20130904/1ba110fc/attachment-0001.html>