[webkit-help] How to make all run-webkit-tests test pass? Or is it possible?

[Jeff]

Yes, I am using Mac Ports.
Maybe it is better to switch to Homebrew.

Thank you.

(2013/09/04 8:46), Benjamin Poulain wrote:
> The errors are odd. They seem to be randomly distributed.
>
> Do you use Mac Ports? I have seen strange errors before when Mac Ports 
> override system libraries.
>
> Benjamin
>
> On 9/3/13 4:11 PM, Jeff wrote:
>> Thank you Benjamin, Alexey
>>
>> This time, "I use run-webkit-tests -2" and following are my test this 
>> time. There are not so much fail, but is it normal?
>>
>> From results.html, I can find:
>> Tests that failed text/pixel/audio diff (5)
>> Tests that had stderr output (5)
>> Flaky tests (failed the first run and passed on retry) (5)
>> Tests expected to fail but passed (20)
>> Maybe I just mis-understanding the results.html?
>>
>> BEGIN OF TEST RESULT
>> =========================
>> Zheng-no-Mac-Pro:WebKit_git zhengxu$ Tools/Scripts/run-webkit-tests -2
>> Using port 'mac-mountainlion-wk2'
>> Test configuration: <mountainlion, x86_64, debug>
>> Placing test results in 
>> /Users/zhengxu/03_workspace/webkit/WebKit_git/WebKitBuild/Debug/layout-test-results
>> Baseline search path: mac-wk2 -> wk2 -> mac -> generic
>> Using Debug build
>> Pixel tests disabled
>> Regular timeout: 80000, slow test timeout: 400000
>> Command line: 
>> /Users/zhengxu/03_workspace/webkit/WebKit_git/WebKitBuild/Debug/WebKitTestRunner 
>> -
>>
>> --lint-test-files warnings:
>> LayoutTests/platform/mac/TestExpectations:1236 More specific entry 
>> for fast/block/float/overhanging-tall-block.html on line 
>> LayoutTests/platform/mac/TestExpectations:1236 overrides line 
>> LayoutTests/platform/mac/TestExpectations:1102. 
>> fast/block/float/overhanging-tall-block.html
>>
>> Found 34997 tests; running 32001, skipping 2996.
>> Running 18 WebKitTestRunners in parallel.
>>
>> [19/32001] 
>> compositing/backgrounds/fixed-background-on-descendant.html passed 
>> unexpectedly
>> [20/32001] compositing/background-color/background-color-alpha.html 
>> passed unexpectedly
>>   compositing/backgrounds/fixed-backgrounds.html -> ref test hashes 
>> didn't match but diff passed
>> [47/32001] compositing/backgrounds/fixed-backgrounds.html passed 
>> unexpectedly
>> compositing/background-color/background-color-change-to-text.html -> 
>> ref test hashes didn't match but diff passed
>> [47/32001] 
>> compositing/background-color/background-color-change-to-text.html 
>> passed unexpectedly
>> [68/32001] 
>> compositing/background-color/background-color-composite.html passed 
>> unexpectedly
>> [79/32001] 
>> compositing/background-color/background-color-container.html passed 
>> unexpectedly
>> [106/32001] 
>> compositing/background-color/background-color-content-clip.html 
>> passed unexpectedly
>> [138/32001] 
>> compositing/background-color/background-color-padding-clip.html 
>> passed unexpectedly
>> [165/32001] compositing/background-color/background-color-simple.html 
>> passed unexpectedly
>> [186/32001] 
>> compositing/background-color/background-color-text-change.html passed 
>> unexpectedly
>> [276/32001] compositing/images/positioned-image-content-rect.html 
>> passed unexpectedly
>> [594/32001] 
>> compositing/geometry/fixed-position-composited-page-scale-smaller-than-viewport.html 
>> passed unexpectedly
>> [694/32001] 
>> http/tests/cache/willsendrequest-returns-null-for-memory-cache-load.html 
>> passed unexpectedly
>>   css3/filters/custom/custom-filter-blend-modes.html -> ref test 
>> hashes didn't match but diff passed
>>   css3/unicode-bidi-isolate-aharon-failing.html -> ref test hashes 
>> didn't match but diff passed
>> [938/32001] css3/unicode-bidi-isolate-aharon-failing.html passed 
>> unexpectedly
>> [1081/32001] compositing/video/video-object-fit.html failed 
>> unexpectedly (text diff)
>> css3/filters/custom/composited/custom-filter-blend-modes.html -> ref 
>> test hashes didn't match but diff passed
>> [2770/32001] 
>> css3/filters/custom/effect-custom-transform-parameters.html passed 
>> unexpectedly
>> [5468/32001] 
>> editing/style/make-text-writing-direction-inline-mac.html failed 
>> unexpectedly (text diff)
>> [5577/32001] 
>> editing/style/make-text-writing-direction-inline-win.html failed 
>> unexpectedly (text diff)
>>   fast/css/sticky/sticky-left-percentage.html -> ref test hashes 
>> didn't match but diff passed
>> [9087/32001] http/tests/inspector/inspect-element.html failed 
>> unexpectedly (text diff)
>> [9694/32001] fast/dom/Range/range-expand.html failed unexpectedly 
>> (text diff)
>> [12142/32001] 
>> fast/layers/no-clipping-overflow-hidden-added-after-transition.html 
>> passed unexpectedly
>> [14235/32001] fast/events/scrollbar-double-click.html failed 
>> unexpectedly (text diff)
>> [15010/32001] fast/repaint/region-painting-in-composited-view.html 
>> passed unexpectedly
>>   fast/regions/fixed-inside-named-flow-zIndex.html -> ref test hashes 
>> didn't match but diff passed
>>   fast/regions/fixed-pos-elem-in-region.html -> ref test hashes 
>> didn't match but diff passed
>> [15560/32001] fast/regions/fixed-pos-elem-in-region.html passed 
>> unexpectedly
>> [15619/32001] fast/regions/fixed-pos-region-in-nested-flow.html 
>> failed unexpectedly (reference mismatch)
>> [15671/32001] fast/text/international/cjk-segmentation.html failed 
>> unexpectedly (text diff)
>> [19221/32001] inspector/storage-panel-dom-storage-update.html failed 
>> unexpectedly (text diff)
>> [20326/32001] jquery/manipulation.html failed unexpectedly (text diff)
>>   svg/repaint/buffered-rendering-static-image.html -> ref test hashes 
>> didn't match but diff passed
>> [27261/32001] svg/repaint/buffered-rendering-static-image.html passed 
>> unexpectedly
>> [29599/32001] http/tests/navigation/back-twice-without-commit.html 
>> passed unexpectedly
>>
>> Retrying 10 unexpected failure(s) ...
>>
>> Running 1 WebKitTestRunner.
>>
>> [3/10] editing/style/make-text-writing-direction-inline-mac.html 
>> failed unexpectedly (text diff)
>> [4/10] editing/style/make-text-writing-direction-inline-win.html 
>> failed unexpectedly (text diff)
>> [5/10] fast/dom/Range/range-expand.html failed unexpectedly (text diff)
>> [7/10] fast/regions/fixed-pos-region-in-nested-flow.html failed 
>> unexpectedly (reference mismatch)
>> [8/10] fast/text/international/cjk-segmentation.html failed 
>> unexpectedly (text diff)
>>
>> 31971 tests ran as expected, 30 didn't:
>>
>>
>> Expected to timeout, but passed: (1)
>>   http/tests/navigation/back-twice-without-commit.html
>>
>> Expected to fail, but passed: (20)
>>   compositing/background-color/background-color-alpha.html
>> compositing/background-color/background-color-change-to-text.html
>>   compositing/background-color/background-color-composite.html
>>   compositing/background-color/background-color-container.html
>> compositing/background-color/background-color-content-clip.html
>> compositing/background-color/background-color-padding-clip.html
>>   compositing/background-color/background-color-simple.html
>> compositing/background-color/background-color-text-change.html
>>   compositing/backgrounds/fixed-background-on-descendant.html
>>   compositing/backgrounds/fixed-backgrounds.html
>> compositing/geometry/fixed-position-composited-page-scale-smaller-than-viewport.html
>>   compositing/images/positioned-image-content-rect.html
>>   css3/filters/custom/effect-custom-transform-parameters.html
>>   css3/unicode-bidi-isolate-aharon-failing.html
>> fast/layers/no-clipping-overflow-hidden-added-after-transition.html
>>   fast/regions/fixed-pos-elem-in-region.html
>>   fast/repaint/region-painting-in-composited-view.html
>> http/tests/cache/willsendrequest-returns-null-for-memory-cache-load.html
>>   svg/repaint/buffered-rendering-static-image.html
>>   tables/mozilla_expected_failures/bugs/bug178855.xml
>>
>>
>> Unexpected flakiness: text-only failures (5)
>>   compositing/video/video-object-fit.html [ Failure Pass ]
>>   fast/events/scrollbar-double-click.html [ Failure Pass ]
>>   http/tests/inspector/inspect-element.html [ Failure Pass ]
>>   inspector/storage-panel-dom-storage-update.html [ Failure Pass ]
>>   jquery/manipulation.html [ Failure Pass ]
>>
>>
>> Regressions: Unexpected text-only failures (4)
>>   editing/style/make-text-writing-direction-inline-mac.html [ Failure ]
>>   editing/style/make-text-writing-direction-inline-win.html [ Failure ]
>>   fast/dom/Range/range-expand.html [ Failure ]
>>   fast/text/international/cjk-segmentation.html [ Failure ]
>>
>> Regressions: Unexpected image-only failures (1)
>>   fast/regions/fixed-pos-region-in-nested-flow.html [ ImageOnlyFailure ]
>>
>> Failed to execute Tools/Scripts/new-run-webkit-tests at 
>> Tools/Scripts/run-webkit-tests line 100.
>> =========================
>> END OF TEST RESULT
>>
>> (2013/09/04 7:08), Benjamin Poulain wrote:
>>> On 9/3/13 6:43 AM, 徐征 wrote:
>>>> I am not familiar with layout test of webkit. I created some patch 
>>>> but hope it is easy to find any degression via layout test.
>>>>
>>>> Everytime, I create some patch, I just run 
>>>> |Tools/Script/run-webkit-tests and all the time there are many 
>>>> fail(more than 100).|
>>>> ||I think maybe I just use wrong option for this script.
>>>> |It there anyway to pass all the test in run-webkit-tests?
>>>> Or, if it is normal that not all test are passed, is there any way 
>>>> to confirm degression efficient?
>>>>
>>>> I am using OSX10.8.4.
>>>> And I find in http://www.webkit.org/quality/testing.html, it is 
>>>> written that "Regression tests must be run on Leopard", should I 
>>>> run test on Leopard?|
>>> The testing webpage is wrong, I'll fix it.
>>>
>>> The common ways to run the tests are "run-webkit-tests -2" and 
>>> "run-webkit-test".
>>>
>>> OS X 10.8.4 is the best platform to run the tests, you should be 
>>> able to run them without problems.
>>> What kind of errors do you get?
>>>
>>> Benjamin
>>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-help/attachments/20130904/652acc1f/attachment.html>