[webkit-dev] Disabled layout tests - we should get them re-enabled

Sun Jun 15 16:07:32 PDT 2008

On Jun 15, 2008, at 3:42 PM, Jean-Charles VERDIE (Pleyo) wrote:

> Hi,
>
> I've ran all theses tests on rev. 34571 to get a first glance. The  
> test tool is not crashing, while most of them are unclear to me  
> whether they are OK or KO.
>
>
> Here is a summary of the results, sorry for the poor layout ...
>
> |./css2.1/t1202-counter-10-b.html-disabled|OK|
> |./css2.1/t1202-counters-10-b.html-disabled|OK|
> |./dom/xhtml/level3/core/domimplementationregistry23.xhtml-disabled| 
> KO (NO CRASH)|
> |./dom/xhtml/level3/core/nodecomparedocumentposition14.xhtml- 
> disabled|KO (NO CRASH)|
> |./dom/xhtml/level3/core/nodecomparedocumentposition15.xhtml- 
> disabled|KO (NO CRASH)|
> |./dom/xhtml/level3/core/noderemovechild03.xhtml-disabled|KO (NO  
> CRASH)|
> |./dom/xhtml/level3/core/nodereplacechild06.xhtml-disabled|KO (NO  
> CRASH)|
> |./dom/xhtml/level3/core/nodereplacechild07.xhtml-disabled|KO (NO  
> CRASH)|
> |./dom/xhtml/level3/core/nodereplacechild08.xhtml-disabled|KO (NO  
> CRASH)|
> |./editing/execCommand/create-list-1.html-disabled|OK|
> |./editing/input/attributed-substring-from-range-lines.html-disabled| 
> UNCLEAR (NO CRASH)|
> |./editing/pasteboard/paste-empty-startcontainer.html-disabled|KO  
> (NO CRASH)|
> |./editing/selection/inconsistent-in-removeChildNode.html-disabled| 
> KO (NO CRASH)|
> |./editing/style/5091898.html-disabled|OK|
> |./fast/canvas/toDataURL-supportedTypes.html-disabled|UNCLEAR (NO  
> CRASH)|
> |./fast/css/css2-system-color.html-disabled|UNCLEAR (NO CRASH)|
> |./fast/dom/Window/window-resize-nan.html-disabled|KO (NO CRASH)|
> |./fast/events/key-events-in-frame.html-disabled|KO (NO CRASH)|
> |./fast/frames/iframe-scroll-page-up-down.html-disabled|OK|
> |./fast/images/svg-background-crash-on-refresh.html-disabled|OK|
> |./fast/js/kde/Date.html-disabled|KO (NO CRASH)|
> |./fast/js/string-concatenate-outofmemory.html-disabled|TIME OUT|
> |./fast/leaks/003.html-disabled|KO (NO CRASH)|
> |./fast/loader/form-events-back-forward.html-disabled|KO (NO CRASH)|
> |./http/tests/navigation/post-frames-goback1.html-disabled|KO (NO  
> CRASH)|
> |./http/tests/navigation/postredirect-frames-goback1.html-disabled| 
> UNCLEAR (NO CRASH) (UPDATE OPENDARWIN)|
> |./http/tests/navigation/success200-frames-goback.html-disabled| 
> UNCLEAR (NO CRASH) (UPDATE OPENDARWIN)|
> |./http/tests/navigation/success200-frames-reload.html-disabled| 
> UNCLEAR (NO CRASH) (UPDATE OPENDARWIN)|
> |./http/tests/navigation/success200-subframeload-goback.html- 
> disabled|UNCLEAR (NO CRASH) (UPDATE OPENDARWIN)|
> |./http/tests/xmlhttprequest/web-apps/006.html-disabled|OK|
> |./http/tests/xmlhttprequest/web-apps/009.html-disabled|TIME OUT|
> |./http/tests/xmlhttprequest/web-apps/016.html-disabled|TIME OUT|
> |./http/tests/xmlhttprequest/web-apps/017.html-disabled|KO (NO CRASH)|
> |./http/tests/xmlhttprequest/web-apps/recursion/001.html-disabled|KO  
> (NO CRASH)|
> |./svg/batik/text/textBiDi.svg-disabled|UNCLEAR (NO CRASH)|
> |./svg/batik/text/textDecoration.svg-disabled|TIME OUT|
> |./svg/custom/font-face-fallback.svg-disabled|UNCLEAR (NO CRASH)|
> |./svg/custom/js-font-test.svg-disabled|KO|
> |./svg/custom/use-on-symbol-inside-pattern.svg-disabled|OK ? (LOOKS  
> GOOD BUT DIFFS IN RESULT)|
> |./svg/dom/viewspec-parser.html-disabled|KO|
> |./svg/W3C-SVG-1.1/fonts-glyph-02-t.svg-disabled|TIME OUT|
> |./svg/W3C-SVG-1.1/resources/filters-comptran-01-f.svg-disabled| 
> UNCLEAR (NO CRASH)|
> |./svg/W3C-SVG-1.1/text-align-08-b.svg-disabled|UNCLEAR (NO CRASH)|
> |./svg/W3C-SVG-1.1/text-fonts-03-t.svg-disabled|UNCLEAR (NO CRASH)|
> |./svg/W3C-SVG-1.1/text-intro-01-t.svg-disabled|UNCLEAR (NO CRASH)|
> |./svg/W3C-SVG-1.1/text-intro-02-b.svg-disabled|UNCLEAR (LOOKS GOOD)|
> |./svg/W3C-SVG-1.1/text-intro-03-b.svg-disabled|UNCLEAR (LOOKS GOOD)|
> |./svg/W3C-SVG-1.1/text-intro-04-t.svg-disabled|UNCLEAR (LOOKS GOOD)|
>
> I suppose that for those OK I just need to do a traditional patch  
> and file a bug for it (I haven't found related tickets to these  
> disabled tests, may be I missed them?).

There isn't an existing bug, you can just file one if you have a patch  
to turn on. I suppose we could also use a master bug to represent  
turning on all disabled tests. Let me know if you file that one.

>
> For the others, I will try to get a closer look but if someone can  
> enlight me about those labelled as "UNCLEAR", that would be good :)
> Some of them end up with a hang report, are they supposed to be re- 
> enabled or not?

Anything that ends up with a hang report probably should not be re- 
enabled until we can stop the hang. If there are specific tests you  
are unsure of we can advise you if you mention which ones and show  
what the results look like.

Regards,
Maciej

>
>
>
> Thanks,
> Jean-Charles
>
>
>
> Le 14 juin 08 à 00:21, Maciej Stachowiak a écrit :
>
>>
>> Hi everyone,
>>
>> A number of the WebKit layout tests are disabled, presumably because
>> at some point they would hang or crash the test tool. It's likely  
>> some
>> of these actually work ok now, but whoever disabled them neglected to
>> turn them on later. I recently re-enabled some of the disabled tests.
>>
>> I think it would be a good exercise to go through these, test which
>> ones can be enabled, turn those back on, and document which ones  
>> still
>> hang or crash (and the reason why, if known).
>>
>> This would be a great task for someone relatively new to WebKit
>> development or testing, as it will teach you about our regression
>> testing system, which is a key part of how we do our work. So rather
>> than doing all the remaining 51 disabled tests myself, I am posting
>> the list here in case anyone else is interested.
>>
>>
>> $ find . -name "*-disabled"
>> ./css2.1/t1202-counter-10-b.html-disabled
>> ./css2.1/t1202-counters-10-b.html-disabled
>> ./dom/xhtml/level3/core/documentadoptnode22.xhtml-disabled
>> ./dom/xhtml/level3/core/documentnormalizedocument06.xhtml-disabled
>> ./dom/xhtml/level3/core/domimplementationregistry12.xhtml-disabled
>> ./dom/xhtml/level3/core/domimplementationregistry23.xhtml-disabled
>> ./dom/xhtml/level3/core/nodecomparedocumentposition14.xhtml-disabled
>> ./dom/xhtml/level3/core/nodecomparedocumentposition15.xhtml-disabled
>> ./dom/xhtml/level3/core/noderemovechild03.xhtml-disabled
>> ./dom/xhtml/level3/core/nodereplacechild06.xhtml-disabled
>> ./dom/xhtml/level3/core/nodereplacechild07.xhtml-disabled
>> ./dom/xhtml/level3/core/nodereplacechild08.xhtml-disabled
>> ./editing/execCommand/create-list-1.html-disabled
>> ./editing/input/attributed-substring-from-range-lines.html-disabled
>> ./editing/pasteboard/paste-empty-startcontainer.html-disabled
>> ./editing/selection/inconsistent-in-removeChildNode.html-disabled
>> ./editing/style/5091898.html-disabled
>> ./fast/canvas/toDataURL-supportedTypes.html-disabled
>> ./fast/css/css2-system-color.html-disabled
>> ./fast/dom/Window/window-resize-nan.html-disabled
>> ./fast/events/key-events-in-frame.html-disabled
>> ./fast/frames/iframe-scroll-page-up-down.html-disabled
>> ./fast/images/svg-background-crash-on-refresh.html-disabled
>> ./fast/js/kde/Date.html-disabled
>> ./fast/js/string-concatenate-outofmemory.html-disabled
>> ./fast/leaks/003.html-disabled
>> ./fast/loader/form-events-back-forward.html-disabled
>> ./http/tests/navigation/post-frames-goback1.html-disabled
>> ./http/tests/navigation/postredirect-frames-goback1.html-disabled
>> ./http/tests/navigation/success200-frames-goback.html-disabled
>> ./http/tests/navigation/success200-frames-reload.html-disabled
>> ./http/tests/navigation/success200-subframeload-goback.html-disabled
>> ./http/tests/xmlhttprequest/web-apps/006.html-disabled
>> ./http/tests/xmlhttprequest/web-apps/009.html-disabled
>> ./http/tests/xmlhttprequest/web-apps/016.html-disabled
>> ./http/tests/xmlhttprequest/web-apps/017.html-disabled
>> ./http/tests/xmlhttprequest/web-apps/recursion/001.html-disabled
>> ./svg/batik/text/textBiDi.svg-disabled
>> ./svg/batik/text/textDecoration.svg-disabled
>> ./svg/custom/font-face-fallback.svg-disabled
>> ./svg/custom/js-font-test.svg-disabled
>> ./svg/custom/use-on-symbol-inside-pattern.svg-disabled
>> ./svg/dom/viewspec-parser.html-disabled
>> ./svg/W3C-SVG-1.1/fonts-glyph-02-t.svg-disabled
>> ./svg/W3C-SVG-1.1/resources/filters-comptran-01-f.svg-disabled
>> ./svg/W3C-SVG-1.1/text-align-08-b.svg-disabled
>> ./svg/W3C-SVG-1.1/text-fonts-03-t.svg-disabled
>> ./svg/W3C-SVG-1.1/text-intro-01-t.svg-disabled
>> ./svg/W3C-SVG-1.1/text-intro-02-b.svg-disabled
>> ./svg/W3C-SVG-1.1/text-intro-03-b.svg-disabled
>> ./svg/W3C-SVG-1.1/text-intro-04-t.svg-disabled
>>
>> _______________________________________________
>> webkit-dev mailing list
>> webkit-dev at lists.webkit.org
>> http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
>
> -- 
> Join OWB team on freenode IRC, channel #owb
>
>
> Jean-Charles Verdié
> Pleyo, CTO			
> mobile: +33 (0)6 282 616 05
> skype: jcverdie
>
>
> <generic Pleyo signature.gif>
>
> _______________________________________________
> webkit-dev mailing list
> webkit-dev at lists.webkit.org
> http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev