[webkit-dev] DOM L1 Core tests analysis

[Curt Arnold]

I was able to drop the self-hosted HTML tests for DOM Level 1 Core  
into the WebCore/layout-tests directory.  The first
run created the "-expected.txt" files and on a second run all the  
tests "succeeded" (in the sense that the response had
not changed from the first run).  Doing a grep on the expected files  
for "failure" and "error" identified the following tests which I have  
added a brief analysis of the cause of the failure.

Safari seems to have serious issues with the self-hosted XHTML tests  
which appears to be due to a failure to pick up the entity  
declarations in the document type declaration, however I haven't  
tried to track those down.  I don't know whether I should expect  
WebKit to have any support for XHTML.

I will later see what happens when the L2 Core and L2 HTML self- 
hosted tests are dropped in.

Manipulation of attributes and attribute child nodes appears to be  
the most significant problem area, however these are problem areas  
for many implementations.  The apparent lack of an implementation for  
Document.cloneNode causes all of the multi-document tests the self- 
tests implementation depends on that method to create the second  
document.   hc_attrcreatetextnode, hc_nodeappendchildinvalidnodetype,  
hc_nodeinsertbeforeinvalidnodetype,
hc_nodereplacechildinvalidnodetype seem like the first issues to attack.




hc_attrappendchild5-expected.txt:                text run at (2,2)  
width 30: "error"
hc_attrappendchild5-expected.txt:                text run at (2,2)  
width 145: "TypeError - Null value"
hc_attrinsertbefore6-expected.txt:                text run at (2,2)  
width 30: "error"
hc_attrinsertbefore6-expected.txt:                text run at (2,2)  
width 145: "TypeError - Null value"
hc_elementwrongdocumenterr-expected.txt:                text run at  
(2,2) width 30: "error"
hc_elementwrongdocumenterr-expected.txt:                text run at  
(2,2) width 145: "TypeError - Null value"
hc_namednodemapwrongdocumenterr-expected.txt:                text run  
at (2,2) width 30: "error"
hc_namednodemapwrongdocumenterr-expected.txt:                text run  
at (2,2) width 145: "TypeError - Null value"
hc_nodeappendchildnewchilddiffdocument-expected.txt:                 
text run at (2,2) width 30: "error"
hc_nodeappendchildnewchilddiffdocument-expected.txt:                 
text run at (2,2) width 145: "TypeError - Null value"
hc_nodeinsertbeforenewchilddiffdocument-expected.txt:                 
text run at (2,2) width 30: "error"
hc_nodeinsertbeforenewchilddiffdocument-expected.txt:                 
text run at (2,2) width 145: "TypeError - Null value"
hc_nodereplacechildnewchilddiffdocument-expected.txt:                 
text run at (2,2) width 30: "error"
hc_nodereplacechildnewchilddiffdocument-expected.txt:                 
text run at (2,2) width 145: "TypeError - Null value"


Document.cloneNode(true) returns null.  Document.cloneNode(true) is  
required to support multiple (identical)
document tests in the self-hosted tests (the JSUnit tests use  
separate IFRAME elements loading the same URL).
Opera and Mozilla will both fail the self-hosted flavor of these test  
in the same manner as Safari.
In the JSUnit tests, IE and Mozilla will not throw the specified  
WRONG_DOCUMENT_ERR exception.

documentinvalidcharacterexceptioncreateentref- 
expected.txt:                text run at (2,2) width 40: "failure"
documentinvalidcharacterexceptioncreateentref1- 
expected.txt:                text run at (2,2) width 40: "failure"
documentinvalidcharacterexceptioncreatepi- 
expected.txt:                text run at (2,2) width 40: "failure"
documentinvalidcharacterexceptioncreatepi1- 
expected.txt:                text run at (2,2) width 40: "failure"
hc_nodevalue03-expected.txt:                text run at (2,2) width  
40: "failure"
hc_attrgetvalue2-expected.txt:                text run at (2,2) width  
40: "failure"


Attempted creation of entity reference and processing instruction.
HTML documents should throw a NOT_SUPPORTED_ERR.  Safari appears to  
return a Node.


hc_attrappendchild1-expected.txt:                text run at (2,2)  
width 40: "failure"

Text node added to attribute node.  High failure test, passed by Xerces,
failed by Crimson, DOM4J, IE6, Mozilla.
Failure consistent with IE 6, append appears to be successful but  
attribute value
not updated.  Mozilla throws "Modifications not allowed" exception.


hc_attrappendchild2-expected.txt:                text run at (2,2)  
width 40: "failure"

Attempt to add element as child of attribute.  HIERARCHY_REQUEST_ERR  
should be thrown, but not thrown by IE 6 or Mozilla.

hc_attrappendchild3-expected.txt:                text run at (2,2)  
width 40: "failure"

Document fragment containing text nodes added to attribute node.   
Same comments as appendchild1.


hc_attrappendchild4-expected.txt:                text run at (2,2)  
width 40: "failure"

Creation of CDATASection in HTML document expected to throw  
NOT_SUPPORTED_ERR.  Passed by Mozilla and IE.


hc_attrappendchild6-expected.txt:                text run at (2,2)  
width 40: "failure"

Similar to appendchild1.


hc_attrchildnodes1-expected.txt:                text run at (2,2)  
width 40: "failure"
hc_attrfirstchild-expected.txt:                text run at (2,2)  
width 40: "failure"
hc_attrlastchild-expected.txt:                text run at (2,2) width  
40: "failure"
hc_attrhaschildnodes-expected.txt:                text run at (2,2)  
width 40: "failure"


Attribute.childNodes expected to contain one text node.  Mozilla  
passes, IE returns null for
Attribute.childNodes, Safari returns empty node list.  Java  
implementations typically
fabricate the text node for attributes only on demand.


hc_attrchildnodes2-expected.txt:                text run at (2,2)  
width 40: "failure"

Text node appended to attribute node, similar to hc_attrappendchild1  
but then checks
Attribute.childNodes.  IE returns null for Attribute.childNodes,  
Mozilla fails with
NO_MODIFICATION_ALLOWED_ERR, Safari child node lists contains only  
the newly
added text node.


hc_attrclonenode1-expected.txt:                text run at (2,2)  
width 40: "failure"

High failure test.  Similar to hc_attrappendchild1 but with an  
additional clone.

hc_attrcreatetextnode-expected.txt:                text run at (2,2)  
width 40: "failure"

Of tested implementations., only Safari fails   Sets  
Attribute.nodeValue = "Y&ent1;".
Safari interprets "&ent1;" as an entity reference when it should have  
been interpreted as
a literal.



hc_attrinsertbefore1-expected.txt:                text run at (2,2)  
width 40: "failure"
hc_attrinsertbefore2-expected.txt:                text run at (2,2)  
width 40: "failure"
hc_attrinsertbefore3-expected.txt:                text run at (2,2)  
width 40: "failure"
hc_attrinsertbefore4-expected.txt:                text run at (2,2)  
width 40: "failure"
hc_attrinsertbefore5-expected.txt:                text run at (2,2)  
width 40: "failure"
hc_attrinsertbefore7-expected.txt:                text run at (2,2)  
width 40: "failure"


Likely all variants of some hc_attrappendchild.


Similar to hc_attrchildnodes1.

hc_attrnormalize-expected.txt:                text run at (2,2) width  
40: "failure"

Similar to an hc_attrappendchild, then calls normalize.

hc_attrremovechild1-expected.txt:                text run at (2,2)  
width 40: "failure"
hc_attrreplacechild1-expected.txt:                text run at (2,2)  
width 40: "failure"
hc_attrreplacechild2-expected.txt:                text run at (2,2)  
width 40: "failure"

Similar to the hc_attrappendchilds.


hc_attrsetvalue1-expected.txt:                text run at (2,2) width  
40: "failure"

Builds on hc_attrfirstchild which fails on IE and Safari.

hc_attrsetvalue2-expected.txt:                text run at (2,2) width  
40: "failure"

Builds on hc_attrfirstchild and high failure test hc_attrappendchild1.


hc_elementnormalize2-expected.txt:                text run at (2,2)  
width 40: "failure"

High failure test.  IE, Mozilla and Safari all fail this test in  
different ways.  Has aspects
of hc_attrfirstchild and hc_attrappendchild1.


hc_elementremoveattribute-expected.txt:                text run at  
(2,2) width 40: "failure"

IE 6, Mozilla and Opera all fail this test the same way, but no Java  
implementation fails.  Test
removes an attribute and then calls Element.getAttribute to get its  
value.  Browsers return null,
though they would have returned "" if the attribute did not appear in  
the original document.

hc_nodeappendchildinvalidnodetype-expected.txt:                text  
run at (2,2) width 40: "failure"
hc_nodeinsertbeforeinvalidnodetype-expected.txt:                text  
run at (2,2) width 40: "failure"
hc_nodereplacechildinvalidnodetype-expected.txt:                text  
run at (2,2) width 40: "failure"

Attempts to add an attribute into node list of element should result  
in HIERARCHY_REQUEST_ERR.
Safari fails in similar manner to IE 6.  No other surveyed  
implementation fails.