<html>
<head>
<base href="https://bugs.webkit.org/" />
</head>
<body><table border="1" cellspacing="0" cellpadding="8">
<tr>
<th>Bug ID</th>
<td><a class="bz_bug_link
bz_status_NEW "
title="NEW - Many of the expected results from imported/w3c are wrong (contain FAIL strings)"
href="https://bugs.webkit.org/show_bug.cgi?id=161003">161003</a>
</td>
</tr>
<tr>
<th>Summary</th>
<td>Many of the expected results from imported/w3c are wrong (contain FAIL strings)
</td>
</tr>
<tr>
<th>Classification</th>
<td>Unclassified
</td>
</tr>
<tr>
<th>Product</th>
<td>WebKit
</td>
</tr>
<tr>
<th>Version</th>
<td>WebKit Nightly Build
</td>
</tr>
<tr>
<th>Hardware</th>
<td>Unspecified
</td>
</tr>
<tr>
<th>OS</th>
<td>Unspecified
</td>
</tr>
<tr>
<th>Status</th>
<td>NEW
</td>
</tr>
<tr>
<th>Severity</th>
<td>Normal
</td>
</tr>
<tr>
<th>Priority</th>
<td>P2
</td>
</tr>
<tr>
<th>Component</th>
<td>Tools / Tests
</td>
</tr>
<tr>
<th>Assignee</th>
<td>webkit-unassigned@lists.webkit.org
</td>
</tr>
<tr>
<th>Reporter</th>
<td>clopez@igalia.com
</td>
</tr>
<tr>
<th>CC</th>
<td>achristensen@apple.com, bugs-noreply@webkitgtk.org, lforschler@apple.com, youennf@gmail.com
</td>
</tr>
<tr>
<th>Blocks</th>
<td>160446
</td>
</tr></table>
<p>
<div>
<pre>I have been checking that many (~328) of the expected results from imported/w3c contain FAIL strings. I understand this means that this tests are actually failing.
$ find LayoutTests/imported/w3c -type f -name '*-expected.txt'|xargs grep -l FAIL|wc -l
328
I wonder:
Wouldn't it have been better to mark this tests as failing in the TestExpectations file instead of uploading a -expected.txt file with FAIL results?
On the GTK+ port some of this tests PASS, but the tooling thing they are failing because the expected file is different than the one that it should be.
Check this example:
Running imported/w3c/web-platform-tests/XMLHttpRequest/send-entity-body-empty.htm on the GTK port I get:
Regressions: Unexpected text-only failures (1)
imported/w3c/web-platform-tests/XMLHttpRequest/send-entity-body-empty.htm [ Failure ]
But then, checking the diff result I see:
--- /home/clopez/webkit/webkit/layout-test-results/imported/w3c/web-platform-tests/XMLHttpRequest/send-entity-body-empty-expected.txt
+++ /home/clopez/webkit/webkit/layout-test-results/imported/w3c/web-platform-tests/XMLHttpRequest/send-entity-body-empty-actual.txt
@@ -1,5 +1,5 @@
PASS XMLHttpRequest: send("") - empty entity body (POST)
PASS XMLHttpRequest: send("") - empty entity body (PUT)
-FAIL XMLHttpRequest: send("") - empty entity body (HEAD) assert_equals: expected "NO" but got "0"
+PASS XMLHttpRequest: send("") - empty entity body (HEAD)
Which means that the test is passing on the GTK+ port, but not on the port where the result was generated (Mac?).
I think the right thing to do is to mark this tests as failing on the corresponding TestExpectations, rather than committing as expected results the result of a test that FAILS.</pre>
</div>
</p>
<hr>
<span>You are receiving this mail because:</span>
<ul>
<li>You are the assignee for the bug.</li>
</ul>
</body>
</html>