WebKit (or at least Chrome) is currently failing a bunch opera's tests located at: http://tc.labs.opera.com/apis/XMLHttpRequest/ It seems that it would be a good idea for us to make use of these tests in WebKit, so I was thinking of importing them into the codebase. This raises the question: Should we import the tests one at a time as we fix the bugs, or should we import the whole suite? The tests look like they will fit into the existing LayoutTests infrastructure pretty easily, but if we import the whole lot it would be good to figure out if it is possible to keep our copy in sync with Opera's, possibly along the lines of WebKitTools/Scripts/update-webgl-conformance-tests . Should they end up in a subdirectory of LayoutTests/http/tests? (They require php scripts on the server). Thanks, Luke
On Sep 20, 2010, at 9:34 PM, Luke Macpherson wrote:
WebKit (or at least Chrome) is currently failing a bunch opera's tests located at: http://tc.labs.opera.com/apis/XMLHttpRequest/
It seems that it would be a good idea for us to make use of these tests in WebKit, so I was thinking of importing them into the codebase. This raises the question: Should we import the tests one at a time as we fix the bugs, or should we import the whole suite? The tests look like they will fit into the existing LayoutTests infrastructure pretty easily, but if we import the whole lot it would be good to figure out if it is possible to keep our copy in sync with Opera's, possibly along the lines of WebKitTools/Scripts/update-webgl-conformance-tests .
Should they end up in a subdirectory of LayoutTests/http/tests? (They require php scripts on the server).
It would be a good idea to drop in the test suite wholesale. On the other hand, it's likely that most of this will turn into the official W3C test suite for the XMLHttpRequest spec. So we should either wait a bit, or be vigilant to avoid adding the same tests twice when we drop in the official test suite. One possibility would be to make this a subdirectory of http/tests/xmlhttprequest. Regards, Maciej
On Tue, Sep 21, 2010 at 2:55 PM, Maciej Stachowiak <mjs@apple.com> wrote:
On Sep 20, 2010, at 9:34 PM, Luke Macpherson wrote:
WebKit (or at least Chrome) is currently failing a bunch opera's tests located at: http://tc.labs.opera.com/apis/XMLHttpRequest/
It seems that it would be a good idea for us to make use of these tests in WebKit, so I was thinking of importing them into the codebase. This raises the question: Should we import the tests one at a time as we fix the bugs, or should we import the whole suite? The tests look like they will fit into the existing LayoutTests infrastructure pretty easily, but if we import the whole lot it would be good to figure out if it is possible to keep our copy in sync with Opera's, possibly along the lines of WebKitTools/Scripts/update-webgl-conformance-tests .
Should they end up in a subdirectory of LayoutTests/http/tests? (They require php scripts on the server).
It would be a good idea to drop in the test suite wholesale. On the other hand, it's likely that most of this will turn into the official W3C test suite for the XMLHttpRequest spec. So we should either wait a bit, or be vigilant to avoid adding the same tests twice when we drop in the official test suite.
One possibility would be to make this a subdirectory of http/tests/xmlhttprequest.
How about we create http/tests/xmlhttprequest/w3c-experimental or something like that? That can tide us over until the official version comes out, at which point, we can delete the w3c-experimental directory and just add a w3c directory. It would be nice if we could start fixing these things before they become part of the official test suite as a way of evaluating whether there are issues with the spec and/or test suite. Also, it seems to me like to does make sense to add an update-experimental-w3c-xhr-tests script or something until there is an official version. Ojan
On Sep 20, 2010, at 10:19 PM, Ojan Vafai wrote:
How about we create http/tests/xmlhttprequest/w3c-experimental or something like that? That can tide us over until the official version comes out, at which point, we can delete the w3c-experimental directory and just add a w3c directory.
It would be nice if we could start fixing these things before they become part of the official test suite as a way of evaluating whether there are issues with the spec and/or test suite.
That is in fact exactly what we should be doing at this stage of the standards process for XHR.
Also, it seems to me like to does make sense to add an update-experimental-w3c-xhr-tests script or something until there is an official version.
Indeed. And even after the test suite is official, it is likely to expand over time. We should also look over our own XHR tests to see if there are any that it would make sense to contribute to the W3C. Cheers, Maciej
Filed https://bugs.webkit.org/show_bug.cgi?id=46164 for the script to pull the tests into our repo. On Tue, Sep 21, 2010 at 3:27 PM, Maciej Stachowiak <mjs@apple.com> wrote:
On Sep 20, 2010, at 10:19 PM, Ojan Vafai wrote:
How about we create http/tests/xmlhttprequest/w3c-experimental or
something like that? That can tide us over until the official version comes out, at which point, we can delete the w3c-experimental directory and just add a w3c directory.
It would be nice if we could start fixing these things before they become
part of the official test suite as a way of evaluating whether there are issues with the spec and/or test suite.
That is in fact exactly what we should be doing at this stage of the standards process for XHR.
Also, it seems to me like to does make sense to add an
update-experimental-w3c-xhr-tests script or something until there is an official version.
Indeed. And even after the test suite is official, it is likely to expand over time. We should also look over our own XHR tests to see if there are any that it would make sense to contribute to the W3C.
Cheers, Maciej
I don't think that the suite is testing much besides what we already have regression tests for. For the most part, it just has different expectations. There doesn't seem to be any harm in importing it as a whole now, but looking over the failing tests, categorizing them and sending feedback to public-webapps working groups is more important at this point. Jian Li from Chromium team is currently filing bugs for tests that fail in WebKit. - WBR, Alexey Proskuryakov 20.09.2010, в 22:34, Ojan Vafai написал(а):
Filed https://bugs.webkit.org/show_bug.cgi?id=46164 for the script to pull the tests into our repo.
On Tue, Sep 21, 2010 at 3:27 PM, Maciej Stachowiak <mjs@apple.com> wrote:
On Sep 20, 2010, at 10:19 PM, Ojan Vafai wrote:
How about we create http/tests/xmlhttprequest/w3c-experimental or something like that? That can tide us over until the official version comes out, at which point, we can delete the w3c-experimental directory and just add a w3c directory.
It would be nice if we could start fixing these things before they become part of the official test suite as a way of evaluating whether there are issues with the spec and/or test suite.
That is in fact exactly what we should be doing at this stage of the standards process for XHR.
Also, it seems to me like to does make sense to add an update-experimental-w3c-xhr-tests script or something until there is an official version.
Indeed. And even after the test suite is official, it is likely to expand over time. We should also look over our own XHR tests to see if there are any that it would make sense to contribute to the W3C.
Cheers, Maciej
_______________________________________________ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Yes, I have just gone through all the tests in this suite and categorized and filed some WebKit bugs. Some other test failures seem to be due to something wrong with tests themselves. I am pinging Anne about these issues. Thanks, Jian On Mon, Sep 20, 2010 at 11:43 PM, Alexey Proskuryakov <ap@webkit.org> wrote:
I don't think that the suite is testing much besides what we already have regression tests for. For the most part, it just has different expectations.
There doesn't seem to be any harm in importing it as a whole now, but looking over the failing tests, categorizing them and sending feedback to public-webapps working groups is more important at this point. Jian Li from Chromium team is currently filing bugs for tests that fail in WebKit.
- WBR, Alexey Proskuryakov
20.09.2010, в 22:34, Ojan Vafai написал(а):
Filed https://bugs.webkit.org/show_bug.cgi?id=46164 for the script to pull the tests into our repo.
On Tue, Sep 21, 2010 at 3:27 PM, Maciej Stachowiak <mjs@apple.com> wrote:
On Sep 20, 2010, at 10:19 PM, Ojan Vafai wrote:
How about we create http/tests/xmlhttprequest/w3c-experimental or
something like that? That can tide us over until the official version comes out, at which point, we can delete the w3c-experimental directory and just add a w3c directory.
It would be nice if we could start fixing these things before they
become part of the official test suite as a way of evaluating whether there are issues with the spec and/or test suite.
That is in fact exactly what we should be doing at this stage of the standards process for XHR.
Also, it seems to me like to does make sense to add an
update-experimental-w3c-xhr-tests script or something until there is an official version.
Indeed. And even after the test suite is official, it is likely to expand over time. We should also look over our own XHR tests to see if there are any that it would make sense to contribute to the W3C.
Cheers, Maciej
_______________________________________________ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
_______________________________________________ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
On Tue, Sep 21, 2010 at 8:43 AM, Alexey Proskuryakov <ap@webkit.org> wrote:
I don't think that the suite is testing much besides what we already have regression tests for. For the most part, it just has different expectations.
FWIW, if you find cases the test suite is missing I'm happy to add them. The only reason the W3C is not hosting this test suite at the moment is because they cannot handle server-side scripts on the test server at the moment. PLH is looking into a solution on their side as I understand things. -- Anne van Kesteren http://annevankesteren.nl/
participants (6)
-
Alexey Proskuryakov
-
Anne van Kesteren
-
Jian Li
-
Luke Macpherson
-
Maciej Stachowiak
-
Ojan Vafai