[webkit-dev] XMLHttpRequest and readyState==3

[Christopher Allen]

We are in the process of trying to make SynchroEdit
<www.synchroedit.com> work with Safari 3.0, with the eventual goal of
seeing if it is possible to synchroedit documents using an iPhone.

SynchroEdit is an open-source simultaneous web editor (in the style of
SubEthaEdit) that currently only works in Mozilla/Firefox/Camino
browsers. The technique it uses is to synchronize the DOM tree between
multiple machines, so it potentially has some interesting uses beyond
synchronous editing.

For the most part, things seem to be working well, but one feature we
make use of heavily is the readyState==3 effect which is supported by
Firefox. We have read that this feature is unsupported by the WHATWG
spec, and that it will most likely not be implemented into Safari.

The reason why we need this functionality is that we are building
highly-interactive web applications, and we have found that we need to
cut down on the number of connections to and from the HTTP server, in
order to gain performance. Opening a new connection for every
input/output request would slow the browser, and the server, down
substantially. As the browser is the initiator of all requests,
pending output (from server to client) requests can only be determined
by repeatedly polling the server for updates, or by keeping some form
of streaming connection using a technique like the readystate
technique as described below.

Basically, the Firefox technique involves reading the
<XMLHttpRequest>.responseText value while the request itself stays in
readyState==3 (loading). The responseText buffer updates as new input
appears, and in this manner a "streaming data connection" is achieved.

Whenever new data is sent to the browser from the server via the
XMLHttpRequest, Firefox triggers another "readystatechange", even
though the readyState changes "from 3 to 3". This lets coders put in
code akin to "onRead" for a regular TCP connection interface, but in
AJAX apps. This technique has been hugely effective not only for
SynchroEdit, but for other applications that keep a continuous
connection to the server such as interactive chat applications.

As an alternative approach that might work with WebKit, which we've
tested in r23558, is to instead make a setInterval() "check_data"
style function, which simply checks the size of the responseText
string for updates. Example code* would look like:

var rt_index = 0;
var req = null;
function itv_check_data() {
	var curr_index = req.responseText.length;
	if (rt_index == curr_index) return; // no new data
	var s = req.responseText.substring(rt_index);
	rt_index = curr_index;
	alert("new input: " + s);
}

function start_reading() {
	req = new_XHR(); // new XMLHttpRequest() in FF
	req.open("POST", "foo.cgi", true);
	req.send("test");
	setInterval("itv_check_data()", 333); // run itv_check_data thrice/sec
}

* The above code is untested and may be bugged.

This alternative technique worked fine in Firefox (but had a higher
latency then the readystate==3 technique), but it does not seem to
work in WebKit r23558 either. It appears WebKit is not setting the
responseText variable until the request has completed. Is there a way
to access this text prior to state 4?

Another possible approach which we are investigating is to close the
connection server-side, whenever new data is available. The connection
is immediately reopened and stays open until new data is available or
until the connection times out. This would remove the need for
readyState==3 but may have too large an impact on performance.

Any suggestions or ideas for us?

-- Christopher Allen

P.S. We think SynchroEdit would be an interested test-bed for WebKit
as it makes extensive use of designMode-enabled frames and our the DOM
synchronization tends to expose bugs in DOM. Let us know if you are
interested in helping us port it to support WebKit/Safari.