Hi all, I've started working on a fix to what WebKit sends in Accept-Language HTTP header, and immediately noticed that it doesn't send anything, relying on an NSURLConnection default. This is strange, because WebKit needs to know what is being sent in this header for proper caching (apparently, that's broken now, though I haven't performed real-life tests, and may have overlooked something). I see several approaches to the problem, and would like to discuss their strong and weak sides before moving on with coding. 1. Do nothing, wait for a fix in NSURLConnection. Downsides: may take a lot of time; not clear what to do with caching. 2. Add a proper Accept-Language header when creating an NSURLRequest, thus overriding the default. Downsides: duplicates work that should be eventually done in NSURLConnection, yet not in a cross-platform way. 3. Generate the Accept-Language header in a cross-platform manner. Downsides: looks like a rather serious rework of code, which I do not feel qualified to attempt yet; there's no automatic way to generate the whole list of preferred languages on other platforms. If it is believed to be an adequate solution, I could do #2. - WBR, Alexey Proskuryakov
On Jun 27, 2005, at 10:03 PM, Alexey Proskuryakov wrote:
1. Do nothing, wait for a fix in NSURLConnection. Downsides: may take a lot of time; not clear what to do with caching.
2. Add a proper Accept-Language header when creating an NSURLRequest, thus overriding the default. Downsides: duplicates work that should be eventually done in NSURLConnection, yet not in a cross-platform way.
What is a "proper" Accept-Language header? Older versions of NSURLConnection used to send the complete list of languages specified in the System Preferences. This created major compatibility headaches with many websites, including sites assuming the user was coming from Germany just because German was one of the listed languages, and web servers that choked on anything but the simplest . We ended up dumbing down this header to match more closely what other browsers send to resolve the compatibility problems. If something's broken about this header, I suggest fixing it in NSURLConnection; if there's a need for WebKit to get the header then we can change NSURLConnection API so that you can query the header value. Adding code in WebKit to work around the fact that a newer NSURLConnection is not available yet sounds OK too. -- Darin
On Tue, 28 Jun 2005 10:24:15 -0700 Darin Adler <darin@apple.com> wrote:
2. Add a proper Accept-Language header when creating an NSURLRequest, thus overriding the default. Downsides: duplicates work that should be eventually done in NSURLConnection, yet not in a cross-platform way.
What is a "proper" Accept-Language header? Older versions of NSURLConnection used to send the complete list of languages specified in the System Preferences.
By proper, I meant: 1) Yes, I intended to send the complete list of preferred languages. 2) Send the same language codes that other browsers do (e.g., Safari sends "ru-ru" instead of "ru", and this breaks Outlook Web Access, rdar://4076004).
This created major compatibility headaches with many websites, including sites assuming the user was coming from Germany just because German was one of the listed languages, and web servers that choked on anything but the simplest . We ended up dumbing down this header to match more closely what other browsers send to resolve the compatibility problems.
I see... I remember that the old implementation had several other issues (such as, but not solely, Japanese always getting a huge weight, breaking some Adobe pages). The current one also has some drawbacks (http://bugzilla.opendarwin.org/show_bug.cgi?id=3510). And since the default in Mozilla is two languages (en-us plus en), the servers that only accept the simplest form probably wouldn't work with Mozilla/Firefox, too? I absolutely agree that it's ultimately a matter of real world compliance. Is it possible to disclose some of the examples that led to the aforementioned change? That would make regression testing a lot easier.
If something's broken about this header, I suggest fixing it in NSURLConnection; if there's a need for WebKit to get the header then we can change NSURLConnection API so that you can query the header value. Adding code in WebKit to work around the fact that a newer NSURLConnection is not available yet sounds OK too.
OK, I'll do the latter if we come to any conclusion as to what the proper Accept-Language header should be. - WBR, Alexey Proskuryakov
On Jun 28, 2005, at 11:12 AM, Alexey Proskuryakov wrote:
1) Yes, I intended to send the complete list of preferred languages.
From the early days of the Safari project, we've seen problems with long Accept-Language header fields. This server failed but started working once we set a maximum length of 255: http://www.ireland.travel.ie These servers generated a "Server Error" with older versions of Safari; I believe they use "Netscape Directory Gateway": http://query.directory.cornell.edu/dsgw/pbhtml/ http://directory.princeton.edu/ http://co.stanford.edu/ds/lang/auth.html http://ldap.chapman.edu/ds/search http://calendar.gwu.edu/ds/search http://gun.teipir.gr/ds/csearch Here's another site that failed: http://www.dirtypictureshow.com/ Any of these sites may have been changed. If you include German in your list of languages, eBay will not allow access to certain items, see <http://docs.info.apple.com/article.html? artnum=107877>.
2) Send the same language codes that other browsers do (e.g., Safari sends "ru-ru" instead of "ru", and this breaks Outlook Web Access, rdar://4076004).
Sure, we should definitely fix this.
(such as, but not solely, Japanese always getting a huge weight, breaking some Adobe pages)
Perhaps what you're referring to here is the fact that Japanese was always listed as second on the list. As far as I know, it always had a correct weight. That code was removed because the reason for it was finally fixed, but then before Tiger shipped we ended up "dumbing down" the string even more because of the requests above.
The current one also has some drawbacks (http:// bugzilla.opendarwin.org/show_bug.cgi?id=3510).
Good point. We probably went too far in "dumbing down" the string -- designing a new string for maximum compatibility will be a challenge. Implementing it shouldn't be too hard, though.
And since the default in Mozilla is two languages (en-us plus en), the servers that only accept the simplest form probably wouldn't work with Mozilla/Firefox, too?
I didn't know that was the Mozilla default. I must admit that I have not "tested the Internet" to find all the sites that work or don't work. But I will say that there are some sites that work with IE and Safari, but not Mozilla, believe it or not!
I absolutely agree that it's ultimately a matter of real world compliance. Is it possible to disclose some of the examples that led to the aforementioned change? That would make regression testing a lot easier.
Those examples above are not necessarily a complete list, but I hope they help.
If something's broken about this header, I suggest fixing it in NSURLConnection; if there's a need for WebKit to get the header then we can change NSURLConnection API so that you can query the header value. Adding code in WebKit to work around the fact that a newer NSURLConnection is not available yet sounds OK too.
OK, I'll do the latter if we come to any conclusion as to what the proper Accept-Language header should be.
We'll figure out what to do once that patch is submitted. I want to be careful to not just put fixes into WebKit because that's open source. In most cases we don't want to work around broken things elsewhere in Mac OS X, but instead fix them. -- Darin
participants (2)
-
Alexey Proskuryakov
-
Darin Adler