[webkit-dev] Fixing Accept-Language

[Darin Adler]

On Jun 28, 2005, at 11:12 AM, Alexey Proskuryakov wrote:

> 1) Yes, I intended to send the complete list of preferred languages.

 From the early days of the Safari project, we've seen problems with  
long Accept-Language header fields.

This server failed but started working once we set a maximum length  
of 255:

     http://www.ireland.travel.ie

These servers generated a "Server Error" with older versions of  
Safari; I believe they use "Netscape Directory Gateway":

     http://query.directory.cornell.edu/dsgw/pbhtml/
     http://directory.princeton.edu/
     http://co.stanford.edu/ds/lang/auth.html
     http://ldap.chapman.edu/ds/search
     http://calendar.gwu.edu/ds/search
     http://gun.teipir.gr/ds/csearch

Here's another site that failed:

     http://www.dirtypictureshow.com/

Any of these sites may have been changed.

If you include German in your list of languages, eBay will not allow  
access to certain items, see <http://docs.info.apple.com/article.html? 
artnum=107877>.

> 2) Send the same language codes that other browsers do (e.g.,  
> Safari sends "ru-ru" instead of "ru", and this breaks Outlook Web  
> Access, rdar://4076004).

Sure, we should definitely fix this.

> (such as, but not solely, Japanese always getting a huge weight,  
> breaking some Adobe pages)

Perhaps what you're referring to here is the fact that Japanese was  
always listed as second on the list. As far as I know, it always had  
a correct weight. That code was removed because the reason for it was  
finally fixed, but then before Tiger shipped we ended up "dumbing  
down" the string even more because of the requests above.

> The current one also has some drawbacks (http:// 
> bugzilla.opendarwin.org/show_bug.cgi?id=3510).

Good point. We probably went too far in "dumbing down" the string --  
designing a new string for maximum compatibility will be a challenge.  
Implementing it shouldn't be too hard, though.

> And since the default in Mozilla is two languages (en-us plus en),  
> the servers that only accept the simplest form probably wouldn't  
> work with Mozilla/Firefox, too?

I didn't know that was the Mozilla default.

I must admit that I have not "tested the Internet" to find all the  
sites that work or don't work. But I will say that there are some  
sites that work with IE and Safari, but not Mozilla, believe it or not!

> I absolutely agree that it's ultimately a matter of real world  
> compliance. Is it possible to disclose some of the examples that  
> led to the aforementioned change? That would make regression  
> testing a lot easier.

Those examples above are not necessarily a complete list, but I hope  
they help.

>> If something's broken about this header, I suggest fixing it in  
>> NSURLConnection; if there's a need for WebKit to get the header  
>> then  we can change NSURLConnection API so that you can query the  
>> header  value. Adding code in WebKit to work around the fact that  
>> a newer  NSURLConnection is not available yet sounds OK too.
>
> OK, I'll do the latter if we come to any conclusion as to what the  
> proper Accept-Language header should be.

We'll figure out what to do once that patch is submitted.

I want to be careful to not just put fixes into WebKit because that's  
open source. In most cases we don't want to work around broken things  
elsewhere in Mac OS X, but instead fix them.

     -- Darin