[webkit-dev] User agent woes

[Michael Catanzaro]

Hi,

You're probably aware that WebKitGTK+ has user agent quirks to make 
various popular websites work, most notably google.com.  For our list 
of quirks, see:

https://trac.webkit.org/browser/webkit/trunk/Source/WebCore/platform/UserAgentQuirks.cpp

Recently we had two major bugs caused by these quirks:

https://bugs.webkit.org/show_bug.cgi?id=171603 (New YouTube displays 
only white page)
https://bugs.webkit.org/show_bug.cgi?id=171770 (Google login fails on 
white page)

In the case of the YouTube bug, I decided to just remove the quirk. 
This case was pretty much a nightmare scenario, because Google decided 
that a random subset of users would receive the new version of YouTube 
long in advance of the rest of us. This means that we had a portion of 
our users complaining that YouTube was "broken" for weeks when we were 
unable to reproduce the issue (and had no way to do so; how could we 
have guessed it was some trial program?).

Anyway, removing the quirk is not a good option for the generic 
google.com quirk in the second bug. If we use a standard user agent, we 
receive a crap unusable 1990s version of Google Calendar, a high source 
of user complaints, and also the awesome 3D earth mode in Google Maps 
that our users expect is unavailable. This makes users think that 
WebKit is bad. I found a solution, but I know it's temporary; give it a 
few more months and Google will break us again, no doubt.

User agent is an extremely demotivating, never-ending game, and it's by 
far our biggest web compatibility problem. It almost feels as if Google 
is deliberately trying to break WebKit, which I know is not true as 
they don't care either way about us... but they do know full well that 
basing logic off of user agent checks serves to harm less-popular 
browsers, so it's hardly unintentional. I cannot think of any aspect of 
WebKit development less gratifying than maintaining our user agent 
quirk list, nor any bigger user agent offender than Google.

For a while I thought there was nothing we could do, but now I have an 
idea: Safari could adopt the same (or lightly-adapted) user agent 
quirks that we use for WebKitGTK+. Of course, only the small handful of 
websites to which we currently need to send user agent quirks would be 
affected by this change: Google, Yahoo, Slack, Whatsapp, Typekit, and 
some Chinese site Taobao. Now, this would do no good for Safari, but it 
would be a huge help to us as it would ensure that if these user agent 
offenders attempt to degrade functionality for browsers not on their 
lists, they will have to at least treat all WebKit browsers equally. 
Presumably they test to make sure their websites work in Safari, so 
that should make this situation much better for other ports. And if we 
run into problems, we at least know the change is limited to this small 
set of websites.

I think the existing quirks would be fine for Safari with minimal 
changes. We would definitely need to add some #if OS(UNIX) && 
!PLATFORM(COCOA) guards inside urlRequiresLinuxDesktopPlatform(), and 
put if !PLATFORM(IOS) around the quirks that are designed to turn off 
mobile versions of websites. This would only be a small amount of work 
to set up, and it would only affect the handful of websites that we 
have identified as problematic. Would Apple be willing to allow this?

Alternatively, we could use the nuclear option and add a Chrome version 
component, similar to how Chrome includes a Safari version in its user 
agent. (Bonus: that will shut up Google's "switch to Chrome" ads for a 
couple weeks until they figure it out.) Edge already includes a Chrome 
version. This would undoubtedly be better for the web as a whole, but 
it would surely also introduce serious short-term compatibility 
problems, as an unknown number of websites would likely break horribly 
in Safari (and some known websites, e.g. YouTube and Google Hangouts), 
so that's a pretty tough ask. Making it more difficult for websites to 
send us crap versions of web pages would be good for WebKit in the long 
term, but it's way safer to start only on quirks for specific sites.

What do you think?

Michael