[webkit-dev] User agent woes

[Maciej Stachowiak]

> On May 6, 2017, at 5:19 PM, Michael Catanzaro <mcatanzaro at igalia.com> wrote:
> 
> Hi,
> 
> You're probably aware that WebKitGTK+ has user agent quirks to make various popular websites work, most notably google.com.  For our list of quirks, see:
> 
> https://trac.webkit.org/browser/webkit/trunk/Source/WebCore/platform/UserAgentQuirks.cpp
> 
> Recently we had two major bugs caused by these quirks:
> 
> https://bugs.webkit.org/show_bug.cgi?id=171603 (New YouTube displays only white page)
> https://bugs.webkit.org/show_bug.cgi?id=171770 (Google login fails on white page)
> 
> In the case of the YouTube bug, I decided to just remove the quirk. This case was pretty much a nightmare scenario, because Google decided that a random subset of users would receive the new version of YouTube long in advance of the rest of us. This means that we had a portion of our users complaining that YouTube was "broken" for weeks when we were unable to reproduce the issue (and had no way to do so; how could we have guessed it was some trial program?).
> 
> Anyway, removing the quirk is not a good option for the generic google.com quirk in the second bug. If we use a standard user agent, we receive a crap unusable 1990s version of Google Calendar, a high source of user complaints, and also the awesome 3D earth mode in Google Maps that our users expect is unavailable. This makes users think that WebKit is bad. I found a solution, but I know it's temporary; give it a few more months and Google will break us again, no doubt.
> 
> User agent is an extremely demotivating, never-ending game, and it's by far our biggest web compatibility problem. It almost feels as if Google is deliberately trying to break WebKit, which I know is not true as they don't care either way about us... but they do know full well that basing logic off of user agent checks serves to harm less-popular browsers, so it's hardly unintentional. I cannot think of any aspect of WebKit development less gratifying than maintaining our user agent quirk list, nor any bigger user agent offender than Google.
> 
> For a while I thought there was nothing we could do, but now I have an idea: Safari could adopt the same (or lightly-adapted) user agent quirks that we use for WebKitGTK+. Of course, only the small handful of websites to which we currently need to send user agent quirks would be affected by this change: Google, Yahoo, Slack, Whatsapp, Typekit, and some Chinese site Taobao. Now, this would do no good for Safari, but it would be a huge help to us as it would ensure that if these user agent offenders attempt to degrade functionality for browsers not on their lists, they will have to at least treat all WebKit browsers equally. Presumably they test to make sure their websites work in Safari, so that should make this situation much better for other ports. And if we run into problems, we at least know the change is limited to this small set of websites.
> 
> I think the existing quirks would be fine for Safari with minimal changes. We would definitely need to add some #if OS(UNIX) && !PLATFORM(COCOA) guards inside urlRequiresLinuxDesktopPlatform(), and put if !PLATFORM(IOS) around the quirks that are designed to turn off mobile versions of websites. This would only be a small amount of work to set up, and it would only affect the handful of websites that we have identified as problematic. Would Apple be willing to allow this?
> 
> Alternatively, we could use the nuclear option and add a Chrome version component, similar to how Chrome includes a Safari version in its user agent. (Bonus: that will shut up Google's "switch to Chrome" ads for a couple weeks until they figure it out.) Edge already includes a Chrome version. This would undoubtedly be better for the web as a whole, but it would surely also introduce serious short-term compatibility problems, as an unknown number of websites would likely break horribly in Safari (and some known websites, e.g. YouTube and Google Hangouts), so that's a pretty tough ask. Making it more difficult for websites to send us crap versions of web pages would be good for WebKit in the long term, but it's way safer to start only on quirks for specific sites.
> 
> What do you think?

It's unlikely that we'll change the user agent that Safari (or other macOS or iOS WebKit clients) send for websites that already work correctly in Safari. I can think of at least four reasons:

(1) In general we'll only add site-specific hacks for prominent websites that are actively broken in Safari, and we do evangelism in parallel to be able remove them.
(2) We don't want the UA string environment observed by sites to become even more complicated.
(3) Changing the UA string is high risk. Even updating our own version number is a risky proposition. To the point that we've seriously considered permanently freezing at one UA strong.
(4) It could significantly throw off browser usage share statistics.

We do understand that user agent checks are a frequently cause of gratuitous site breakage. It's something we often face with Safari as well. I would recommend that other browsers using Trunk WebKit should mostly copy the Safari UA string (including the Safari part and perhaps the Mac part). 

You mentioned that for some sites, that wouldn't work. I think it's reasonable to have more specific exceptions just for those, and we can review them together. Specifically, for Google Hangouts, you mention that claiming to be Mac Safari will offer a non-working plugin. However, it looks like WebKitGTK+ pretends to be Linux Firefox for Google sites instead. It seems very likely that a Linux Firefox UA string would break Google Hangouts in Mac Safari for pretty much the same reason.

It's not clear to me what the issues are with typekit or slack, or why you claim to be Chrome instead of Mac Safari on those. Maybe claiming to be Mac Safari would reduce future breakage. Or if not, it would be interesting to know what the issue is.

We may also be able to reach out to some of the problem websites or help you get contact info.

Regards,
Maciej