[webkit-dev] Adding ENABLE_NAVIGATOR_BUILDTYPE to WebCore

[Adam Barth]

On Thu, Jun 7, 2012 at 11:48 AM, Konstantin Tokarev <annulen at yandex.ru> wrote:
>
>
> 07.06.2012, 21:07, "Annie Sullivan" <sullivan at chromium.org>:
>> Oops, forgot to reply-all.
>>
>> On Thu, Jun 7, 2012 at 9:05 AM, Annie Sullivan <sullivan at chromium.org> wrote:
>>
>>>  On Wed, Jun 6, 2012 at 4:59 PM, Darin Adler <darin at apple.com> wrote:
>>>>  Our past experience with this sort of thing at Apple is that it led to bugs we didn't find until after we shipped "final" software because they did not show up during earlier testing. Since then, we've gone out of our way to avoid differences, since one of the main things we want to do with non-final builds is to approximate the final releases as closely as possible to get useful testing.
>>>>
>>>>  To oversimplify, website developers notoriously make mistakes with these types of attributes doing things like version and OS checks, leading to website incompatibilities.
>>>  Yes, this is definitely a concern we have as well.
>>>>  Maybe you could make your case for the usefulness of the attribute?
>>>  The problem we're experiencing in Chrome is for sites that collect
>>>  usage data--it turns out there's a selection bias caused by users who
>>>  opt in to our canary, dev, and beta channels.
>>>
>>>  The specific case I'm looking at right now is sites collecting
>>>  performance data from their user base. We'd love for these sites to be
>>>  able to report regressions they see in Chrome's performance as early
>>>  as possible. But it turns out users on different channels actually
>>>  show different performance characteristics. Beta users seem to have
>>>  faster machines, for example.
>
> Are you sure that this differences are statistically significant?

Yes.

Adam


>>> So in order to compare two versions of
>>>  Chrome, we need to compare data from users on the same release
>>>  channel. So we'd like sites who collect performance information to be
>>>  able to collect the build type in order to do that comparison.
>>>
>>>  We considered a few alternatives:
>>>  1) Adding a marker in the user agent string that indicates the
>>>  channel. The problem with this is that there is a lot of very fragile
>>>  code in the wild which parses user agent strings and breaks at the
>>>  slightest change. For example, code that parses Opera 10 as Opera 1.
>>>  2) Updating the version number as Chrome advances from canary to dev
>>>  to beta to stable, or encoding the build type in one of the octets of
>>>  the version number. The problem with this is that there's a big
>>>  possibility of sites that do version checking accidentally turning off
>>>  features on some channels. There are also a lot of things that we *do*
>>>  want to track across release channels for a specific version, like
>>>  bugs. So changing the version number could cause confusion there.
>>>  3) Sending an "x-buildtype" header. If we do this on every request,
>>>  it's a lot of bloat. We can't restrict it to requests that send a
>>>  corresponding "x-tell-me-the-buildtype" request header because most
>>>  metrics collection libraries send their metrics in an image request,
>>>  so they can't send custom headers.
>>
>> _______________________________________________
>> webkit-dev mailing list
>> webkit-dev at lists.webkit.org
>> http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
>
> --
> Regards,
> Konstantin
> _______________________________________________
> webkit-dev mailing list
> webkit-dev at lists.webkit.org
> http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev