[webkit-dev] Existing metrics for deprecated features

[Adam Barth]

On Thu, Sep 13, 2012 at 10:58 PM, Pavel Feldman <pfeldman at chromium.org> wrote:
> On Fri, Sep 14, 2012 at 4:13 AM, Adam Barth <abarth at webkit.org> wrote:
>> In connection with the thread on webkitPostMessage, I took a look at
>> what sorts of metrics we have currently for depreciated features.  We
>> have high quality data for two groups of APIs:
>>
>> For mutation events, we measure what percent of Documents have
>> listeners for various kinds of mutation events:
>>
>> DOMAPI.PerDocumentMutationEventUsage.DOMCharacterDataModified: 2.85%
>> of all Documents
>> DOMAPI.PerDocumentMutationEventUsage.DOMNodeInserted: 4.78% of all
>> Documents
>> DOMAPI.PerDocumentMutationEventUsage.DOMNodeInsertedIntoDocument:
>> 0.21% of all Documents
>> DOMAPI.PerDocumentMutationEventUsage.DOMNodeRemoved: 0.07% of all
>> Documents
>> DOMAPI.PerDocumentMutationEventUsage.DOMNodeRemovedFromDocument:
>> 0.0003% of all Documents
>> DOMAPI.PerDocumentMutationEventUsage.DOMSubtreeModified: 2.30% of all
>> Documents
>>
>> From this data, it's tempting to remove support for DOMNodeRemoved and
>> DOMNodeRemovedFromDocument as those are used extremely rarely.  We
>> don't have any data for how much of this usage is attributable to
>> extensions.  It will be interesting to watch this data as we
>> evangelize MutationObservers, especially among extension developers.
>
> Is there an absolute number for those 0.07%?

There is, but I'm not able to share the absolute number publicly.  I
suspect you have access to the metrics yourself.  You can look at the
histograms with those names.

> How is it calculated?

It is calculated by this code:

http://trac.webkit.org/browser/trunk/Source/WebCore/dom/Document.cpp#L584

Basically, when a document is destructed, we look to see whether there
are event listeners of these types.  I'm not sure that's the optimal
way to measure the usage of these events, but that's the data we have
currently.

> Like Web
> Inspector has been using DOMNodeRemoved for its viewport-based text editor
> until very recently, so your 0.07% is likely to include millions of users.
> Is that expected?

The data is per-Document, not per user data.  In general, there is no
way to convert from one to the other.  You know the phrase "lies, damn
lies, and statistics"?  If we're going to make these sorts of
decisions based on metrics, we should make sure we're measuring the
right things.

> I wonder what will the drop be with the next major Chrome release.

This is data from the Stable channel on Windows, which is currently
Chrome 21.  There's also data from the Beta channel, which you or I
could look up if you think that would make a difference.

Adam


>> Another metric we have is for Blob.webkitSlice:
>>
>> Ratio of Blob.webkitSlice calls to Blob.slice: 14.87%
>> Ratio of Blob.webkitSlice calls to Document creation: 0.0053%
>>
>> It's difficult to know how to interpret this data because we don't
>> actually correlate calls to webkitSlice with Documents or Pages.
>> Instead, we just count the total number of calls across all Documents.
>>  This gives us an upper bound on how many Documents (or Pages) would
>> be affected by deleting Blob.webkitSlice, but doesn't measure that
>> information as accurately as the data we have for mutation events.
>>
>> We are also gathering metrics on the usage of vendor prefixes in CSS
>> properties, but that histogram doesn't appear to have been in the
>> field long enough to have meaningful data.
>>
>> Adam
>> _______________________________________________
>> webkit-dev mailing list
>> webkit-dev at lists.webkit.org
>> http://lists.webkit.org/mailman/listinfo/webkit-dev
>
>