[webkit-dev] How to collect usage metrics for deprecated DOM APIs

[Adam Barth]

I recently added FeatureObserver, which makes it easier to measure how
often web pages use deprecated DOM APIs.  Here's how to use
FeatureObserver:

Step 1: Add an enum value to FeatureObserver.h:
http://trac.webkit.org/changeset/128798/trunk/Source/WebCore/page/FeatureObserver.h

Step 2: Add the [V8MeasureAs=...] IDL attribute to the DOM API you
want to measure:
http://trac.webkit.org/changeset/128798/trunk/Source/WebCore/Modules/indexeddb/DOMWindowIndexedDatabase.idl

Step 3: There is no step 3.

There are, however, some limitations currently.  FeatureObserver only
works on the main thread and only works in the Chromium port.  If you
want to measure usage of a feature that exists both on the main thread
and in workers, you can use the following pattern:
<http://trac.webkit.org/changeset/128797/trunk/Source/WebCore/fileapi/WebKitBlobBuilder.cpp>.

Q: What exactly does FeatureObserver measure?
A: Feature observer counts the number of Pages that use a particular
feature.  If a page calls a DOM API many times, we only count it once
per page.  We use this approach so we can get a sense of what fraction
of pages will be affected when we delete the API.

Q: How long does it take to get results?
A: We'll get rough data from the Canary channel within the week, but
usually we're more interested in data from the Stable channel.
Getting high-quality data will take about 12-16 weeks.

Q: How can I make this work for my port?
A: The main thing you need to do is implement the
<http://trac.webkit.org/browser/trunk/Source/WebCore/platform/HistogramSupport.h>
interface.  HistogramSupport is already used in a number of places
throughout WebCore, so implementing that interface will let you
measure a variety of interesting data about how your port uses WebKit.
 (You might also need to do a little CodeGeneratorJS.pm hacking to
wire up the IDL attribute as well.)

Q: How will we use this data to decide whether to delete a deprecated API?
A: That's a much harder question to answer.  If FeatureObserver sees
very little usage of an API, we might reasonably believe that deleting
the API will have little compatibility impact, at least relative to
the population we're measuring.  The converse, however, is not
necessarily true.  Hopefully FeatureObserver will give us useful
information, but I suspect we will still need to make these sorts of
decisions with care.

Adam