[webkit-gtk] dns prefetching knob

[Marco Peereboom]

On Wed, Mar 09, 2011 at 10:30:00AM -0800, Evan Martin wrote:
> On Wed, Mar 9, 2011 at 6:18 AM, David Hill <dhill at mindcry.org> wrote:
> >> Prefetch isn't mentioned in the HTTP spec at all. And I don't think
> >> anyone was doing it back in the modem age. AFAIK, Google started doing
> >> it with Chrome, and other people followed.
> >
> > If it is not mentioned in the spec, I think it should be optionized
> > then, not hardcoded. ?Google started doing it with Chrome, but was it a
> > performance booster, or (imo) a way to track users even more?
> 
> Your conspiracy theory is impressive, but as far as I know Google
> doesn't get to observe the DNS traffic of every person on the
> internet.  Maybe you know something about DNS that I don't.  ;)

http://code.google.com/speed/public-dns/

just sayin' :-)

> We did measure the benefit of DNS prefetching.  When we prefetch a
> hostname, we record the time that we needed to wait for the resolution
> to complete.  When we later do a real fetch for that host, we record
> that the time we spent prefetching was a gain.  You can see these
> numbers in Chrome by visiting
>   about:histograms/DNS.ResolveSuccess
> 
> For users who have opted in to reporting anonymous statistics, we send
> those numbers (no hostnames, just the millisecond counts) to Google.
> Looking at that data right now, I see that 10% of users' requests were
> sped up by 400ms or more, which is huge in terms of web speed.
> (Consider that you can't even start fetching subresources until you've
> successfully made the initial connection the host.)
> There's a screenshot of some older data here:
> http://blog.chromium.org/2008/09/dns-prefetching-or-pre-resolving.html

Not sure I read the results correctly but I seem to only see raw DNS
lookup speed gain.  Not overall "user visible page load times".  So
since I am not sure I get the results I won't comment further on it.

> 
> > I have tested this patch and the performance boost is unbelievable.
> 
> I guess on IRC Marco was saying that DNS on OpenBSD is single

Uhm, I don't recall being on IRC ;-)

> threaded?  Maybe I misunderstood.  It would sound like a bug in
> libsoup to me if so.  Certainly if you can't resolve DNS in parallel I
> would expect it's better to do fewer resolutions.

The OpenBSD libc resolver is a single pipe and it obviously exacerbates
the problem however, I use a resolver that parallelizes things.
Unfortunately I seem to mostly use really crappy nameservers so I see
issues all the time.

> For the record, I fully support you adding the option to disable
> prefetching.  I just hope that you want to do it for a good reason.

Sure, we agree here.  I have a number of reasons from performance to
security to paranoia to bandwitdh etc.

The fact that I am paranoid doesn't mean they aren't watching :-)