[webkit-gtk] dns prefetching knob

[Marco Peereboom]

On Wed, Mar 09, 2011 at 07:16:14AM -0500, Dan Winship wrote:
> On 03/08/2011 07:07 PM, Marco Peereboom wrote:
> > I created a bug report to add a DNS prefetch knob but I was advised to
> > also send the patch to the mailing list.
> > 
> > With this knob I can make the web browser I wrote (xxxterm
> > http://opensource.conformal.com/wiki/XXXTerm ) much snappier on sites
> > with lots of embedded domain names (news.google.com for example).
> 
> The entire point of DNS prefetching is that it's supposed to make the
> browser faster. If it's not doing that, then the right thing would be to
> remove it entirely, not to add a configurable "be broken / don't be
> broken" option.

I completely agree and that is what I suggested to the OpenBSD ports
people.  There is a whole lot of inertia on this.  For some reason
people are still attached to this relic of the modem era.  DNS prefetch
is optional in the HTTP spec and the time has come to admit that it no
longer serves a purpose.  Back in the day when modems were king and DNS
lookups were expensive you could save some time when a user clicked on a
link.  Times have changes and the web has evolved.  DNS lookups are
nowhere as expensive as they used to be and pages these days have
hundreds or even thousands of links.  The chances that a user clicks a
link and saves a couple of ms does no longer outweigh the cost of
numerous lookups.

This also reeks of committee design :-)
Trade a perceived performance issue for a real one.

> But I'm pretty sure the problem is just that we're prefetching too much:
> https://bugs.webkit.org/show_bug.cgi?id=41630. (Although I think it's
> gotten a little better since then. I don't see the spurious "no such
> host"s any more...)

My tests are with a caching DNS server that spoofs crappy ad servers to
NXDOMAIN.  If I disable that it slows down even more.  Cookies and
JavaScript were also disabled.  So the test is best case scenario.  What
blew my mind is that news.google.com prefetches 350 odd domains while
loading the page.  Some of those are sent in parallel but at 350 one
exhausts that parallel number and it becomes a waiting game.