[webkit-dev] trac.webkit.org links via Google.com

[Mark Rowe]

On 2009-11-30, at 22:36, Eric Seidel wrote:

> It's bothered me for a while that I can't just type "trac webkit Document.cpp" into Google and have it give me a trac link to our Document.cpp page.
> http://trac.webkit.org/browser/trunk/WebCore/dom/Document.cpp
> 
> I checked http://trac.macosforge.org/robots.txt tonight and low and behold we disallow "browser/" (which is where all these links live).  Curious if this is intentional, and if we should change this setting?

Web crawler indexing of Trac is seriously painful for the servers involved.  The entire SVN history of the repository is accessible.  File content.  Changes.  Annotations.  Everything.  That's not cheap to compute and serve up.

- Mark

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20091130/2a273118/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3166 bytes
Desc: not available
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20091130/2a273118/attachment.bin>