[Webkit-unassigned] [Bug 71726] New: https://bugs.webkit.org/robots.txt should allow indexing of bugs

[bugzilla-daemon at webkit.org]

https://bugs.webkit.org/show_bug.cgi?id=71726

           Summary: https://bugs.webkit.org/robots.txt should allow
                    indexing of bugs
           Product: WebKit
           Version: 528+ (Nightly build)
          Platform: Unspecified
        OS/Version: Unspecified
            Status: NEW
          Severity: Normal
          Priority: P2
         Component: Tools / Tests
        AssignedTo: webkit-unassigned at lists.webkit.org
        ReportedBy: eric at webkit.org
                CC: abarth at webkit.org, wsiegrist at apple.com


https://bugs.webkit.org/robots.txt should allow indexing of bugs

We can give it an insanely high crawl-delay if we're worried about server load.  I just want to be able to search for bugs from 3 years ago in Google, since bugzilla's built in stemming is very primitive.

http://en.wikipedia.org/wiki/Robots_exclusion_standard#Crawl-delay_directive

-- 
Configure bugmail: https://bugs.webkit.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.