<div class="gmail_quote">On Sun, Apr 29, 2012 at 3:44 PM, Ryosuke Niwa <span dir="ltr"><<a href="mailto:rniwa@webkit.org" target="_blank">rniwa@webkit.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div class="gmail_extra">On Fri, Apr 27, 2012 at 1:49 AM, Nat Duca <span dir="ltr"><<a href="mailto:nduca@chromium.org" target="_blank">nduca@chromium.org</a>></span> wrote:<br><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">



<div class="gmail_extra"><div class="gmail_extra">I'm concerned at how well this would work graphics performance tests.</div><div class="gmail_extra"><br></div><div class="gmail_extra">Consider:</div><div class="gmail_extra">




<a href="http://web.archive.org/web/20110111083848/http://techcrunch.com/" style="color:rgb(17,85,204)" target="_blank">http://web.archive.org/web/20110111083848/http://techcrunch.com/</a></div><div class="gmail_extra"><p>




<a href="http://web.archive.org/web/20110222032916/http://www.nytimes.com/" style="color:rgb(17,85,204)" target="_blank">http://web.archive.org/web/20110222032916/http://www.nytimes.com/</a></p><p><a href="http://web.archive.org/web/20110429194113/http://www.thewildernessdowntown.com/" style="color:rgb(17,85,204)" target="_blank">http://web.archive.org/web/20110429194113/http://www.thewildernessdowntown.com/</a></p>



<p>What do we do for the cases where <a href="http://archive.org" target="_blank">archive.org</a> is getting bad/incomplete ... erm, archives?</p></div></div></blockquote><div>There's no fix to it. If <a href="http://archive.org" target="_blank">archive.org</a> doesn't work, then we need to pull data directly from the website. We can do that. The infrastructure I'm developing is agnostic of whether we use <a href="http://archive.org" target="_blank">archive.org</a> or not. However, pulling data directly from websites will make the test suite behave differently depending on when you run the test so the test suite can't be open that way.</div>
<span class="HOEnZb"><font color="#888888">


<div><br></div></font></span></div></div></blockquote><div><br></div><div>Does it matter if the page contents are bad/incomplete?  It seems like all that matters is that they are consistent from pull-to-pull and somewhat representative of pages we'd care to optimize.  Is the concern that those URLs are missing too much content to be useful?</div>
<div><br></div><div>Note: The page cyclers used by Chromium all have data sets that are bad/incomplete.  This was intentional.  For example, if a subresource was not available for whatever reason, then the request to fetch it was neutered (e.g., all "http" substrings were replaced with "httpdisabled").</div>
<div><br></div><div>-Darin </div></div>