[webkit-dev] Top 100 sites browsing tests proprosal

[Peter Kasting]

On Sun, Jan 22, 2012 at 11:55 AM, Balazs Kelemen <kbalazs at webkit.org> wrote:

> As the goal is to test "real world" use case I think it can be even better
> to simply load the sites from network.
> I see only two disadvantage of that but neither of them are blocker:
>    1. The sites changing over time. However, as this would be a smoke test
> (and not a regtest or a perf test) I don't think it's a big problem
>    2. Cannot test offline. Well, I don't think taking part in the
> development of WebKit is possible without being online anyway :)
> Do you see other disadvantages?
>

Loading over a network connection is slower.  Especially if you expect
developers to run these tests locally, this could be a serious time sink.

Loading real-world sites over a real network connection adds numerous
possible flakiness/failure sources.  I would never use something like this
as part of an automated system.

Arguably, it's not kind to site authors to constantly reload their real
sites as part of an automated testing program.

>From experience using offline copies of popular sites for testing in both
Firefox and Chrome, I strongly suggest you go the offline route.

PK
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20120122/b6b0ed6f/attachment.html>