[webkit-dev] The http tests are very flaky

[Eric Seidel]

If you look at the flaky tests reported by the commit queue (since we
started tracking about a month ago):
https://bugs.webkit.org/show_bug.cgi?id=50856

You see that well over half are http tests. [1]


I'm curious if anyone has any theories?  These tests are causing a
majority of our regular sheriff-bot spam and green/red tree confusion.

Maybe something is wrong with Apache on Snow Leopard?  Maybe CFNetwork
getting stuck?

If you click on the individual timeouts, any failures within the last
week include failure type information (timeout, text failure, etc.)
and any failures within the last few days will have the -diffs.txt
file attached.
See https://bugs.webkit.org/show_bug.cgi?id=52398 or
https://bugs.webkit.org/show_bug.cgi?id=51613 as examples.


Looking at the Chromium Flakiness dashboard (which has results going
back for months!) we see that many of the same http tests are flaky
for chromium too:
http://test-results.appspot.com/dashboards/flakiness_dashboard.html

I believe they use a different http server (lighttpd, but my
information could be old?) and certainly a separate network stack.


Solving these flakes would greatly reduce commit-queue delays and
noise on build.webkit.org.

Thanks.

-eric

1.  Here is a direct link to all the bugs, you can get this from the
master bug by looking at the "show dependency tree" and then clicking
"view bugs as list":

https://bugs.webkit.org/buglist.cgi?bug_id=51976%2C50855%2C51019%2C51613%2C50958%2C51987%2C51020%2C52164%2C51763%2C51138%2C52117%2C51835%2C52209%2C51102%2C51582%2C51039%2C51558%2C50880%2C51817%2C51594%2C51624%2C29939%2C51913%2C50875%2C52115%2C51314%2C51098%2C51056%2C51733%2C51690%2C51028%2C50995%2C51891%2C51757%2C51649%2C52047%2C52016%2C51009%2C51765%2C51002%2C51130%2C51024%2C52158%2C50959%2C52398%2C52249&field-1-0-0=bug_id&query_format=advanced&remaction=&type-1-0-0=anyexact&value-1-0-0=51976%2C50855%2C51019%2C51613%2C50958%2C51987%2C51020%2C52164%2C51763%2C51138%2C52117%2C51835%2C52209%2C51102%2C51582%2C51039%2C51558%2C50880%2C51817%2C51594%2C51624%2C29939%2C51913%2C50875%2C52115%2C51314%2C51098%2C51056%2C51733%2C51690%2C51028%2C50995%2C51891%2C51757%2C51649%2C52047%2C52016%2C51009%2C51765%2C51002%2C51130%2C51024%2C52158%2C50959%2C52398%2C52249&order=bugs.short_desc%20desc%2Cbugs.bug_id&query_based_on=