[webkit-dev] Rolling out a patch requires a justification beyond a test failure in downstream projects

[Ryosuke Niwa]

Hi,

I've encountered a couple of incidences where people roll out patches
saying that test X is failing on some downstream project Y without giving
any details as to how those tests are failing and why that's a real WebKit
regression we should care about.

First off, I don't think we should be rolling out patches based solely on a
downstream test unless there is a clear evidence that the failure is a real
regression in WebKit that affects more than just the said downstream
project. You may talk to the author and he or she might be nice enough to
agree to roll out the patch, but I don't think we should be rolling out
patches right away regardless.

Second, if there is a clear WebKit regression, then you should communicate
the following information at minimum:

   1. The exact location of the test that failed - URL, etc...
   2. The nature of the failure - assertion failure, feature it's testing,
   etc...
   3. The output (before and) after the failure started happening.
   4. Instructions to run the tests locally

Without this, the author is left with no clue whatsoever to diagnose and/or
fix the problem.

- R. Niwa
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20121211/10829d28/attachment.html>