[webkit-dev] Bug numbers in TestExpectations

[Ojan Vafai]

In practice, with the chromium TestExpectations, I've found that it forces
people to document the history of why a test was added and gives a forum
for discussing fixes (e.g. for flaky tests). It's hard to do this with a
single comment. It has been a net positive in my opinion.

Take the following example (one of many) from the platform/mac/Skipped file:
# --- Media ---
media/controls-styling.html
media/media-document-audio-repaint.html
media/video-zoom-controls.html

That doesn't tell you anything about why those lines were put there, when
they started failing, etc. While it's true that a bug doesn't force you to
say anything useful either, in my experience, people put useful
descriptions in the bugs for these lines.


On Thu, Sep 20, 2012 at 2:28 PM, Ryosuke Niwa <rniwa at webkit.org> wrote:

> I support making bug URL or Bug(~) optional.
>
> On Thu, Sep 20, 2012 at 10:34 AM, Alexey Proskuryakov <ap at webkit.org>wrote:
>
>>
>> I've been repeatedly finding it that bug numbers in TestExpectations
>> don't lead to bugs that are helpful in the context of the specific test.
>> Often the bug doesn't have any information beyond what's already in the
>> expectation (that the test is skipped of failing, without a posted diff or
>> any other useful detail). Other times, the linked bug is one that the
>> person was working on at the time of changing expectations, so the bug link
>> is more like "see related bug XXXX, and try to figure out how exactly it's
>> related", not "this failure is tracked by XXXX". Free form comments worked
>> great for both cases in Skipped files.
>>
>> It appears that the rigid format that requires putting a bug URL is
>> causing more harm than good in practice. I suggest making the URL optional.
>>
>> - WBR, Alexey Proskuryakov
>> _______________________________________________
>> webkit-dev mailing list
>> webkit-dev at lists.webkit.org
>> http://lists.webkit.org/mailman/listinfo/webkit-dev
>>
>
>
> _______________________________________________
> webkit-dev mailing list
> webkit-dev at lists.webkit.org
> http://lists.webkit.org/mailman/listinfo/webkit-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20120920/8b417926/attachment.html>