[webkit-dev] PSA: run-webkit-tests now generates pixel results in retry

[Glenn Adams]

On Wed, Mar 27, 2013 at 11:46 AM, Glenn Adams <glenn at skynav.com> wrote:

>
> On Wed, Mar 27, 2013 at 9:16 AM, Glenn Adams <glenn at skynav.com> wrote:
>
>>
>> On Fri, Mar 22, 2013 at 3:24 PM, Ryosuke Niwa <rniwa at webkit.org> wrote:
>>
>>> Hi all,
>>>
>>> I've made a change to NRWT in http://trac.webkit.org/changeset/146657so that it always generate pixel results in retries even when pixel tests
>>> are disabled by default (on all but Chromium ports).
>>>
>>> We now enable pixel tests while we're retrying tests and generate
>>> -expected.png results.
>>>
>>> This change has two important implications:
>>>
>>>    - We can now grab -expected.png results from build.webkit.orgbuilders for those tests that have resulted in non-image-only failures.
>>>    - Chromium and Mac EWS bots will now upload ZIP archives with those
>>>    images (recall http://trac.webkit.org/changeset/146443)
>>>
>>> This will allow us to rebaseline BOTH test and image expected results
>>> PRIOR to landing patches.
>>>
>>
>> Now that layout-test-results are being attached to their related bugs, I
>> wonder if there is a way to delete these attachments (in BZ) so as not to
>> choke disk space on the BZ server. Otherwise, I have a feeling disk usage
>> will increase rapidly.
>>
>> For example, after one has reviewed and extracted the useful results from
>> the attachments, it would be useful to delete (and not just make obsolete)
>> those (large) attachments. I also notice that when an EWS has multiple
>> fails on the same patch, that multiple layout-test-results get attached.
>>
>> It might also be useful to have a cron job on the BZ server go through
>> and delete obsoleted layout-test-result attachments (in case it is not
>> straightforward to permit attachment deletion). At least, then the dev
>> could obsolete result attachments after extracting the useful data and the
>> cron job could do the rest of the work of deleting the attachments.
>>
>
> Perhaps adding a DeleteObsoleteLayoutResults keyword would make automating
> deletion a viable process. It could either be added by default when the
> first layout results are attached (letting the dev decide whether to remove
> the keyword and keeping obsolete layout results) or the dev could
> explicitly add the keyword. I would suggest the former (add it by default),
> forcing an explicit action by the dev to retain obsolete layout results.
>

I've filed a bug suggesting this enhancement:
https://bugs.webkit.org/show_bug.cgi?id=113428
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20130327/bb17868d/attachment-0001.html>