[webkit-dev] PSA: run-webkit-tests now generates pixel results in retry

[Glenn Adams]

On Wed, Mar 27, 2013 at 9:16 AM, Glenn Adams <glenn at skynav.com> wrote:

>
> On Fri, Mar 22, 2013 at 3:24 PM, Ryosuke Niwa <rniwa at webkit.org> wrote:
>
>> Hi all,
>>
>> I've made a change to NRWT in http://trac.webkit.org/changeset/146657 so
>> that it always generate pixel results in retries even when pixel tests are
>> disabled by default (on all but Chromium ports).
>>
>> We now enable pixel tests while we're retrying tests and generate
>> -expected.png results.
>>
>> This change has two important implications:
>>
>>    - We can now grab -expected.png results from build.webkit.orgbuilders for those tests that have resulted in non-image-only failures.
>>    - Chromium and Mac EWS bots will now upload ZIP archives with those
>>    images (recall http://trac.webkit.org/changeset/146443)
>>
>> This will allow us to rebaseline BOTH test and image expected results
>> PRIOR to landing patches.
>>
>
> Now that layout-test-results are being attached to their related bugs, I
> wonder if there is a way to delete these attachments (in BZ) so as not to
> choke disk space on the BZ server. Otherwise, I have a feeling disk usage
> will increase rapidly.
>
> For example, after one has reviewed and extracted the useful results from
> the attachments, it would be useful to delete (and not just make obsolete)
> those (large) attachments. I also notice that when an EWS has multiple
> fails on the same patch, that multiple layout-test-results get attached.
>
> It might also be useful to have a cron job on the BZ server go through and
> delete obsoleted layout-test-result attachments (in case it is not
> straightforward to permit attachment deletion). At least, then the dev
> could obsolete result attachments after extracting the useful data and the
> cron job could do the rest of the work of deleting the attachments.
>

Perhaps adding a DeleteObsoleteLayoutResults keyword would make automating
deletion a viable process. It could either be added by default when the
first layout results are attached (letting the dev decide whether to remove
the keyword and keeping obsolete layout results) or the dev could
explicitly add the keyword. I would suggest the former (add it by default),
forcing an explicit action by the dev to retain obsolete layout results.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20130327/5a24a897/attachment.html>