[Webkit-unassigned] [Bug 108288] On EWS bots, run tests on patch once instead of twice if some tests fail.

[bugzilla-daemon at webkit.org]

https://bugs.webkit.org/show_bug.cgi?id=108288





--- Comment #14 from Timothy Loh <timloh at chromium.org>  2013-01-30 21:37:26 PST ---
(In reply to comment #8)
> (From update of attachment 185388 [details])
> View in context: https://bugs.webkit.org/attachment.cgi?id=185388&action=review
> 
> > Tools/Scripts/webkitpy/layout_tests/run_webkit_tests_integrationtest.py:645
> > -        res, err, _ = logging_run(['--no-retry-failures', '--clobber-old-results', 'failures/flaky'], tests_included=True, host=host)
> > +        res, err, _ = logging_run(['--retry-failures=0', '--clobber-old-results', 'failures/flaky'], tests_included=True, host=host)
> 
> I think users use --no-retry-failures on NRWT, so this may be slightly confusing and worth announcing on webkit-dev.

I can of course leave the flag in, so as to not bother others.

> > Tools/Scripts/webkitpy/tool/bot/patchanalysistask.py:197
> > +        # FIXME: We'll want to add LayoutTestResults.flaky_tests() first
> > +        # Only report flaky tests if we were successful at parsing results.json and archiving results.
> > +        # if first_results and first_results_archive:
> > +        #     self._report_flaky_tests(first_results.failing_test_results(), first_results_archive)
> 
> So now we're going to depend on NRWT reporting the flaky tests to us?  That sounds reasonable, but may not work for other methods of testing?

Good point. Do we expect that we'll be using the code with other tests though? The existing code already does some re-running to detect flaky tests, which I imagine probably won't end up used by the other subclasses of PatchAnalysisTask (StyleQueueTask and PerfalizerTask), but you might think otherwise.

-- 
Configure bugmail: https://bugs.webkit.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.