[webkit-reviews] review requested: [Bug 72841] NRWT: option --skip-pixel-test-if-no-baseline support on DRT : [Attachment 137313] Patch

[bugzilla-daemon at webkit.org]

Nandor Huszka <hnandor at inf.u-szeged.hu> has asked  for review:
Bug 72841: NRWT: option --skip-pixel-test-if-no-baseline support on DRT
https://bugs.webkit.org/show_bug.cgi?id=72841

Attachment 137313: Patch
https://bugs.webkit.org/attachment.cgi?id=137313&action=review

------- Additional Comments from Nandor Huszka <hnandor at inf.u-szeged.hu>
> That object is a driver (a GtkDriver). You changed that line from
self._start() to self._driver.start() (probably a cut&paste from DriverProxy in
driver.py). Change it back :).
Thank you, it was a carelessness on my part.

> I think this is probably wrong, and that you need to use the logic on lines
205-206 to set is_reftest, and then probably (I haven't tried this) set
should_run_pixel_tests to (self._options.pixel_tests or
driver_input.image_hash).
I have run unittests with
    is_reftest = driver_input.should_run_pixel_test
    driver_input.should_run_pixel_test = (self._options.pixel_tests or
driver_input.image_hash)
modifications in mock_drt.py:188, but 16 fail appeared, they were similar to
this one:

ERROR: webkitpy.layout_tests.port.mock_drt_unittest.MockDRTTest.test_textonly
-------------------------------------------------------------------------------
-
Traceback (most recent call last):
  File
"/home/hnandor/WebKit/Tools/Scripts/webkitpy/layout_tests/port/mock_drt_unittes
t.py", line 192, in test_textonly
    self.assertTest('passes/image.html', False)
  File
"/home/hnandor/WebKit/Tools/Scripts/webkitpy/layout_tests/port/mock_drt_unittes
t.py", line 154, in assertTest
    res = drt.run()
  File
"/home/hnandor/WebKit/Tools/Scripts/webkitpy/layout_tests/port/mock_drt.py",
line 188, in run
    is_reftest = (self._port.reference_files(test_name) or
NameError: global name 'test_name' is not defined

Then I tried this:
    is_reftest = driver_input.should_run_pixel_test
    driver_input.should_run_pixel_test = (self._options.pixel_tests or
driver_input.image_hash)
Only one unittest failed with it:

FAILURE:
webkitpy.layout_tests.port.mock_drt_unittest.MockDRTTest.test_missing_image
-------------------------------------------------------------------------------
-
Traceback (most recent call last):
  File
"/home/hnandor/WebKit/Tools/Scripts/webkitpy/layout_tests/port/mock_drt_unittes
t.py", line 198, in test_missing_image
    self.assertTest('failures/expected/missing_image.html', True)
  File
"/home/hnandor/WebKit/Tools/Scripts/webkitpy/layout_tests/port/mock_drt_unittes
t.py", line 160, in assertTest
    self.assertEqual(stdout.buflist, drt_output)
AssertionError: Lists differ: ['Content-Type: text/plain\n',... !=
['Content-Type: text/plain\n',...

First differing element 3:

#EOF

First list contains 3 additional elements.
First extra element 4:
ActualHash: None

+ ['Content-Type: text/plain\n', 'missing_image-txt', '#EOF\n', '#EOF\n']
- ['Content-Type: text/plain\n',
-  'missing_image-txt',
-  '#EOF\n',
-  '\n',
-  'ActualHash: None\n',
-  'ExpectedHash: None\n',
-  '#EOF\n']

And with this combination:
    driver_input.should_run_pixel_test = (self._options.pixel_tests or
driver_input.image_hash)
    is_reftest = driver_input.should_run_pixel_tests
11 fails were generated.

But with only this line:
    is_reftest = driver_input.should_run_pixel_test
all of the unittests passed. It means that there is no need for the logic on
lines 205-206 here?
You were right, I have not run unittests before uploading the last patch and
some other unittests failed because of wrong parameter number (output_for_test)
and missing renaming (test_input.is_reftest) in mock_drt.py.
Now the run-webkit-tests --platform mock-gtk, run-webkit-tests --platform
mock-mac-lion and testwebkitpy run without any errors, and I hope it means
everything is OK with this version of patch. :)