I inspected all of our bots this morning, none of them are chronically in a bad state: <https://webkit-queues.webkit.org/queue-status/ios-sim-ews <https://webkit-queues.webkit.org/queue-status/ios-sim-ews>>. Also, we are getting results for your patch <https://webkit-queues.webkit.org/patch/351200/ios-sim-ews <https://webkit-queues.webkit.org/patch/351200/ios-sim-ews>>, it’s just occasionally failing to boot simulators. There isn’t much more we can do from an infrastructure perspective other than what Ryan did on the back end of last week. The Peace GM simulators are taking a very long time to boot, and the log here indicates that we timed out waiting for them to boot. Ryan bumped the timeout to 10 minutes last week. We can’t fully inoculate our bots from this failure, ultimately it’s either a simulator bug or an iOS bug which we can only reproduce on customer machines. Jonathan

On Sep 29, 2018, at 1:19 AM, Ryosuke Niwa <rniwa@webkit.org> wrote:

Looks like this is still happening in the remaining bots:

e.g. https://webkit-queues.webkit.org/results/9391751 <https://webkit-queues.webkit.org/results/9391751>

- R. Niwa

On Thu, Sep 27, 2018 at 9:08 AM Ryan Haddad <ryanhaddad@apple.com <mailto:ryanhaddad@apple.com>> wrote: It appears that a few bots are having issues. I have taken them offline and will investigate.

Ryan

...

On Sep 27, 2018, at 2:06 AM, Ryosuke Niwa <rniwa@webkit.org <mailto:rniwa@webkit.org>> wrote:

e.g. https://webkit-queues.webkit.org/results/9366267 <https://webkit-queues.webkit.org/results/9366267> RuntimeError raised: Timed out while waiting for data migration Traceback (most recent call last): File "/Volumes/Data/EWS/WebKit/Tools/Scripts/webkitpy/layout_tests/run_webkit_tests.py", line 85, in main run_details = run(port, options, args, stderr) File "/Volumes/Data/EWS/WebKit/Tools/Scripts/webkitpy/layout_tests/run_webkit_tests.py", line 448, in run run_details = manager.run(args) File "/Volumes/Data/EWS/WebKit/Tools/Scripts/webkitpy/layout_tests/controllers/manager.py", line 246, in run if not self._set_up_run(tests_to_run): File "/Volumes/Data/EWS/WebKit/Tools/Scripts/webkitpy/layout_tests/controllers/manager.py", line 181, in _set_up_run self._port.setup_test_run(self._options.device_class) File "/Volumes/Data/EWS/WebKit/Tools/Scripts/webkitpy/port/ios.py", line 160, in setup_test_run self._create_devices(device_class) File "/Volumes/Data/EWS/WebKit/Tools/Scripts/webkitpy/port/ios_simulator.py", line 103, in _create_devices SimulatedDeviceManager.initialize_devices([request] * self.child_processes(), self.host) File "/Volumes/Data/EWS/WebKit/Tools/Scripts/webkitpy/xcode/simulated_device.py", line 368, in initialize_devices SimulatedDeviceManager.wait_until_data_migration_is_done(host, max(0, deadline - time.time())) File "/Volumes/Data/EWS/WebKit/Tools/Scripts/webkitpy/xcode/simulated_device.py", line 428, in wait_until_data_migration_is_done raise RuntimeError('Timed out while waiting for data migration') RuntimeError: Timed out while waiting for data migration Error in atexit._run_exitfuncs: Traceback (most recent call last): File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/atexit.py", line 24, in _run_exitfuncs func(*targs, **kargs) File "/Volumes/Data/EWS/WebKit/Tools/Scripts/webkitpy/xcode/simulated_device.py", line 446, in tear_down device.platform_device._tear_down(deadline - time.time()) File "/Volumes/Data/EWS/WebKit/Tools/Scripts/webkitpy/xcode/simulated_device.py", line 522, in _tear_down self._shut_down(deadline - time.time()) File "/Volumes/Data/EWS/WebKit/Tools/Scripts/webkitpy/xcode/simulated_device.py", line 508, in _shut_down raise RuntimeError('Timed out while waiting for {} to shut down'.format(self.udid)) RuntimeError: Timed out while waiting for 3FDF5418-1774-4710-B947-9952835E2563 to shut down

- R. Niwa _______________________________________________ webkit-dev mailing list webkit-dev@lists.webkit.org <mailto:webkit-dev@lists.webkit.org> https://lists.webkit.org/mailman/listinfo/webkit-dev <https://lists.webkit.org/mailman/listinfo/webkit-dev>