[Webkit-unassigned] [Bug 100247] New: [Qt][WK2] Zillion API tests fail with newer Qt5

bugzilla-daemon at webkit.org bugzilla-daemon at webkit.org
Wed Oct 24 07:15:58 PDT 2012 

https://bugs.webkit.org/show_bug.cgi?id=100247

           Summary: [Qt][WK2] Zillion API tests fail with newer Qt5
           Product: WebKit
           Version: 420+
          Platform: All
        OS/Version: All
            Status: NEW
          Keywords: Qt, QtTriaged
          Severity: Blocker
          Priority: P1
         Component: New Bugs
        AssignedTo: webkit-unassigned at lists.webkit.org
        ReportedBy: ossy at webkit.org
                CC: hausmann at webkit.org, jocelyn.turcotte at digia.com
            Blocks: 70236,76773


The WK2 API tests work more or less (apart from https://wkb.ug/100224) 
with the actual Qt5 hash (34f1dbc74a893db80786a7723c6a90ff2cf7d7b7).
I tested the trunk qt5.git hash - 98a605eb6b085a0d95a8c7c3926e46ba35e865a5, but unfortunately
almost all API tests fail with it:

$ run-in-xvfb.sh python ./Tools/Scripts/run-qtwebkit-tests --output-file=qt-unit-tests.html --do-not-open-results --timeout=120 WebKitBuild/Release/Source/WebKit2/UIProcess/API/qt/tests/
INFO:Exec:Running... WebKitBuild/Release/Source/WebKit2/UIProcess/API/qt/tests/qmltests/tst_qmltests_DesktopBehavior
INFO:Exec:Running... WebKitBuild/Release/Source/WebKit2/UIProcess/API/qt/tests/qmltests/tst_qmltests_WebView
INFO:Exec:Running... WebKitBuild/Release/Source/WebKit2/UIProcess/API/qt/tests/qrawwebview/tst_qrawwebview
INFO:Exec:Running... WebKitBuild/Release/Source/WebKit2/UIProcess/API/qt/tests/inspectorserver/tst_inspectorserver
INFO:Exec:Running... WebKitBuild/Release/Source/WebKit2/UIProcess/API/qt/tests/publicapi/tst_publicapi
INFO:Exec:Running... WebKitBuild/Release/Source/WebKit2/UIProcess/API/qt/tests/qquickwebview/tst_qquickwebview
INFO:Exec:Finished WebKitBuild/Release/Source/WebKit2/UIProcess/API/qt/tests/publicapi/tst_publicapi
INFO:Exec:Finished WebKitBuild/Release/Source/WebKit2/UIProcess/API/qt/tests/inspectorserver/tst_inspectorserver
INFO:Exec:Finished WebKitBuild/Release/Source/WebKit2/UIProcess/API/qt/tests/qmltests/tst_qmltests_DesktopBehavior
INFO:Exec:Finished WebKitBuild/Release/Source/WebKit2/UIProcess/API/qt/tests/qrawwebview/tst_qrawwebview
ERROR:Exec:Timeout, process 'WebKitBuild/Release/Source/WebKit2/UIProcess/API/qt/tests/qmltests/tst_qmltests_WebView' (18674) was terminated
ERROR:Exec:Timeout, process 'WebKitBuild/Release/Source/WebKit2/UIProcess/API/qt/tests/qquickwebview/tst_qquickwebview' (18680) was terminated
INFO:Exec:Finished WebKitBuild/Release/Source/WebKit2/UIProcess/API/qt/tests/qmltests/tst_qmltests_WebView
INFO:Exec:Finished WebKitBuild/Release/Source/WebKit2/UIProcess/API/qt/tests/qquickwebview/tst_qquickwebview
WARNING:Main:Missing sub-summary: WebKitBuild/Release/Source/WebKit2/UIProcess/API/qt/tests/qmltests/tst_qmltests_WebView
WARNING:Main:Missing sub-summary: WebKitBuild/Release/Source/WebKit2/UIProcess/API/qt/tests/qmltests/tst_qmltests_WebView
********* Start testing of qmltests *********
Config: Using QTest library 5.0.0, Qt 5.0.0
PASS   : qmltests::DesktopWebViewLinkHovered::initTestCase()
FAIL!  : qmltests::DesktopWebViewLinkHovered::test_linkHovered() 'verify()' returned FALSE. ()
   Loc: [/home/oszi/WebKit/Source/WebKit2/UIProcess/API/qt/tests/qmltests/DesktopBehavior/tst_linkHovered.qml(47)]
FAIL!  : qmltests::DesktopWebViewLinkHovered::test_linkHoveredDoesntEmitRepeated() 'verify()' returned FALSE. ()
   Loc: [/home/oszi/WebKit/Source/WebKit2/UIProcess/API/qt/tests/qmltests/DesktopBehavior/tst_linkHovered.qml(63)]
PASS   : qmltests::DesktopWebViewLinkHovered::cleanupTestCase()
PASS   : qmltests::DesktopWebViewLoadHtml::initTestCase()
FAIL!  : qmltests::DesktopWebViewLoadHtml::test_baseUrlAfterLoadHtml() 'verify()' returned FALSE. ()
   Loc: [/home/oszi/WebKit/Source/WebKit2/UIProcess/API/qt/tests/qmltests/DesktopBehavior/tst_loadHtml.qml(45)]
PASS   : qmltests::DesktopWebViewLoadHtml::cleanupTestCase()
PASS   : qmltests::DesktopWebViewNavigationRequested::initTestCase()
FAIL!  : qmltests::DesktopWebViewNavigationRequested::test_downloadPolicy() 'verify()' returned FALSE. ()
   Loc: [/home/oszi/WebKit/Source/WebKit2/UIProcess/API/qt/tests/qmltests/DesktopBehavior/tst_navigationRequested.qml(103)]
FAIL!  : qmltests::DesktopWebViewNavigationRequested::test_ignorePolicy() 'verify()' returned FALSE. ()
   Loc: [/home/oszi/WebKit/Source/WebKit2/UIProcess/API/qt/tests/qmltests/DesktopBehavior/tst_navigationRequested.qml(92)]
FAIL!  : qmltests::DesktopWebViewNavigationRequested::test_usePolicy() 'verify()' returned FALSE. ()
   Loc: [/home/oszi/WebKit/Source/WebKit2/UIProcess/API/qt/tests/qmltests/DesktopBehavior/tst_navigationRequested.qml(83)]
PASS   : qmltests::DesktopWebViewNavigationRequested::cleanupTestCase()
Totals: 6 passed, 6 failed, 0 skipped
********* Finished testing of qmltests *********

********* Start testing of qmltests *********
Config: Using QTest library 5.0.0, Qt 5.0.0
PASS   : qmltests::WebViewApplicationSchemes::initTestCase()
FAIL!  : qmltests::WebViewApplicationSchemes::test_applicationScheme() 'verify()' returned FALSE. ()
   Loc: [/home/oszi/WebKit/Source/WebKit2/UIProcess/API/qt/tests/qmltests/WebView/tst_applicationScheme.qml(75)]
FAIL!  : qmltests::WebViewApplicationSchemes::test_charsets() 'verify()' returned FALSE. ()
   Loc: [/home/oszi/WebKit/Source/WebKit2/UIProcess/API/qt/tests/qmltests/WebView/tst_applicationScheme.qml(108)]
FAIL!  : qmltests::WebViewApplicationSchemes::test_multipleSchemes() 'verify()' returned FALSE. ()
   Loc: [/home/oszi/WebKit/Source/WebKit2/UIProcess/API/qt/tests/qmltests/WebView/tst_applicationScheme.qml(83)]
FAIL!  : qmltests::WebViewApplicationSchemes::test_multipleUrlsForScheme() 'verify()' returned FALSE. ()
   Loc: [/home/oszi/WebKit/Source/WebKit2/UIProcess/API/qt/tests/qmltests/WebView/tst_applicationScheme.qml(95)]
PASS   : qmltests::WebViewApplicationSchemes::cleanupTestCase()
PASS   : qmltests::WebViewColorChooser::initTestCase()
FAIL!  : qmltests::WebViewColorChooser::test_accept() 'verify()' returned FALSE. ()
   Loc: [/home/oszi/WebKit/Source/WebKit2/UIProcess/API/qt/tests/qmltests/WebView/tst_colorChooser.qml(47)]
FAIL!  : qmltests::WebViewColorChooser::test_currentValue() 'verify()' returned FALSE. ()
   Loc: [/home/oszi/WebKit/Source/WebKit2/UIProcess/API/qt/tests/qmltests/WebView/tst_colorChooser.qml(47)]
FAIL!  : qmltests::WebViewColorChooser::test_reject() 'verify()' returned FALSE. ()
   Loc: [/home/oszi/WebKit/Source/WebKit2/UIProcess/API/qt/tests/qmltests/WebView/tst_colorChooser.qml(47)]
PASS   : qmltests::WebViewColorChooser::cleanupTestCase()
PASS   : qmltests::DevicePixelRatio::initTestCase()
FAIL!  : qmltests::DevicePixelRatio::test_devicePixelRatio() 'verify()' returned FALSE. ()
   Loc: [/home/oszi/WebKit/Source/WebKit2/UIProcess/API/qt/tests/qmltests/WebView/tst_devicePixelRatio.qml(33)]
FAIL!  : qmltests::DevicePixelRatio::test_devicePixelRatioMediaQuery() 'verify()' returned FALSE. ()
   Loc: [/home/oszi/WebKit/Source/WebKit2/UIProcess/API/qt/tests/qmltests/WebView/tst_devicePixelRatio.qml(50)]
PASS   : qmltests::DevicePixelRatio::cleanupTestCase()
PASS   : qmltests::DoubleTapToZoom::initTestCase()
FAIL!  : qmltests::DoubleTapToZoom::test_basic() 'verify()' returned FALSE. ()
   Loc: [/home/oszi/WebKit/Source/WebKit2/UIProcess/API/qt/tests/qmltests/WebView/tst_doubleTapToZoom.qml(82)]
PASS   : qmltests::DoubleTapToZoom::cleanupTestCase()
PASS   : qmltests::WebViewDownload::initTestCase()
FAIL!  : qmltests::WebViewDownload::test_downloadRequest() 'wait for signal downloadRequested' returned FALSE. ()
   Loc: [/home/oszi/WebKit/Source/WebKit2/UIProcess/API/qt/tests/qmltests/WebView/tst_download.qml(65)]
FAIL!  : qmltests::WebViewDownload::test_expectedLength() 'wait for signal downloadRequested' returned FALSE. ()
   Loc: [/home/oszi/WebKit/Source/WebKit2/UIProcess/API/qt/tests/qmltests/WebView/tst_download.qml(72)]
FAIL!  : qmltests::WebViewDownload::test_succeeded() 'wait for signal downloadRequested' returned FALSE. ()
   Loc: [/home/oszi/WebKit/Source/WebKit2/UIProcess/API/qt/tests/qmltests/WebView/tst_download.qml(80)]
PASS   : qmltests::WebViewDownload::cleanupTestCase()
PASS   : qmltests::JavaScriptEvaluation::initTestCase()
FAIL!  : qmltests::JavaScriptEvaluation::test_basic() 'verify()' returned FALSE. ()
   Loc: [/home/oszi/WebKit/Source/WebKit2/UIProcess/API/qt/tests/qmltests/WebView/tst_evaluateJavaScript.qml(45)]
FAIL!  : qmltests::JavaScriptEvaluation::test_booleanValue() 'verify()' returned FALSE. ()
   Loc: [/home/oszi/WebKit/Source/WebKit2/UIProcess/API/qt/tests/qmltests/WebView/tst_evaluateJavaScript.qml(87)]
FAIL!  : qmltests::JavaScriptEvaluation::test_floatValue() 'verify()' returned FALSE. ()
   Loc: [/home/oszi/WebKit/Source/WebKit2/UIProcess/API/qt/tests/qmltests/WebView/tst_evaluateJavaScript.qml(138)]
FAIL!  : qmltests::JavaScriptEvaluation::test_integerValue() 'verify()' returned FALSE. ()
   Loc: [/home/oszi/WebKit/Source/WebKit2/UIProcess/API/qt/tests/qmltests/WebView/tst_evaluateJavaScript.qml(121)]
FAIL!  : qmltests::JavaScriptEvaluation::test_propertyObjectWithChild() 'verify()' returned FALSE. ()
   Loc: [/home/oszi/WebKit/Source/WebKit2/UIProcess/API/qt/tests/qmltests/WebView/tst_evaluateJavaScript.qml(62)]
FAIL!  : qmltests::JavaScriptEvaluation::test_queryById() 'verify()' returned FALSE. ()
   Loc: [/home/oszi/WebKit/Source/WebKit2/UIProcess/API/qt/tests/qmltests/WebView/tst_evaluateJavaScript.qml(175)]
FAIL!  : qmltests::JavaScriptEvaluation::test_queryTitle() 'verify()' returned FALSE. ()
   Loc: [/home/oszi/WebKit/Source/WebKit2/UIProcess/API/qt/tests/qmltests/WebView/tst_evaluateJavaScript.qml(156)]
FAIL!  : qmltests::JavaScriptEvaluation::test_stringValue() 'verify()' returned FALSE. ()
   Loc: [/home/oszi/WebKit/Source/WebKit2/UIProcess/API/qt/tests/qmltests/WebView/tst_evaluateJavaScript.qml(104)]
PASS   : qmltests::JavaScriptEvaluation::cleanupTestCase()
PASS   : qmltests::WebViewLoadFavIcon::initTestCase()
FAIL!  : qmltests::WebViewLoadFavIcon::test_favIconLoad() 'wait for signal iconChanged' returned FALSE. ()
   Loc: [/home/oszi/WebKit/Source/WebKit2/UIProcess/API/qt/tests/qmltests/WebView/tst_favIconLoad.qml(39)]

CRASHED: WebKitBuild/Release/Source/WebKit2/UIProcess/API/qt/tests/qmltests/tst_qmltests_WebView

********* Start testing of tst_qrawwebview *********
Config: Using QTest library 5.0.0, Qt 5.0.0
PASS   : tst_qrawwebview::initTestCase()
FAIL!  : tst_qrawwebview::paint() '::waitForSignal(this, SIGNAL(loaded()), 5000)' returned FALSE. ()
   Loc: [/home/oszi/WebKit/Source/WebKit2/UIProcess/API/qt/tests/qrawwebview/tst_qrawwebview.cpp(90)]
FAIL!  : tst_qrawwebview::paint() 'compareImages(QImage(expectation), image.rgbSwapped(), 5)' returned FALSE. ()
   Loc: [/home/oszi/WebKit/Source/WebKit2/UIProcess/API/qt/tests/qrawwebview/tst_qrawwebview.cpp(233)]
FAIL!  : tst_qrawwebview::noBackground1() '::waitForSignal(this, SIGNAL(loaded()), 5000)' returned FALSE. ()
   Loc: [/home/oszi/WebKit/Source/WebKit2/UIProcess/API/qt/tests/qrawwebview/tst_qrawwebview.cpp(90)]
FAIL!  : tst_qrawwebview::noBackground1() '::waitForSignal(this, SIGNAL(loaded()), 5000)' returned FALSE. ()
   Loc: [/home/oszi/WebKit/Source/WebKit2/UIProcess/API/qt/tests/qrawwebview/tst_qrawwebview.cpp(90)]
FAIL!  : tst_qrawwebview::noBackground2() '::waitForSignal(this, SIGNAL(loaded()), 5000)' returned FALSE. ()
   Loc: [/home/oszi/WebKit/Source/WebKit2/UIProcess/API/qt/tests/qrawwebview/tst_qrawwebview.cpp(90)]
FAIL!  : tst_qrawwebview::noBackground2() '::waitForSignal(this, SIGNAL(loaded()), 5000)' returned FALSE. ()
   Loc: [/home/oszi/WebKit/Source/WebKit2/UIProcess/API/qt/tests/qrawwebview/tst_qrawwebview.cpp(90)]
FAIL!  : tst_qrawwebview::noBackground3() '::waitForSignal(this, SIGNAL(loaded()), 5000)' returned FALSE. ()
   Loc: [/home/oszi/WebKit/Source/WebKit2/UIProcess/API/qt/tests/qrawwebview/tst_qrawwebview.cpp(90)]
FAIL!  : tst_qrawwebview::noBackground3() '::waitForSignal(this, SIGNAL(loaded()), 5000)' returned FALSE. ()
   Loc: [/home/oszi/WebKit/Source/WebKit2/UIProcess/API/qt/tests/qrawwebview/tst_qrawwebview.cpp(90)]
FAIL!  : tst_qrawwebview::noBackground3() 'compareImages(QImage(expectation), image.rgbSwapped(), 5)' returned FALSE. ()
   Loc: [/home/oszi/WebKit/Source/WebKit2/UIProcess/API/qt/tests/qrawwebview/tst_qrawwebview.cpp(233)]
PASS   : tst_qrawwebview::cleanupTestCase()
Totals: 2 passed, 9 failed, 0 skipped
********* Finished testing of tst_qrawwebview *********

********* Start testing of tst_InspectorServer *********
Config: Using QTest library 5.0.0, Qt 5.0.0
PASS   : tst_InspectorServer::initTestCase()
QWARN  : tst_InspectorServer::testPageList() Inspector server started successfully. Try pointing a WebKit browser to http://127.0.0.1:23654
PASS   : tst_InspectorServer::testPageList()
FAIL!  : tst_InspectorServer::testRemoteDebuggingMessage() Compared values are not the same
   Actual   (webSocketQueryWebView->title()):
   Expected (jsExpressionResult): 4
   Loc: [/home/oszi/WebKit/Source/WebKit2/UIProcess/API/qt/tests/inspectorserver/tst_inspectorserver.cpp(153)]
FAIL!  : tst_InspectorServer::openRemoteDebuggingSession() '!inspectorWebView->title().isEmpty()' returned FALSE. ()
   Loc: [/home/oszi/WebKit/Source/WebKit2/UIProcess/API/qt/tests/inspectorserver/tst_inspectorserver.cpp(178)]
PASS   : tst_InspectorServer::cleanupTestCase()
Totals: 3 passed, 2 failed, 0 skipped
********* Finished testing of tst_InspectorServer *********

********* Start testing of tst_QQuickWebView *********
Config: Using QTest library 5.0.0, Qt 5.0.0
PASS   : tst_QQuickWebView::initTestCase()
PASS   : tst_QQuickWebView::accessPage()
PASS   : tst_QQuickWebView::navigationStatusAtStartup()
FAIL!  : tst_QQuickWebView::stopEnabledAfterLoadStarted() Compared values are not the same
   Actual   (webView()->loading()): 0
   Expected (true): 1
   Loc: [/home/oszi/WebKit/Source/WebKit2/UIProcess/API/qt/tests/qquickwebview/tst_qquickwebview.cpp(148)]
PASS   : tst_QQuickWebView::baseUrl()
PASS   : tst_QQuickWebView::loadEmptyUrl()
FAIL!  : tst_QQuickWebView::loadEmptyPageViewVisible() 'waitForLoadSucceeded(webView())' returned FALSE. ()
   Loc: [/home/oszi/WebKit/Source/WebKit2/UIProcess/API/qt/tests/qquickwebview/tst_qquickwebview.cpp(176)]
FAIL!  : tst_QQuickWebView::loadEmptyPageViewHidden() 'waitForLoadSucceeded(webView())' returned FALSE. ()
   Loc: [/home/oszi/WebKit/Source/WebKit2/UIProcess/API/qt/tests/qquickwebview/tst_qquickwebview.cpp(176)]
FAIL!  : tst_QQuickWebView::loadNonexistentFileUrl() 'waitForLoadFailed(webView())' returned FALSE. ()
   Loc: [/home/oszi/WebKit/Source/WebKit2/UIProcess/API/qt/tests/qquickwebview/tst_qquickwebview.cpp(186)]
FAIL!  : tst_QQuickWebView::backAndForward() 'waitForLoadSucceeded(webView())' returned FALSE. ()
   Loc: [/home/oszi/WebKit/Source/WebKit2/UIProcess/API/qt/tests/qquickwebview/tst_qquickwebview.cpp(194)]
FAIL!  : tst_QQuickWebView::reload() 'waitForLoadSucceeded(webView())' returned FALSE. ()
   Loc: [/home/oszi/WebKit/Source/WebKit2/UIProcess/API/qt/tests/qquickwebview/tst_qquickwebview.cpp(217)]
FAIL!  : tst_QQuickWebView::stop() 'waitForLoadSucceeded(webView())' returned FALSE. ()
   Loc: [/home/oszi/WebKit/Source/WebKit2/UIProcess/API/qt/tests/qquickwebview/tst_qquickwebview.cpp(230)]
FAIL!  : tst_QQuickWebView::loadProgress() 'waitForLoadSucceeded(webView())' returned FALSE. ()
   Loc: [/home/oszi/WebKit/Source/WebKit2/UIProcess/API/qt/tests/qquickwebview/tst_qquickwebview.cpp(244)]
FAIL!  : tst_QQuickWebView::scrollRequest() 'waitForViewportReady(webView())' returned FALSE. ()
   Loc: [/home/oszi/WebKit/Source/WebKit2/UIProcess/API/qt/tests/qquickwebview/tst_qquickwebview.cpp(447)]
PASS   : tst_QQuickWebView::show()
FAIL!  : tst_QQuickWebView::showWebView() 'waitForLoadSucceeded(webView())' returned FALSE. ()
   Loc: [/home/oszi/WebKit/Source/WebKit2/UIProcess/API/qt/tests/qquickwebview/tst_qquickwebview.cpp(264)]
FAIL!  : tst_QQuickWebView::removeFromCanvas() 'waitForLoadSucceeded(webView())' returned FALSE. ()
   Loc: [/home/oszi/WebKit/Source/WebKit2/UIProcess/API/qt/tests/qquickwebview/tst_qquickwebview.cpp(264)]
QFATAL : tst_QQuickWebView::multipleWebViewWindows() Received signal 15
FAIL!  : tst_QQuickWebView::multipleWebViewWindows() Received a fatal error.
   Loc: [Unknown file(0)]
Totals: 6 passed, 12 failed, 0 skipped
********* Finished testing of tst_QQuickWebView *********

********* Start testing of tst_publicapi *********
Config: Using QTest library 5.0.0, Qt 5.0.0
PASS   : tst_publicapi::initTestCase()
PASS   : tst_publicapi::publicAPI()
PASS   : tst_publicapi::cleanupTestCase()
Totals: 3 passed, 0 failed, 0 skipped
********* Finished testing of tst_publicapi *********
**********************************************************************
**        TOTALS: 20 passed, 29 failed, 0 skipped, 1 crashed        **
**********************************************************************

-- 
Configure bugmail: https://bugs.webkit.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

More information about the webkit-unassigned mailing list