[webkit-gtk] WebKit tests

[Carlos Alberto Lopez Perez]

On 09/05/17 19:21, Konstantin Tokarev wrote:
> 
> 
> 09.05.2017, 20:16, "Michael Catanzaro" <mcatanzaro at igalia.com>:
>> On Tue, May 9, 2017 at 11:53 AM, Nagendra K <pingnagendra at gmail.com>
>> wrote:
>>>  Thank you Tokarev and Michel, I am able to build now with your
>>>  suggestions.
>>>  One more question, how to run the tests on an embedded device, can we
>>>  run the tests from host to the device?
>>>
>>>  Thanks and Regards,
>>>  Nagendra
>>
>> You'll have to run the tests on the device itself. Otherwise, what
>> would you really be testing?
> 
> I think question was about running tests on device without running run-webkit-tests
> and related Python stuff there. AFAIK it's not yet implemented and would be really
> nice to have.
> 

We do have this for the JavaScript Core tests (script:
run-javascriptcore-tests)

For the layout tests (script: run-webkit-tests) there isn't currently
possible to run the tests remotely. It will be nice to have.

In any case, notice that the layout test results are highly dependent on
a lot of system libraries.

Many tests are going to give different results depending on the version
of GTK+, cairo, gstremaer, etc.. your system has.

To try to workaround this problem, what we do is to first build very
specific versions of all the libraries we have identified that can
affect the test results. That is defined on the file
Tools/gtk/jhbuild.modules. Then we build WebKitGTK+ using the script
build-webkit that enables many still-in-development features that are
also needed for having the expected test results.

Very summarized, the commands you have to run to be able to get on the
layout tests the expected results:

1) Ensure we start from a clean-build state.
rm -fr WebKitBuild

2) Install some dependencies from your distribution:
Tools/gtk/install-dependencies

3) Build the jhbuild.
Tools/Scripts/update-webkitgtk-libs

4) Build webkit with the script:
Tools/Scripts/build-webkit --gtk --release

5) Finally run the layout tests:
Tools/Scripts/run-webkit-tests --release --gtk


And to have even more accurate results its recommended that you use as
distribution Debian 8 (that is what our bots run currently).
Because, several developers have reported many times getting different
results when using other distributions (even when using the internal
JHBuild). We didn't mastered yet the art of identifying all the
libraries that affect the results of the layout tests.


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 897 bytes
Desc: OpenPGP digital signature
URL: <http://lists.webkit.org/pipermail/webkit-gtk/attachments/20170510/db011df7/attachment.bin>