[webkit-help] how to run Layout tests

[SravanKumar Sandela]

Hi,

Comments added in line.

On Fri, Jun 3, 2011 at 2:13 PM, thouraya andolsi <thouraya.andolsi at gmail.com
> wrote:

> Hi,
>
> Thank you very much for your explanation.
>
> >When ever you run the tests on a new platform any thing other than
> Leopard, we will have to generate new "-expected" files. This is called
> Rebase-lining. Do >this only if you are sure about the obtained out-put and
> it is correct output. Rebase-lining is nothing but copying the "correct"
> actual outputs of the tests to >respective expected files. You can do this
> using --reset-results option also. But check upstream if they are broken in
> actual webkit also before rebase-lining.
>
> How to do the rebase-lining ?
> Is there a command line to do it ?
>

[Sravan] :  There are different ways of doing it, chromium guys do it with
their own shell script, if you know perl you can make a local hack to the
existing script such that actual out-put will be written on to its
corresponding -expected files(Time taking and do it only if you are expert
in perl) or else you can do this using --reset-results option provided by
the script it self. Double check after base-lining that things did not go
wrong because of your base-lining activity.

There is a less confusing and easy way and that is to manually copy the
actual outputs to expected files. But this is time-taking though
accurate.(Definitely not suggested if you have more tests to baseline.)

But like i mentioned already, do the baselining only on those tests that you
know are giving correct results and showing "tests had incorrect layout"

>
> running "run-webkit-tests --gtk --skipped=ignore --no-launch-safari
>  --root=$ROOT css1 --reset-results" it will regenerate the expected files.
> But how the be sure that tests are passing when running "run-webkit-tests
> --gtk --skipped=ignore --no-launch-safari  --root=$ROOT css1"?
> May be all the tests will pass successfully  since we are generating
> expected results?
>

[Sravan] : Yes you are right in saying that all the tests will pass if we
regenerate using --reset-results. But i have also mentioned one point that *you
should do this rebase-lining only when you are sure that the actual results
you are getting are correct results*. And when you are sure that actual
results are true, then obviously all the tests will pass. So, please verify
if the actual outputs generated by the test results are correct or not. If
you think they are wrong, it means DRT is not able to render the test
properly and i think normal procedure is to raise a bug.

>
>
> >But check upstream if they are broken in actual webkit also before
> rebase-lining.
> How to do it.
>
>
[Sravan] : One good way is to check the test case for which you are getting
wrong results and verify the test case is already present in bugs.webkit.org


> Thanks in advance.
> Thouraya.
>
>
> 2011/6/3 SravanKumar Sandela <sravan.ken at gmail.com>
>
>> Hi Thouraya,
>>
>> The output you mentioned "tests had incorrect layout.", might get
>> generated when you run them on a platform other than Leapard, as the checked
>> in results are the out-puts generated by running the tests on Mac
>> Leopard platform. Hence when you run the tests the actual output will be
>> compared w.r.t their corresponding "-expected" files and if there is a
>> mis-match then the script would throw this kind of output.
>>
>> Coming to your second question "how to run layout tests?
>> should I regenerate expected tests using the option --reset-results or
>> not?"
>>
>> When ever you run the tests on a new platform any thing other than
>> Leopard, we will have to generate new "-expected" files. This is called
>> Rebase-lining. Do this only if you are sure about the obtained out-put and
>> it is correct output. Rebase-lining is nothing but copying the "correct"
>> actual outputs of the tests to respective expected files. You can do this
>> using --reset-results option also. But check upstream if they are broken in
>> actual webkit also before rebase-lining.
>>
>> Hope this information helps.
>>
>> Regards,
>> -Sravan
>>
>> On Thu, Jun 2, 2011 at 2:56 PM, thouraya andolsi <
>> thouraya.andolsi at gmail.com> wrote:
>>
>>> Hello,
>>>
>>> I compiled webkit for Gtk/Directfb backend.
>>> Trying to run css1 layout tests using the following command line
>>> "run-webkit-tests --gtk --skipped=ignore --no-launch-safari  --root=$ROOT
>>> css1"  I get : tests had incorrect layout.
>>>
>>> how to run layout tests?
>>> should I regenerate expected tests using the option --reset-results or
>>> not?
>>>
>>> Regards,
>>> Thouraya.
>>>
>>> _______________________________________________
>>> webkit-help mailing list
>>> webkit-help at lists.webkit.org
>>> http://lists.webkit.org/mailman/listinfo.cgi/webkit-help
>>>
>>>
>>
>>
>> --
>> Living for the unseen and undone....
>>
>
>


-- 
Living for the unseen and undone....
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-help/attachments/20110603/a8f6d62f/attachment-0001.html>