[webkit-dev] Sharing WebKit mocks across platforms

[Hajime Morita]

Hi Alexey, thank you for sharing your thought.

I agree that using platform APIs for testing is better than using mocks.
On the other hand, I believe the automated testing is our invaluable assert.
And some real API is hard to automate. Here is a conflict. I think we need both.

Currently WebKit has3 mocks under platform/mock
- DeviceOrientationClientMock
- GeolocationServiceMock
- SpeechInputClientMock
It looks that these features are hard to automate without mocks.
Another advantage of mocked test is that it is easier to have high
code coverage
than others, due to the ease to control which mocks provide.

So having these mocks somewhere out of WebCore looks a reasonable compromise
if we have some manual test counterparts for mocked automated tests.

But I'd like to hear the implementors voice, as Alexey hoped so.

--
morrita


On Thu, Nov 11, 2010 at 2:53 AM, Alexey Proskuryakov <ap at webkit.org> wrote:
>
> Sorry for a delayed response - I hoped for someone else to weigh in.
>
> This approach is certainly better than having code in shipping WebCore. But I still think that testing via platform APIs is much more desirable than implementing and maintaining a separate "pseudo-API" for mocks. This can't be too bad for productivity either - the social contract seems to be that you implement DumpRenderTree for one or two platforms at most, and file bugs for others.
>
> - WBR, Alexey Proskuryakov
>
>
> 27.10.2010, в 22:08, Hajime Morita написал(а):
>
>> Hi Alexey, thank you for revising this topic!
>>
>> I understand your concern about having a testing infrastructure inside
>> the production code.
>> On the other hand, having separate but similar mocks for each port
>> hurts our productivity.
>> And we cannot automate testing without mocks anyway.
>>
>> So how about to have separate "WebCoreTesting" library/framework?
>> The WebCoreTesting will:
>> - access WebCore types directly, as WebKit library does
>> - have mock implementations of WebCore-provided interfaces (abstract classes)
>> and
>> - WebCore will provide abstract classes and injection points for their
>> intenaces, but not provide class implementations
>> - WebKit will provide a priviate method to enable the mocks, which
>> uses WebKitTesting.
>>  WebKitTesting library will be linked lazily (using weak symbol
>> mechanism for Mac, or dlopen() family for other ports.)
>>
>> In this approach, we
>> - can split the testing infrastructure out from the production code,
>> just by removing WebCoreTesting library.
>> - can share mock implementations between ports.
>>
>> Even with this approach, some redundancy remains in WebCore, like
>> abstract class for mocks.
>> But actual code, which is a possible cause of vulnerability, can be
>> removed from the production.
>>
>> I once prototyped that approach for introducing WebCore::LayoutTestController.
>> So the patch might help to see what I meant to say:
>> https://bug-42612-attachments.webkit.org/attachment.cgi?id=63403
>> (Note this patch has rough edges and need to polish anyway.)
>>
>> Believing what I proposed above is a reasonable tradeoff between
>> security/efficiency and productivity,
>> I wonder there might be other possibilities to explore.
>> So I'd love to hear from you and any folks who have interest.
>>
>> Regards.
>> --
>> morrita
>>
>>
>> On Thu, Oct 28, 2010 at 4:07 AM, Alexey Proskuryakov <ap at webkit.org> wrote:
>>>
>>> 02.08.2010, в 4:38, Alexey Proskuryakov написал(а):
>>>
>>>>
>>>> 29.07.2010, в 8:16, Adam Barth написал(а):
>>>>
>>>>> Plumbing this mock API all the way through WebKit for each port seems like a waste.
>>>>
>>>>
>>>> One benefit of this is that it makes us test the API layer. Another one is that it gives WebKit developers better familiarity with APIs - that's not as minor as it sounds, as some of us (including myself) have limited knowledge of WebKit APIs, even for their favorite platform.
>>>>
>>>> It certainly seems like a waste when private methods are added just for DumpRenderTree support. But maybe it has prompted an API introduction a few times before, I just don't know.
>>>>
>>>> This is all getting much better with WebKit2's cross-platform API, although of course we'll need to support current APIs for a long time.
>>>
>>> This thread died, but I still don't feel good about mock objects in WebCore. Having test infrastructure in WebCore means shipping unused code in the framework, and as mentioned before, we lack testing of the API layer.
>>>
>>> We're now getting more mock objects in the tree, such as one for speech synthesis testing.
>>>
>>> - WBR, Alexey Proskuryakov
>>>
>>> _______________________________________________
>>> webkit-dev mailing list
>>> webkit-dev at lists.webkit.org
>>> http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
>>>
>>
>>
>>
>> --
>> morrita
>
>
>



-- 
morrita