Sharing WebKit mocks across platforms
I'm in the process of adding a mock client for DeviceOrientation, which will be used in DumpRenderTree to test the feature. In order to share the mock across platforms, I'd like to add the mock to WebCore/platform/mock. An interface to the mock will have to be exposed to the embedder through the platform's WebKit API, so that it can be configured by DRT, eg ... mWebView.getDeviceOrientationClientMock().setOrientation(...); To avoid each platform having to produce it's own WebKit wrapper for the mock, I'm considering adding a common WebKit wrapper, perhaps to WebKit/common, and I wanted to get some feedback on the idea. The mock would be shared between all C++ WebKit platforms. (Note that this is for convenience only, a platform could equally use it's own WebKit wrapper around the WebCore mock (eg Mac may do so in ObjectiveC), or use its own mock altogether.) Of course we also need WebKit wrappers for all of the non-POD types used by the mock's interface, and these have to be common between all platforms. One obvious potential difficulty is the wrapper for WebCore::String. Each platforms already has a wrapper for this type, but there's no guarantee of interoperability, so we'd need to write a new common interface if we're to use the string type. If a wrapper for string ends up being too problematic, the approach could still be used for mocks that don't need the string type (of which DeviceOrientation is one), but the approach then seems less compelling. Do people think that this is a reasonable proposal and worth pursuing? Has there been any attempt to do anything similar before? Or is any attempt to write this kind of common WebKit code not worth the effort and destined to failure? You can see the work in progress for DeviceOrientation at https://bugs.webkit.org/show_bug.cgi?id=39589 and a similar patch for SpeechInput mocks at https://bugs.webkit.org/show_bug.cgi?id=42603 I'd appreciate any feedback you may have. Thanks, Steve -- Google UK Limited Registered Office: Belgrave House, 76 Buckingham Palace Road, London SW1W 9TQ Registered in England Number: 3977902
Thanks for bring this question to the list. I don't have a strong opinion here, but I want to make sure we think project-wide and pick something scalable. This discussion is also related to the discussion about adding something like a layoutTestController object to WebCore. Plumbing this mock API all the way through WebKit for each port seems like a waste. If we had something like a WebCore::LayoutTestController, it would make a lot more sense to expose that functionality there. Adam On Wed, Jul 28, 2010 at 11:30 AM, Steve Block <steveblock@google.com> wrote:
I'm in the process of adding a mock client for DeviceOrientation, which will be used in DumpRenderTree to test the feature. In order to share the mock across platforms, I'd like to add the mock to WebCore/platform/mock.
An interface to the mock will have to be exposed to the embedder through the platform's WebKit API, so that it can be configured by DRT, eg ...
mWebView.getDeviceOrientationClientMock().setOrientation(...);
To avoid each platform having to produce it's own WebKit wrapper for the mock, I'm considering adding a common WebKit wrapper, perhaps to WebKit/common, and I wanted to get some feedback on the idea. The mock would be shared between all C++ WebKit platforms. (Note that this is for convenience only, a platform could equally use it's own WebKit wrapper around the WebCore mock (eg Mac may do so in ObjectiveC), or use its own mock altogether.)
Of course we also need WebKit wrappers for all of the non-POD types used by the mock's interface, and these have to be common between all platforms. One obvious potential difficulty is the wrapper for WebCore::String. Each platforms already has a wrapper for this type, but there's no guarantee of interoperability, so we'd need to write a new common interface if we're to use the string type.
If a wrapper for string ends up being too problematic, the approach could still be used for mocks that don't need the string type (of which DeviceOrientation is one), but the approach then seems less compelling.
Do people think that this is a reasonable proposal and worth pursuing? Has there been any attempt to do anything similar before? Or is any attempt to write this kind of common WebKit code not worth the effort and destined to failure?
You can see the work in progress for DeviceOrientation at https://bugs.webkit.org/show_bug.cgi?id=39589 and a similar patch for SpeechInput mocks at https://bugs.webkit.org/show_bug.cgi?id=42603
I'd appreciate any feedback you may have.
Thanks, Steve
-- Google UK Limited Registered Office: Belgrave House, 76 Buckingham Palace Road, London SW1W 9TQ Registered in England Number: 3977902
With a WebCore::LayoutTestController, would there be a need for code in WebKit to convert the call parameters from WebKit types (which DRT uses) to WebCore types (which the mocks may use) ? If yes seems like we need wrappers to proxy the mock calls anyway on different platforms.. Cheers Satish On Thu, Jul 29, 2010 at 4:16 PM, Adam Barth <abarth@webkit.org> wrote:
Thanks for bring this question to the list. I don't have a strong opinion here, but I want to make sure we think project-wide and pick something scalable.
This discussion is also related to the discussion about adding something like a layoutTestController object to WebCore. Plumbing this mock API all the way through WebKit for each port seems like a waste. If we had something like a WebCore::LayoutTestController, it would make a lot more sense to expose that functionality there.
Adam
On Wed, Jul 28, 2010 at 11:30 AM, Steve Block <steveblock@google.com> wrote:
I'm in the process of adding a mock client for DeviceOrientation, which will be used in DumpRenderTree to test the feature. In order to share the mock across platforms, I'd like to add the mock to WebCore/platform/mock.
An interface to the mock will have to be exposed to the embedder through the platform's WebKit API, so that it can be configured by DRT, eg ...
mWebView.getDeviceOrientationClientMock().setOrientation(...);
To avoid each platform having to produce it's own WebKit wrapper for the mock, I'm considering adding a common WebKit wrapper, perhaps to WebKit/common, and I wanted to get some feedback on the idea. The mock would be shared between all C++ WebKit platforms. (Note that this is for convenience only, a platform could equally use it's own WebKit wrapper around the WebCore mock (eg Mac may do so in ObjectiveC), or use its own mock altogether.)
Of course we also need WebKit wrappers for all of the non-POD types used by the mock's interface, and these have to be common between all platforms. One obvious potential difficulty is the wrapper for WebCore::String. Each platforms already has a wrapper for this type, but there's no guarantee of interoperability, so we'd need to write a new common interface if we're to use the string type.
If a wrapper for string ends up being too problematic, the approach could still be used for mocks that don't need the string type (of which DeviceOrientation is one), but the approach then seems less compelling.
Do people think that this is a reasonable proposal and worth pursuing? Has there been any attempt to do anything similar before? Or is any attempt to write this kind of common WebKit code not worth the effort and destined to failure?
You can see the work in progress for DeviceOrientation at https://bugs.webkit.org/show_bug.cgi?id=39589 and a similar patch for SpeechInput mocks at https://bugs.webkit.org/show_bug.cgi?id=42603
I'd appreciate any feedback you may have.
Thanks, Steve
-- Google UK Limited Registered Office: Belgrave House, 76 Buckingham Palace Road, London SW1W 9TQ Registered in England Number: 3977902
29.07.2010, в 8:16, Adam Barth написал(а):
Plumbing this mock API all the way through WebKit for each port seems like a waste.
One benefit of this is that it makes us test the API layer. Another one is that it gives WebKit developers better familiarity with APIs - that's not as minor as it sounds, as some of us (including myself) have limited knowledge of WebKit APIs, even for their favorite platform. It certainly seems like a waste when private methods are added just for DumpRenderTree support. But maybe it has prompted an API introduction a few times before, I just don't know. This is all getting much better with WebKit2's cross-platform API, although of course we'll need to support current APIs for a long time. - WBR, Alexey Proskuryakov
02.08.2010, в 4:38, Alexey Proskuryakov написал(а):
29.07.2010, в 8:16, Adam Barth написал(а):
Plumbing this mock API all the way through WebKit for each port seems like a waste.
One benefit of this is that it makes us test the API layer. Another one is that it gives WebKit developers better familiarity with APIs - that's not as minor as it sounds, as some of us (including myself) have limited knowledge of WebKit APIs, even for their favorite platform.
It certainly seems like a waste when private methods are added just for DumpRenderTree support. But maybe it has prompted an API introduction a few times before, I just don't know.
This is all getting much better with WebKit2's cross-platform API, although of course we'll need to support current APIs for a long time.
This thread died, but I still don't feel good about mock objects in WebCore. Having test infrastructure in WebCore means shipping unused code in the framework, and as mentioned before, we lack testing of the API layer. We're now getting more mock objects in the tree, such as one for speech synthesis testing. - WBR, Alexey Proskuryakov
Hi Alexey, thank you for revising this topic! I understand your concern about having a testing infrastructure inside the production code. On the other hand, having separate but similar mocks for each port hurts our productivity. And we cannot automate testing without mocks anyway. So how about to have separate "WebCoreTesting" library/framework? The WebCoreTesting will: - access WebCore types directly, as WebKit library does - have mock implementations of WebCore-provided interfaces (abstract classes) and - WebCore will provide abstract classes and injection points for their intenaces, but not provide class implementations - WebKit will provide a priviate method to enable the mocks, which uses WebKitTesting. WebKitTesting library will be linked lazily (using weak symbol mechanism for Mac, or dlopen() family for other ports.) In this approach, we - can split the testing infrastructure out from the production code, just by removing WebCoreTesting library. - can share mock implementations between ports. Even with this approach, some redundancy remains in WebCore, like abstract class for mocks. But actual code, which is a possible cause of vulnerability, can be removed from the production. I once prototyped that approach for introducing WebCore::LayoutTestController. So the patch might help to see what I meant to say: https://bug-42612-attachments.webkit.org/attachment.cgi?id=63403 (Note this patch has rough edges and need to polish anyway.) Believing what I proposed above is a reasonable tradeoff between security/efficiency and productivity, I wonder there might be other possibilities to explore. So I'd love to hear from you and any folks who have interest. Regards. -- morrita On Thu, Oct 28, 2010 at 4:07 AM, Alexey Proskuryakov <ap@webkit.org> wrote:
02.08.2010, в 4:38, Alexey Proskuryakov написал(а):
29.07.2010, в 8:16, Adam Barth написал(а):
Plumbing this mock API all the way through WebKit for each port seems like a waste.
One benefit of this is that it makes us test the API layer. Another one is that it gives WebKit developers better familiarity with APIs - that's not as minor as it sounds, as some of us (including myself) have limited knowledge of WebKit APIs, even for their favorite platform.
It certainly seems like a waste when private methods are added just for DumpRenderTree support. But maybe it has prompted an API introduction a few times before, I just don't know.
This is all getting much better with WebKit2's cross-platform API, although of course we'll need to support current APIs for a long time.
This thread died, but I still don't feel good about mock objects in WebCore. Having test infrastructure in WebCore means shipping unused code in the framework, and as mentioned before, we lack testing of the API layer.
We're now getting more mock objects in the tree, such as one for speech synthesis testing.
- WBR, Alexey Proskuryakov
_______________________________________________ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
-- morrita
Sorry for a delayed response - I hoped for someone else to weigh in. This approach is certainly better than having code in shipping WebCore. But I still think that testing via platform APIs is much more desirable than implementing and maintaining a separate "pseudo-API" for mocks. This can't be too bad for productivity either - the social contract seems to be that you implement DumpRenderTree for one or two platforms at most, and file bugs for others. - WBR, Alexey Proskuryakov 27.10.2010, в 22:08, Hajime Morita написал(а):
Hi Alexey, thank you for revising this topic!
I understand your concern about having a testing infrastructure inside the production code. On the other hand, having separate but similar mocks for each port hurts our productivity. And we cannot automate testing without mocks anyway.
So how about to have separate "WebCoreTesting" library/framework? The WebCoreTesting will: - access WebCore types directly, as WebKit library does - have mock implementations of WebCore-provided interfaces (abstract classes) and - WebCore will provide abstract classes and injection points for their intenaces, but not provide class implementations - WebKit will provide a priviate method to enable the mocks, which uses WebKitTesting. WebKitTesting library will be linked lazily (using weak symbol mechanism for Mac, or dlopen() family for other ports.)
In this approach, we - can split the testing infrastructure out from the production code, just by removing WebCoreTesting library. - can share mock implementations between ports.
Even with this approach, some redundancy remains in WebCore, like abstract class for mocks. But actual code, which is a possible cause of vulnerability, can be removed from the production.
I once prototyped that approach for introducing WebCore::LayoutTestController. So the patch might help to see what I meant to say: https://bug-42612-attachments.webkit.org/attachment.cgi?id=63403 (Note this patch has rough edges and need to polish anyway.)
Believing what I proposed above is a reasonable tradeoff between security/efficiency and productivity, I wonder there might be other possibilities to explore. So I'd love to hear from you and any folks who have interest.
Regards. -- morrita
On Thu, Oct 28, 2010 at 4:07 AM, Alexey Proskuryakov <ap@webkit.org> wrote:
02.08.2010, в 4:38, Alexey Proskuryakov написал(а):
29.07.2010, в 8:16, Adam Barth написал(а):
Plumbing this mock API all the way through WebKit for each port seems like a waste.
One benefit of this is that it makes us test the API layer. Another one is that it gives WebKit developers better familiarity with APIs - that's not as minor as it sounds, as some of us (including myself) have limited knowledge of WebKit APIs, even for their favorite platform.
It certainly seems like a waste when private methods are added just for DumpRenderTree support. But maybe it has prompted an API introduction a few times before, I just don't know.
This is all getting much better with WebKit2's cross-platform API, although of course we'll need to support current APIs for a long time.
This thread died, but I still don't feel good about mock objects in WebCore. Having test infrastructure in WebCore means shipping unused code in the framework, and as mentioned before, we lack testing of the API layer.
We're now getting more mock objects in the tree, such as one for speech synthesis testing.
- WBR, Alexey Proskuryakov
_______________________________________________ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
-- morrita
Hi Alexey, thank you for sharing your thought. I agree that using platform APIs for testing is better than using mocks. On the other hand, I believe the automated testing is our invaluable assert. And some real API is hard to automate. Here is a conflict. I think we need both. Currently WebKit has3 mocks under platform/mock - DeviceOrientationClientMock - GeolocationServiceMock - SpeechInputClientMock It looks that these features are hard to automate without mocks. Another advantage of mocked test is that it is easier to have high code coverage than others, due to the ease to control which mocks provide. So having these mocks somewhere out of WebCore looks a reasonable compromise if we have some manual test counterparts for mocked automated tests. But I'd like to hear the implementors voice, as Alexey hoped so. -- morrita On Thu, Nov 11, 2010 at 2:53 AM, Alexey Proskuryakov <ap@webkit.org> wrote:
Sorry for a delayed response - I hoped for someone else to weigh in.
This approach is certainly better than having code in shipping WebCore. But I still think that testing via platform APIs is much more desirable than implementing and maintaining a separate "pseudo-API" for mocks. This can't be too bad for productivity either - the social contract seems to be that you implement DumpRenderTree for one or two platforms at most, and file bugs for others.
- WBR, Alexey Proskuryakov
27.10.2010, в 22:08, Hajime Morita написал(а):
Hi Alexey, thank you for revising this topic!
I understand your concern about having a testing infrastructure inside the production code. On the other hand, having separate but similar mocks for each port hurts our productivity. And we cannot automate testing without mocks anyway.
So how about to have separate "WebCoreTesting" library/framework? The WebCoreTesting will: - access WebCore types directly, as WebKit library does - have mock implementations of WebCore-provided interfaces (abstract classes) and - WebCore will provide abstract classes and injection points for their intenaces, but not provide class implementations - WebKit will provide a priviate method to enable the mocks, which uses WebKitTesting. WebKitTesting library will be linked lazily (using weak symbol mechanism for Mac, or dlopen() family for other ports.)
In this approach, we - can split the testing infrastructure out from the production code, just by removing WebCoreTesting library. - can share mock implementations between ports.
Even with this approach, some redundancy remains in WebCore, like abstract class for mocks. But actual code, which is a possible cause of vulnerability, can be removed from the production.
I once prototyped that approach for introducing WebCore::LayoutTestController. So the patch might help to see what I meant to say: https://bug-42612-attachments.webkit.org/attachment.cgi?id=63403 (Note this patch has rough edges and need to polish anyway.)
Believing what I proposed above is a reasonable tradeoff between security/efficiency and productivity, I wonder there might be other possibilities to explore. So I'd love to hear from you and any folks who have interest.
Regards. -- morrita
On Thu, Oct 28, 2010 at 4:07 AM, Alexey Proskuryakov <ap@webkit.org> wrote:
02.08.2010, в 4:38, Alexey Proskuryakov написал(а):
29.07.2010, в 8:16, Adam Barth написал(а):
Plumbing this mock API all the way through WebKit for each port seems like a waste.
One benefit of this is that it makes us test the API layer. Another one is that it gives WebKit developers better familiarity with APIs - that's not as minor as it sounds, as some of us (including myself) have limited knowledge of WebKit APIs, even for their favorite platform.
It certainly seems like a waste when private methods are added just for DumpRenderTree support. But maybe it has prompted an API introduction a few times before, I just don't know.
This is all getting much better with WebKit2's cross-platform API, although of course we'll need to support current APIs for a long time.
This thread died, but I still don't feel good about mock objects in WebCore. Having test infrastructure in WebCore means shipping unused code in the framework, and as mentioned before, we lack testing of the API layer.
We're now getting more mock objects in the tree, such as one for speech synthesis testing.
- WBR, Alexey Proskuryakov
_______________________________________________ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
-- morrita
-- morrita
participants (5)
-
Adam Barth
-
Alexey Proskuryakov
-
Hajime Morita
-
Satish Sampath
-
Steve Block