[wpe-webkit] Best way to expose native functionality to HTML

Thu Jun 14 13:34:46 PDT 2018

Hello Johan,

Thank you for your interest in WPE — it is great to see people around trying
to make use of it even when we have only recently started making releases :)

On Fri, 8 Jun 2018 16:26:54 +0100, Johan Saji <johansaji.dev at gmail.com> wrote:

> I was looking into WPE webkit for last couple of months for a project.
> previously we have been using QT Webkit. In Qt webkit, we have
> qtwebkit-bridge for exposing native C++ objects to Javascript environment
> to access these native functionality. We are looking for something similar
> in WPE Webkit. [...]

I took a quick look to qtwebkit-bridge [1] and I am afraid that at the moment
we do not have anything similar in WPE that would allow to easily expose a
GObject to the JavaScript execution context.

The functionality we have at the moment is somewhat lower-level, but known
to work well. There are at least three different mechanisms that can be used
for native functionality to be exposed into the JavaScript context — more
below.

It should be possible implement something similar to qtwebkit-bridge using
infrastructure like GObject-Introspection [2] and Seed [3]. It just does
not exist yet.

> [...] We came across injected bundles in webkit for the same. Is there a
> best method that you recommend.  While searching across the web saw your
> presentation in igalia site, in that for a query, presenter is telling this
> is possible. Are you referring to injected bundles....or is there another
> solution for this.

There are three different ways in which native functionality can be exposed
to JavaScript. You may want to use one or another depending on the needs of
your application. Let's go one by one:

Custom URI scheme handlers
--------------------------

This tells WebKit to intercept loading of resources with a certain URI
scheme defined by the user, and invoke a callback function whenever one
of those are triggered. The communication is synchronous.

Let's imagine that we are implemented a music player, which has a native
media playback backend. We could have a URI scheme named “media”, made
known to WebKit using:

    WebKitWebContext *ctx = webkit_web_view_get_context (web_view);
    webkit_web_context_register_uri_scheme (ctx, "media",
					on_media_uri_requested, NULL, NULL);

Once this has been done, content loaded by the web view can use links
to URIs which use this scheme. For example, the following HTML snippet
could be part of the UI of the application:

    <a class="button-previous" href="media://playlist/previous">Previous</a>
	<a class="button-playpause" href="media://playback/playpause">Play</a>
	<a class="button-next" href="media://playlist/next">Next</a>

The callback function would be triggered when the user clicks in one of the
above links, and it can determine what to do by inspecting the request:

    void on_media_uri_requested (WebKitURISchemeRequest *request,
                                gpointer                user_data) {
	    const gchar *path = webkit_uri_scheme_request_get_path (request);
		if (strcmp (path, "/playback/previous") == 0)
		    play_playlist_previous ();
		else ...

		 webkit_uri_scheme_request_finish (request, result_data_stream,
								result_data_length, result_mime_type);
    }

Note that the callback returns data back, so rather than than this simplistic
example, you may want to use XmlHttpRequest to trigger the “media://...” URIs,
return a stream of JSON data, and then use the JSON result data returned by
the native code in the JS code of the UI.

You can see this in action in Epiphany's code for handling the “about:”
URI scheme:

    https://gitlab.gnome.org/GNOME/epiphany/blob/master/embed/ephy-about-handler.c

User script messages
--------------------

This allows asynchronous communication starting from the JavaScript context
to the native code. Note that this is unidirectional, so if the data sent
requires a reply, additional plumbing may be needed.

Going back to the media player example, this could be used to trigger
showing native UI elements when there is no immediate response, like
opening a preferences dialog, or the song lyrics. First, we register the
message channel with:

    WebKitUserContentManager *ucm =
	    webkit_web_view_get_user_content_manager (web_view);
	webkit_user_content_manager_register_script_message_handler (ucm, "playerUi");

We need to handle the “script-message-received” signal:

    g_signal_connect (ucm, "script-message-received::playerUi",
	    G_CALLBACK (on_player_ui_message_received), NULL);

Now, in order to trigger the native callback function from JavaScript, we
need to send some data using the following:

    window.webkit.messageHandlers.playerUi.postMessage("showPrefs");

I leave implementing the callback function to you. Again, you can find
examples of using user script messages in Epiphany, which provides a few
differente message channels, for example to trigger deletion of Web
applications listed in the “about:applications” page:

    https://gitlab.gnome.org/GNOME/epiphany/blob/master/embed/ephy-embed-shell.c

Triggering JS code execution
----------------------------

This is quite straightforward to explain: the GLib C API provides functions
to run JavaScript code in the same context used by the loaded Web content.
This can be used as an asynchronous, unidirectional communication channel
from the native code to the JavaScript context.

Search for “webkit_web_view_run_javascript()” and
“webkit_web_view_run_javascript_from_gresource()” in the API documentation,
and/or their usage in Epiphany.

WebExtensions
-------------

Injected bundles is the WebKit internal mechanism that we use to provide
a nicer API on top that we call WebExtensions. Here one has complete access
to the JavaScript execution context, and therefore it's possible for the
native code to create new JavaScript objects and functions, which may be
implemented in native code.

I won't get into details because from your e-mail it looks like this is what
you are using already. I wanted to give you a quick tour of the other
mechanisms we have in place, because depending on what you are trying to
achieve, they could be much simpler to use — please do think a bit about this,
and if they suits your needs, consider using the previous methods (which tend
to be simpler).

> [...] while using the Injected bundle, we are seeing an issue that the
> "settimout" call is not working in the callback function invoked from
> bundle.

Regarding this issue, I am going to need more details. What do you mean
with “not working”: does it raise an exception, is the callback passed to
“setTimeout()” not invoked, or is it something else? Who is calling
“setTimeout()”, native code or other JS code? If you can provide a small
test program, that would be great, if not a link to your code (if it's
public) would be most useful.

I hope the above info allows you to choose a simpler way of implementing
your program. If you can provide more information about your issue with
“setTimeout()” then we may be able to help you out with it.

Best regards,

--
 Adrián 🎩

---
[1] https://doc.qt.io/archives/qt-4.8/qtwebkit-bridge.html
[2] https://wiki.gnome.org/Projects/GObjectIntrospection
[3] https://wiki.gnome.org/Projects/Seed
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 195 bytes
Desc: not available
URL: <http://lists.webkit.org/pipermail/webkit-wpe/attachments/20180614/24e57c97/attachment.bin>