[webkit-dev] Implementing the <device> element

[Leandro Graciá Gil]

Hi,

El 22 de febrero de 2011 12:20, Adam Bergkvist
<adam.bergkvist at ericsson.com>escribió:

> Hi,
>
> On 2011-02-15 17:50, Leandro Graciá Gil wrote:
> >     Yes, the platform independent code in the device element will
> determine
> >     if a selection has been made, or altered, in the list of available
> >     devices
> >     Provided by the platform. Before passing it to the device element,
> the
> >     list can be filtered by the platform to exclude some entries for
> >     whatever
> >     reason. Selections can also be automatically applied without showing
> the
> >     Device selector based on, e.g., stored preferences. The device
> element
> >     will examine the list for selection changes to determine if a new
> device
> >     handler should be created (and a change event dispatched).
> >
> >     Consider the following example:
> >     1. The user clicks the device element button and a list of devices is
> >        presented.
> >     2. The user selects a webcam in the list and clicks OK.
> >     *A new Stream is created*
> >     3. The user once again clicks the device element button and the list
> >        is presented with the webcam selected from before.
> >     4. The user clicks OK.
> >     *No new Stream is created*
> >     5. The user once again clicks the device element button and the list
> >        is presented with the webcam selected from before.
> >     6. The user selects a microphone in the list and clicks OK.
> >     *A new Stream is created*
> >
> >     This is the behavior that should be consistent between ports.
> >
> >
> > I agree that this behavior should be consisted between ports. However,
> > we seem to be missing a case here: what happens if the user wants to
> > revoke access to a device in the middle of a streaming session? We need
> > to be able to allow this case for obvious privacy reasons.
> >
>
> I agree that this is definitively something we need to consider, but
> perhaps
> as a (near) future improvement. In fact, we have previously been discussing
> how to revoke device access and came to the conclusion that it must be done
> via the browser chrome since the device element can be removed from the
> page
> as soon as access has been granted. A natural choice would be to combine
> this with the mechanism for indicating that a camera/mic is active
> (discussed in the initial device thread
> http://lists.w3.org/Archives/Public/public-device-apis/2009Dec/0149.html).
> That would however require additions to the WebKit API. We have left this
> out in the first version, simply because it is not mentioned in the spec,
> and prioritized to land an initial implemenation that could be used as a
> basis for improvements.
>

I'm glad we agree on the importance of allowing the user to revoke device
access. However we think that this point is important at the design level
rather than the implementation level. If access revoking is introduced, then
the concept of access lifetime arises and needs to be managed within the
code. This access lifetime, let's call it session for instance, is exactly
the concept we introduced (and misnamed) as 'connections'. These sessions
and their management are the core of our proposed model, causing most of the
design differences and problems between our proposals. That's why we would
like to get this point clear at the design level before submitting any
implementation to be patched later.

I also like your proposal of providing access revocation in the same UI used
to notify active device sessions. However I'm not sure about how do you plan
to do this using the chrome client without explicitly managing the device
access lifetime (sessions) in the common WebCore code. It also sounds to me
more like an event that WebCore receives from the UA than a request from
WebCore to the chrome client. Could you please elaborate on this?

Also about your proposal for access revocation, I would like to point
that it's unintuitive to have to click on one button inside the page to
enable access to camera and to have to click on another button, located in a
different place altogether, to end this access. Furthermore, consider mobile
browsers where the screen real state is extremely expensive. Where would you
place this chrome-based UI? To be realistic, I think we'd have to allow
both: let the <device> renderer have state and act as a toggle, as well as
allow the possibility to have the device session ended via some UI element
in the browser's chrome.


>
> > We need a way to let the UA maintain the lifetime of Stream objects.
> > These can be killed by the UA at any point (perhaps due to user action
> > or hardware failure), so we need to specify what happens to these
> > objects in such a case. We also need to design our implementation in a
> > way that can handle this situation.
> >
> > As far as I know, and please correct me if I'm wrong, your proposed
> > design doesn't provide the user a clear way to invalidate a stream if he
> > wants to. The Stream's lifetime is something completely handled by the
> > platform-specific client and with no presence at all in the common
> > WebCore code. We think this should also be an essential part of the
> > common behaviour across platforms.
> >
>
> The patch for bug 47264 does not contain anything related to Stream; it is
> a device element only patch (bug 47265 is for the Stream API). We should
> not confuse the lifetime of Stream objects (or any other device handler
> API)
> with the API's way of handling errors (e.g. that its access to the device
> has been revoked). Revoking access to a device should probably be handled
> in
> the same way (in the device handler API) as physically unplugging the
> device.
>

I agree with your last statement about handling the access revocation in the
same way that physically unplugging the device.

Since some of our discussion points are about design and implementation
details for the Stream API, would you mind to upload a patch with your
proposal for it? Not necessarily for review, but for allowing all the people
interested in this to review and analyze it.


> >     It will be up to the device handler API (e.g. Stream) and its
> potential
> >     consumers to handle any errors that may occur. Exclusiveness for
> certain
> >     types of devices being one type of error. Another error occurs when a
> >     peripheral device is disconnected while in use. Even if you have
> >     established a "connection" to the device via the device element,
> >     handling such an error in the device handler API is still required
> since
> >     it is not possible to revoke a device handler instance.
> >
> >
> > I think this impossibility to revoke a device handler instance is
> > currently the essence of our main divergence point, and something that
> > should be discussed in the specification.
> >
>
> Let me clearify what I meant. It is not possible to revoke the actual
> instance
> of the device handler. The device handler API would need to handle that it
> no
> longer has access to the device it was created to control.
>
> >     Do you see anything in our implementation that could not be easily
> >     delegated? As I said before, I would be happy to make any changes
> that
> >     would be required to implement the platform specific parts for
> Chromium.
> >
> >
> > And we thank you for that, but before fixing any implementation details
> > we think that the problem first needs to be discussed at the
> > specification level.
> >
>
> According to Hixie, the specification is in a state where implementation
> experience is needed to drive the spec forward. We should try to land the
> initial implementation and then incrementally improve it.
>

 I think that implementation feedback is exactly what we are getting from
this discussion, especially since we have two implementation candidates with
different approaches. This feedback it's already achieving its purpose: we
are arising new questions about the specification that need to be adressed
and solved.

We are not worried about minor implementation details or incompatibilities
that can be easily patched, but about basic design differences caused by
different expectations of what the device element should be and do according
to the current specification. We think that trying to land any of the
proposals without agreeing on these basic points will leave most of the code
not being used by each other, not helping the specification at all because
of the divergence of expectations and design assumptions.


> BR
> Adam
>

Thanks,
Leandro
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20110228/b9e76966/attachment.html>