[webkit-dev] Implementing the <device> element

[Adam Bergkvist]

Hi,

On 2011-02-28 19:01, Leandro Graciá Gil wrote:
> I'm glad we agree on the importance of allowing the user to revoke 
> device access. However we think that this point is important at the 
> design level rather than the implementation level. If access revoking 
> is introduced, then the concept of access lifetime arises and needs to 
> be managed within the code. This access lifetime, let's call it 
> session for instance, is exactly the concept we introduced (and 
> misnamed) as 'connections'. These sessions and their management are 
> the core of our proposed model, causing most of the design differences 
> and problems between our proposals. That's why we would like to get 
> this point clear at the design level before submitting any 
> implementation to be patched later.
>

The "concept of access lifetime" already exists since a device can be
physically disconnected at any time and the device handler APIs will
need to handle that the underlying device is unavailable. The main
concern I have with your session concept is that it involves the device
element. Our approach, which is in line with the discussions about the
device element being modeled after <input type="file">, implements the
device element as a device selector and a factory for e.g. Stream
objects, which no longer depend on the device element that created them
(similar to File objects). As a result, the same device element could
be used over and over again to create new Stream objects. I assume that
is not possible in your design since a device element enters a state
where it manages a session to the previously selected device(s)).
Having the device element involved in the session feels strange when
it can be removed from the page while the Stream object continues to
exist. Similar to how File objects work, we do not see a "session" being
started when you select a device, but rather when the device is used.

We see the indicator in the browser chrome as a more important feature,
since it is uncertain how common it will be that you trust a page enough
to grant it access to you camera and then stop trusting it, but still
want to stay on the page instead of simply closing it.

> I also like your proposal of providing access revocation in the same 
> UI used to notify active device sessions. However I'm not sure about 
> how do you plan to do this using the chrome client without explicitly 
> managing the device access lifetime (sessions) in the common WebCore 
> code. It also sounds to me more like an event that WebCore receives 
> from the UA than a request from WebCore to the chrome client. Could 
> you please elaborate on this?
>

As mentioned above, our view of a session differs from yours. As a
result, we will not toggle the indicators in the browser chrome based
on the device being in use, but rather on information that a page has
been granted access to use a device. To toggle the indicators in the
chrome, and possibly offer revoking, we will need to expose
WebCore/WebKit APIs to explicitly manage device status and access.

> Also about your proposal for access revocation, I would like to point 
> that it's unintuitive to have to click on one button inside the page 
> to enable access to camera and to have to click on another button, 
> located in a different place altogether, to end this access. 
> Furthermore, consider mobile browsers where the screen real state is 
> extremely expensive. Where would you place this chrome-based UI? To be 
> realistic, I think we'd have to allow both: let the <device> renderer 
> have state and act as a toggle, as well as allow the possibility to 
> have the device session ended via some UI element in the browser's chrome.
>

I agree with you that it is not ideal to grant and revoke access from two
different places. However, having the revoke button (rendered by the device
element) in the page is dangerous since it can be replaced by a button that
mimics the device element, but actually does not revoke access. This gives
a sense of false security. The user must check the indicator in the chrome
to verify that the access actually has been revoked and we cannot rely on
the user doing that. The fact that the user anyhow must check (and possibly
revoke access) from the chrome makes the posibility to revoke from the page
questionable.


> I agree with your last statement about handling the access revocation 
> in the same way that physically unplugging the device.
> 
> Since some of our discussion points are about design and 
> implementation details for the Stream API, would you mind to upload a 
> patch with your proposal for it? Not necessarily for review, but for 
> allowing all the people interested in this to review and analyze it.
>

The majority of our Stream API implementation platform/component (GStreamer)
dependent code that would not add so much to this discusson. Basically the
Stream is a quite thin JavaScript object which references pipelines in the
media backend.

>   I think that implementation feedback is exactly what we are getting 
> from this discussion, especially since we have two implementation 
> candidates with different approaches. This feedback it's already 
> achieving its purpose: we are arising new questions about the 
> specification that need to be adressed and solved.
>
> We are not worried about minor implementation details or 
> incompatibilities that can be easily patched, but about basic design 
> differences caused by different expectations of what the device 
> element should be and do according to the current specification. We 
> think that trying to land any of the proposals without agreeing on 
> these basic points will leave most of the code not being used by each 
> other, not helping the specification at all because of the divergence 
> of expectations and design assumptions.
>

The device element specification, in its current state, is rather slim and
gives a lot of room for interpretation. There is even a note that states
that it is an "idea; yet to be specified".  However, by gathering bits of
information from various sources (e.g. public mailing lists and IRC logs)
you can get hints how the device element is intended to work. That is what
we have done and that is why we interpret <device> as the equivalent of
<input type=file> for devices.

BR
Adam