[webkit-dev] Implementing the <device> element

[Leandro Graciá Gil]

Hi,

El 10 de febrero de 2011 15:15, Adam Bergkvist
<adam.bergkvist at ericsson.com>escribió:

> Hi,
>
> On 2011-02-09 22:10, Leandro Graciá Gil wrote:
> > So, does that mean that a WebCore platform-independent code is going
> > to determine if a new device handler should be created? I think that
> > some UAs may like to, for example, keep a track of trusted pages for
> > specific types of devices to determine if such a handler should be
> > created. And that is not likely to be implemented in the same way by
> > all the different platforms even if they intend to perform a set of
> > basic common security steps. I completely agree with you that the
> > behaviour should be consistent across platforms, especially the
> > security and privacy aspects, but I don't think that forcing the
> > common code to use available device lists is the way to do it. If we
> > want a consistent behaviour we should ask for it in the specification
> > and leave the implementation specific details open, not the other way
> around.
> >
>
> Yes, the platform independent code in the device element will determine
> if a selection has been made, or altered, in the list of available devices
> Provided by the platform. Before passing it to the device element, the
> list can be filtered by the platform to exclude some entries for whatever
> reason. Selections can also be automatically applied without showing the
> Device selector based on, e.g., stored preferences. The device element
> will examine the list for selection changes to determine if a new device
> handler should be created (and a change event dispatched).
>
> Consider the following example:
> 1. The user clicks the device element button and a list of devices is
>   presented.
> 2. The user selects a webcam in the list and clicks OK.
> *A new Stream is created*
> 3. The user once again clicks the device element button and the list
>   is presented with the webcam selected from before.
> 4. The user clicks OK.
> *No new Stream is created*
> 5. The user once again clicks the device element button and the list
>   is presented with the webcam selected from before.
> 6. The user selects a microphone in the list and clicks OK.
> *A new Stream is created*
>
> This is the behavior that should be consistent between ports.
>

I agree that this behavior should be consisted between ports. However, we
seem to be missing a case here: what happens if the user wants to revoke
access to a device in the middle of a streaming session? We need to be able
to allow this case for obvious privacy reasons.

We need a way to let the UA maintain the lifetime of Stream objects. These
can be killed by the UA at any point (perhaps due to user action or hardware
failure), so we need to specify what happens to these objects in such a
case. We also need to design our implementation in a way that can handle
this situation.

As far as I know, and please correct me if I'm wrong, your proposed design
doesn't provide the user a clear way to invalidate a stream if he wants to.
The Stream's lifetime is something completely handled by the
platform-specific client and with no presence at all in the common WebCore
code. We think this should also be an essential part of the common behaviour
across platforms.

Because of this reason we would like to raise this discussion to the
specification level in the whatwg list. We think that the expected behaviour
for the device element is something that still needs to be discussed before
facing any implementation-specific details.


>
> > One important point that a selection-based design misses is that
> > potentially many devices, if not all, will require exclusive access to
> > them. Let me give a example. We have two different pages with device
> > elements. First, a user selects some device (e.g. a microphone) in the
> > first page, but the page makes no use of it for the moment and hence
> > the real connection is not performed. Minutes later and without
> > closing the first page, the same user selects the same device from
> > before on the second device element (if you have only one microphone
> > and an average user this could be very easy to happen) but this page
> > also makes no use for it at that moment. What happens if for example
> > the first page starts making use of the device and causes the
> > connections on the second page to fail? This is almost for sure not
> > the behaviour that the user is expecting or wanting.
> >
>
> It will be up to the device handler API (e.g. Stream) and its potential
> consumers to handle any errors that may occur. Exclusiveness for certain
> types of devices being one type of error. Another error occurs when a
> peripheral device is disconnected while in use. Even if you have
> established a "connection" to the device via the device element,
> handling such an error in the device handler API is still required since
> it is not possible to revoke a device handler instance.
>

I think this impossibility to revoke a device handler instance is currently
the essence of our main divergence point, and something that should be
discussed in the specification.


>
> If we again take <input type=file> and the File API as a model, there you
> select a filename from a list and get a File object that represents the
> file. The file itself can be removed from disk but the File object still
> exists. When you then attempt to use a FileReader on the File it will fail.
>
> Regarding exclusiveness for Stream objects, we don't have that limitation
> in our implementation. It is possible to select the same webcam in several
> device elements.
>

It's not a limitation but a feature. We're not talking about restricting a
webcam to a single device element, but about having the possibility to do so
if required by the  device itself, the use case or by security reasons. We
should keep in mind that unless the specification finally decides to
implement only the media type we should design in a way that is potentially
compatible with other devices and use cases.


>
> > Yes, you're right on this. I pointed out in my last email that I don't
> > think that there is any problem in having some platform specific code
> > in WebCore as long as any platform can choose to use it or to easily
> > delegate this task to its own WebKit implementation. Sorry if my
> > proposed class diagram mentioned communicating with WebKit as it's not
> > necessarily the case. We have already fixed this for the new diagrams.
> >
>
> Do you see anything in our implementation that could not be easily
> delegated? As I said before, I would be happy to make any changes that
> would be required to implement the platform specific parts for Chromium.
>

And we thank you for that, but before fixing any implementation details we
think that the problem first needs to be discussed at the specification
level.


>
> BR
> Adam
>

Thanks,
Leandro

PS: I'm sorry but I'll be out of the office from tomorrow 16th February to
the 22th (Feb. also). I won't be able to replay quickly to the emails. I'm
very sorry for any inconveniences.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20110215/d14c3d2f/attachment.html>