[webkit-changes] [WebKit/WebKit] 6b87ac: [MediaStream] Add support for ImageCapture.takePhoto

[Eric Carlson]

  Branch: refs/heads/main
  Home:   https://github.com/WebKit/WebKit
  Commit: 6b87ac816d8acb2ee9016d88e7a5d639b19e5818
      https://github.com/WebKit/WebKit/commit/6b87ac816d8acb2ee9016d88e7a5d639b19e5818
  Author: Eric Carlson <eric.carlson at apple.com>
  Date:   2023-11-12 (Sun, 12 Nov 2023)

  Changed paths:
    A LayoutTests/fast/mediastream/image-capture-take-photo-expected.txt
    A LayoutTests/fast/mediastream/image-capture-take-photo.html
    M Source/WebCore/Modules/mediastream/ImageCapture.cpp
    M Source/WebCore/Modules/mediastream/ImageCapture.h
    M Source/WebCore/Modules/mediastream/ImageCapture.idl
    M Source/WebCore/Modules/mediastream/MediaStreamTrack.cpp
    M Source/WebCore/Modules/mediastream/MediaStreamTrack.h
    M Source/WebCore/PAL/pal/cocoa/AVFoundationSoftLink.h
    M Source/WebCore/PAL/pal/cocoa/AVFoundationSoftLink.mm
    M Source/WebCore/dom/TaskSource.h
    M Source/WebCore/platform/mediastream/MediaStreamTrackPrivate.cpp
    M Source/WebCore/platform/mediastream/MediaStreamTrackPrivate.h
    M Source/WebCore/platform/mediastream/PhotoSettings.h
    M Source/WebCore/platform/mediastream/RealtimeMediaSource.cpp
    M Source/WebCore/platform/mediastream/RealtimeMediaSource.h
    M Source/WebCore/platform/mediastream/RealtimeVideoCaptureSource.cpp
    M Source/WebCore/platform/mediastream/RealtimeVideoCaptureSource.h
    M Source/WebCore/platform/mediastream/gstreamer/MockRealtimeVideoSourceGStreamer.cpp
    M Source/WebCore/platform/mediastream/mac/AVVideoCaptureSource.h
    M Source/WebCore/platform/mediastream/mac/AVVideoCaptureSource.mm
    M Source/WebCore/platform/mediastream/mac/MockRealtimeVideoSourceMac.mm
    M Source/WebCore/platform/mock/MockRealtimeVideoSource.cpp
    M Source/WebCore/platform/mock/MockRealtimeVideoSource.h
    M Source/WebKit/GPUProcess/webrtc/RemoteSampleBufferDisplayLayer.h
    M Source/WebKit/UIProcess/Cocoa/UserMediaCaptureManagerProxy.cpp
    M Source/WebKit/UIProcess/Cocoa/UserMediaCaptureManagerProxy.h
    M Source/WebKit/UIProcess/Cocoa/UserMediaCaptureManagerProxy.messages.in
    M Source/WebKit/WebProcess/cocoa/RemoteRealtimeMediaSource.cpp
    M Source/WebKit/WebProcess/cocoa/RemoteRealtimeMediaSource.h
    M Source/WebKit/WebProcess/cocoa/RemoteRealtimeMediaSourceProxy.cpp
    M Source/WebKit/WebProcess/cocoa/RemoteRealtimeMediaSourceProxy.h

  Log Message:
  -----------
  [MediaStream] Add support for ImageCapture.takePhoto
https://bugs.webkit.org/show_bug.cgi?id=262467
rdar://116322637

Reviewed by Youenn Fablet.

Implement ImageCapture.takePhoto in AVVideoCaptureSource and MockRealtimeVideoSource.
We reconfigure the capture source when asked for a photo larger than the current preset,
so it always returns an image at least as large as the requested size, but it doesn't
yet resize the image if the requested size doesn't exactly match the preset.

Taking a photo is an asynchronous operation that can take an arbitrary amount of time, so
we don't do it on the main thread. Because we may need to reconfigure the capture device
to take a photo with the requested settings, we need to prevent the capture device from
being reconfigured by `applyConstraints` while a photo is being generated. To do this we
serialize calls to `takePhoto` and `applyConstraints` so they don't interfere with one
another.

* LayoutTests/fast/mediastream/image-capture-take-photo-expected.txt: Added.
* LayoutTests/fast/mediastream/image-capture-take-photo.html: Added.
* Source/WebCore/Modules/mediastream/ImageCapture.cpp:
(WebCore::ImageCapture::takePhoto):
* Source/WebCore/Modules/mediastream/ImageCapture.h:
* Source/WebCore/Modules/mediastream/ImageCapture.idl:
* Source/WebCore/Modules/mediastream/MediaStreamTrack.cpp:
(WebCore::MediaStreamTrack::stopTrack):
(WebCore::MediaStreamTrack::queueAndProcessSerialAction):
(WebCore::MediaStreamTrack::processNextSerialAction):
(WebCore::MediaStreamTrack::takePhoto):
(WebCore::MediaStreamTrack::applyConstraints):
* Source/WebCore/Modules/mediastream/MediaStreamTrack.h:
* Source/WebCore/PAL/pal/cocoa/AVFoundationSoftLink.h:
* Source/WebCore/PAL/pal/cocoa/AVFoundationSoftLink.mm:
* Source/WebCore/platform/mediastream/MediaStreamTrackPrivate.cpp:
(WebCore::MediaStreamTrackPrivate::takePhoto):
* Source/WebCore/platform/mediastream/MediaStreamTrackPrivate.h:
* Source/WebCore/platform/mediastream/PhotoSettings.h:
(WebCore::operator==):
* Source/WebCore/platform/mediastream/RealtimeMediaSource.cpp:
(WebCore::RealtimeMediaSource::takePhoto):
* Source/WebCore/platform/mediastream/RealtimeMediaSource.h:
* Source/WebCore/platform/mediastream/RealtimeVideoCaptureSource.cpp:
(WebCore::RealtimeVideoCaptureSource::bestSupportedSizeFrameRateAndZoomConsideringObservers):
(WebCore::RealtimeVideoCaptureSource::setSizeFrameRateAndZoom):
(WebCore::RealtimeVideoCaptureSource::takePhotoInternal):
(WebCore::RealtimeVideoCaptureSource::takePhoto):
* Source/WebCore/platform/mediastream/RealtimeVideoCaptureSource.h:
* Source/WebCore/platform/mediastream/mac/AVVideoCaptureSource.h:
* Source/WebCore/platform/mediastream/mac/AVVideoCaptureSource.mm:
(WebCore::toFillLightMode):
(WebCore::toAVCaptureFlashMode):
(WebCore::photoQueue):
(WebCore::AVVideoCaptureSource::~AVVideoCaptureSource):
(WebCore::AVVideoCaptureSource::stopProducingData):
(WebCore::AVVideoCaptureSource::stopSession):
(WebCore::AVVideoCaptureSource::photoOutput):
(WebCore::AVVideoCaptureSource::resolvePendingPhotoRequest):
(WebCore::AVVideoCaptureSource::rejectPendingPhotoRequest):
(WebCore::AVVideoCaptureSource::maxPhotoSizeForCurrentPreset const):
(WebCore::AVVideoCaptureSource::photoConfiguration):
(WebCore::AVVideoCaptureSource::takePhotoInternal):
(WebCore::AVVideoCaptureSource::getPhotoSettings):
(WebCore::AVVideoCaptureSource::captureOutputDidFinishProcessingPhoto):
(-[WebCoreAVVideoCaptureSourceObserver captureOutput:didFinishProcessingPhoto:error:]):
* Source/WebCore/platform/mediastream/mac/MockRealtimeVideoSourceMac.mm:
(WebCore::MockRealtimeVideoSourceMac::updateSampleBuffer):
* Source/WebCore/platform/mock/MockRealtimeVideoSource.cpp:
(WebCore::takePhotoRunLoop):
(WebCore::MockRealtimeVideoSource::DrawingState::fontDescription):
(WebCore::MockRealtimeVideoSource::DrawingState::timeFont):
(WebCore::MockRealtimeVideoSource::DrawingState::bipBopFont):
(WebCore::MockRealtimeVideoSource::DrawingState::statsFont):
(WebCore::MockRealtimeVideoSource::takePhotoInternal):
(WebCore::MockRealtimeVideoSource::invalidateDrawingState):
(WebCore::MockRealtimeVideoSource::drawingState):
(WebCore::MockRealtimeVideoSource::settingsDidChange):
(WebCore::MockRealtimeVideoSource::drawText):
(WebCore::MockRealtimeVideoSource::generatePhoto):
(WebCore::MockRealtimeVideoSource::generateFrameInternal):
(WebCore::MockRealtimeVideoSource::generateFrame):
(WebCore::MockRealtimeVideoSource::imageBuffer):
(WebCore::MockRealtimeVideoSource::imageBufferInternal):
(WebCore::MockRealtimeVideoSource::imageBuffer const): Deleted.
* Source/WebCore/platform/mock/MockRealtimeVideoSource.h:
(WebCore::MockRealtimeVideoSource::DrawingState::DrawingState):
(WebCore::MockRealtimeVideoSource::DrawingState::baseFontSize const):
(WebCore::MockRealtimeVideoSource::DrawingState::statsFontSize const):
* Source/WebKit/GPUProcess/webrtc/RemoteSampleBufferDisplayLayer.h:
* Source/WebKit/UIProcess/Cocoa/UserMediaCaptureManagerProxy.cpp:
(WebKit::UserMediaCaptureManagerProxy::SourceProxy::takePhoto):
(WebKit::UserMediaCaptureManagerProxy::takePhoto):
* Source/WebKit/UIProcess/Cocoa/UserMediaCaptureManagerProxy.h:
* Source/WebKit/UIProcess/Cocoa/UserMediaCaptureManagerProxy.messages.in:
* Source/WebKit/WebProcess/cocoa/RemoteRealtimeMediaSource.cpp:
(WebKit::RemoteRealtimeMediaSource::takePhoto):
* Source/WebKit/WebProcess/cocoa/RemoteRealtimeMediaSource.h:
* Source/WebKit/WebProcess/cocoa/RemoteRealtimeMediaSourceProxy.cpp:
(WebKit::RemoteRealtimeMediaSourceProxy::takePhoto):
* Source/WebKit/WebProcess/cocoa/RemoteRealtimeMediaSourceProxy.h:
ef8342e

Canonical link: https://commits.webkit.org/270621@main