<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
"http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head><meta http-equiv="content-type" content="text/html; charset=utf-8" />
<title>[214806] trunk</title>
</head>
<body>
<style type="text/css"><!--
#msg dl.meta { border: 1px #006 solid; background: #369; padding: 6px; color: #fff; }
#msg dl.meta dt { float: left; width: 6em; font-weight: bold; }
#msg dt:after { content:':';}
#msg dl, #msg dt, #msg ul, #msg li, #header, #footer, #logmsg { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt; }
#msg dl a { font-weight: bold}
#msg dl a:link { color:#fc3; }
#msg dl a:active { color:#ff0; }
#msg dl a:visited { color:#cc6; }
h3 { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt; font-weight: bold; }
#msg pre { overflow: auto; background: #ffc; border: 1px #fa0 solid; padding: 6px; }
#logmsg { background: #ffc; border: 1px #fa0 solid; padding: 1em 1em 0 1em; }
#logmsg p, #logmsg pre, #logmsg blockquote { margin: 0 0 1em 0; }
#logmsg p, #logmsg li, #logmsg dt, #logmsg dd { line-height: 14pt; }
#logmsg h1, #logmsg h2, #logmsg h3, #logmsg h4, #logmsg h5, #logmsg h6 { margin: .5em 0; }
#logmsg h1:first-child, #logmsg h2:first-child, #logmsg h3:first-child, #logmsg h4:first-child, #logmsg h5:first-child, #logmsg h6:first-child { margin-top: 0; }
#logmsg ul, #logmsg ol { padding: 0; list-style-position: inside; margin: 0 0 0 1em; }
#logmsg ul { text-indent: -1em; padding-left: 1em; }#logmsg ol { text-indent: -1.5em; padding-left: 1.5em; }
#logmsg > ul, #logmsg > ol { margin: 0 0 1em 0; }
#logmsg pre { background: #eee; padding: 1em; }
#logmsg blockquote { border: 1px solid #fa0; border-left-width: 10px; padding: 1em 1em 0 1em; background: white;}
#logmsg dl { margin: 0; }
#logmsg dt { font-weight: bold; }
#logmsg dd { margin: 0; padding: 0 0 0.5em 0; }
#logmsg dd:before { content:'\00bb';}
#logmsg table { border-spacing: 0px; border-collapse: collapse; border-top: 4px solid #fa0; border-bottom: 1px solid #fa0; background: #fff; }
#logmsg table th { text-align: left; font-weight: normal; padding: 0.2em 0.5em; border-top: 1px dotted #fa0; }
#logmsg table td { text-align: right; border-top: 1px dotted #fa0; padding: 0.2em 0.5em; }
#logmsg table thead th { text-align: center; border-bottom: 1px solid #fa0; }
#logmsg table th.Corner { text-align: left; }
#logmsg hr { border: none 0; border-top: 2px dashed #fa0; height: 1px; }
#header, #footer { color: #fff; background: #636; border: 1px #300 solid; padding: 6px; }
#patch { width: 100%; }
#patch h4 {font-family: verdana,arial,helvetica,sans-serif;font-size:10pt;padding:8px;background:#369;color:#fff;margin:0;}
#patch .propset h4, #patch .binary h4 {margin:0;}
#patch pre {padding:0;line-height:1.2em;margin:0;}
#patch .diff {width:100%;background:#eee;padding: 0 0 10px 0;overflow:auto;}
#patch .propset .diff, #patch .binary .diff {padding:10px 0;}
#patch span {display:block;padding:0 10px;}
#patch .modfile, #patch .addfile, #patch .delfile, #patch .propset, #patch .binary, #patch .copfile {border:1px solid #ccc;margin:10px 0;}
#patch ins {background:#dfd;text-decoration:none;display:block;padding:0 10px;}
#patch del {background:#fdd;text-decoration:none;display:block;padding:0 10px;}
#patch .lines, .info {color:#888;background:#fff;}
--></style>
<div id="msg">
<dl class="meta">
<dt>Revision</dt> <dd><a href="http://trac.webkit.org/projects/webkit/changeset/214806">214806</a></dd>
<dt>Author</dt> <dd>commit-queue@webkit.org</dd>
<dt>Date</dt> <dd>2017-04-03 09:59:41 -0700 (Mon, 03 Apr 2017)</dd>
</dl>
<h3>Log Message</h3>
<pre>captureStream is getting black frames with webgl canvas
https://bugs.webkit.org/show_bug.cgi?id=170325
Patch by Youenn Fablet <youenn@apple.com> on 2017-04-03
Reviewed by Dean Jackson.
Source/WebCore:
Test: fast/mediastream/captureStream/canvas3d.html
Changing the webgl context to save buffers in case the canvas is captured.
Adding a canvas changed notification in case of clear.
In the future, we might want to change this notification and do it when endPaint or similar is called.
Adding an Internals API to grab the RGBA equivalent of the next track frame.
For that purpose, adding a bunch of WEBCORE_EXPORT.
* Modules/mediastream/CanvasCaptureMediaStreamTrack.cpp:
(WebCore::CanvasCaptureMediaStreamTrack::Source::Source): Adding constraints support so that track settings
getter actually transmits the width and height of the source.
(WebCore::CanvasCaptureMediaStreamTrack::Source::canvasChanged): ensuring webgl canvas context keep their drawing buffer.
* Modules/mediastream/MediaStreamTrack.h:
* bindings/js/JSDOMGuardedObject.h:
* bindings/js/JSDOMPromise.h:
(WebCore::DeferredPromise::resolve):
(WebCore::DeferredPromise::reject):
* dom/ActiveDOMCallback.h:
* html/HTMLCanvasElement.cpp:
(WebCore::HTMLCanvasElement::captureStream):
* html/ImageData.h:
* html/ImageData.idl:
* html/canvas/WebGLRenderingContext.cpp:
(WebCore::WebGLRenderingContext::clear): ensuring canvas observers get notified in case of clear calls.
* html/canvas/WebGLRenderingContextBase.h:
(WebCore::WebGLRenderingContextBase::preserveDrawingBuffer): Added to allow canvas capture to update this property.
* platform/MediaSample.h:
(WebCore::MediaSample::getRGBAImageData): Added for internals API.
* platform/graphics/avfoundation/MediaSampleAVFObjC.h:
* platform/graphics/avfoundation/objc/MediaSampleAVFObjC.mm:
(WebCore::MediaSampleAVFObjC::getRGBAImageData):
* platform/graphics/cv/PixelBufferConformerCV.cpp:
(WebCore::PixelBufferConformerCV::convert): Helper routine for getRGBAImageData.
* platform/graphics/cv/PixelBufferConformerCV.h:
* platform/mediastream/RealtimeMediaSourceSettings.h:
(WebCore::RealtimeMediaSourceSettings::setSupportedConstraints):
(WebCore::RealtimeMediaSourceSettings::setSupportedConstraits): Deleted.
* platform/mediastream/mac/AVMediaCaptureSource.mm:
(WebCore::AVMediaCaptureSource::initializeSettings):
* platform/mediastream/openwebrtc/RealtimeAudioSourceOwr.h:
* platform/mediastream/openwebrtc/RealtimeVideoSourceOwr.h:
* platform/mock/MockRealtimeMediaSource.cpp:
(WebCore::MockRealtimeMediaSource::initializeSettings):
* testing/Internals.cpp:
(WebCore::Internals::grabNextMediaStreamTrackFrame):
(WebCore::Internals::videoSampleAvailable):
* testing/Internals.h:
* testing/Internals.idl:
LayoutTests:
* fast/mediastream/captureStream/canvas3d-expected.txt: Added.
* fast/mediastream/captureStream/canvas3d.html: Added.</pre>
<h3>Modified Paths</h3>
<ul>
<li><a href="#trunkLayoutTestsChangeLog">trunk/LayoutTests/ChangeLog</a></li>
<li><a href="#trunkSourceWebCoreChangeLog">trunk/Source/WebCore/ChangeLog</a></li>
<li><a href="#trunkSourceWebCoreModulesmediastreamCanvasCaptureMediaStreamTrackcpp">trunk/Source/WebCore/Modules/mediastream/CanvasCaptureMediaStreamTrack.cpp</a></li>
<li><a href="#trunkSourceWebCoreModulesmediastreamMediaStreamTrackh">trunk/Source/WebCore/Modules/mediastream/MediaStreamTrack.h</a></li>
<li><a href="#trunkSourceWebCorebindingsjsJSDOMGuardedObjecth">trunk/Source/WebCore/bindings/js/JSDOMGuardedObject.h</a></li>
<li><a href="#trunkSourceWebCorebindingsjsJSDOMPromiseh">trunk/Source/WebCore/bindings/js/JSDOMPromise.h</a></li>
<li><a href="#trunkSourceWebCoredomActiveDOMCallbackh">trunk/Source/WebCore/dom/ActiveDOMCallback.h</a></li>
<li><a href="#trunkSourceWebCorehtmlHTMLCanvasElementcpp">trunk/Source/WebCore/html/HTMLCanvasElement.cpp</a></li>
<li><a href="#trunkSourceWebCorehtmlImageDatah">trunk/Source/WebCore/html/ImageData.h</a></li>
<li><a href="#trunkSourceWebCorehtmlImageDataidl">trunk/Source/WebCore/html/ImageData.idl</a></li>
<li><a href="#trunkSourceWebCorehtmlcanvasWebGLRenderingContextcpp">trunk/Source/WebCore/html/canvas/WebGLRenderingContext.cpp</a></li>
<li><a href="#trunkSourceWebCorehtmlcanvasWebGLRenderingContextBaseh">trunk/Source/WebCore/html/canvas/WebGLRenderingContextBase.h</a></li>
<li><a href="#trunkSourceWebCoreplatformMediaSampleh">trunk/Source/WebCore/platform/MediaSample.h</a></li>
<li><a href="#trunkSourceWebCoreplatformgraphicsavfoundationMediaSampleAVFObjCh">trunk/Source/WebCore/platform/graphics/avfoundation/MediaSampleAVFObjC.h</a></li>
<li><a href="#trunkSourceWebCoreplatformgraphicsavfoundationobjcMediaSampleAVFObjCmm">trunk/Source/WebCore/platform/graphics/avfoundation/objc/MediaSampleAVFObjC.mm</a></li>
<li><a href="#trunkSourceWebCoreplatformgraphicscvPixelBufferConformerCVcpp">trunk/Source/WebCore/platform/graphics/cv/PixelBufferConformerCV.cpp</a></li>
<li><a href="#trunkSourceWebCoreplatformgraphicscvPixelBufferConformerCVh">trunk/Source/WebCore/platform/graphics/cv/PixelBufferConformerCV.h</a></li>
<li><a href="#trunkSourceWebCoreplatformmediastreamRealtimeMediaSourceSettingsh">trunk/Source/WebCore/platform/mediastream/RealtimeMediaSourceSettings.h</a></li>
<li><a href="#trunkSourceWebCoreplatformmediastreammacAVMediaCaptureSourcemm">trunk/Source/WebCore/platform/mediastream/mac/AVMediaCaptureSource.mm</a></li>
<li><a href="#trunkSourceWebCoreplatformmediastreamopenwebrtcRealtimeAudioSourceOwrh">trunk/Source/WebCore/platform/mediastream/openwebrtc/RealtimeAudioSourceOwr.h</a></li>
<li><a href="#trunkSourceWebCoreplatformmediastreamopenwebrtcRealtimeVideoSourceOwrh">trunk/Source/WebCore/platform/mediastream/openwebrtc/RealtimeVideoSourceOwr.h</a></li>
<li><a href="#trunkSourceWebCoreplatformmockMockRealtimeMediaSourcecpp">trunk/Source/WebCore/platform/mock/MockRealtimeMediaSource.cpp</a></li>
<li><a href="#trunkSourceWebCoretestingInternalscpp">trunk/Source/WebCore/testing/Internals.cpp</a></li>
<li><a href="#trunkSourceWebCoretestingInternalsh">trunk/Source/WebCore/testing/Internals.h</a></li>
<li><a href="#trunkSourceWebCoretestingInternalsidl">trunk/Source/WebCore/testing/Internals.idl</a></li>
</ul>
<h3>Added Paths</h3>
<ul>
<li><a href="#trunkLayoutTestsfastmediastreamcaptureStreamcanvas3dexpectedtxt">trunk/LayoutTests/fast/mediastream/captureStream/canvas3d-expected.txt</a></li>
<li><a href="#trunkLayoutTestsfastmediastreamcaptureStreamcanvas3dhtml">trunk/LayoutTests/fast/mediastream/captureStream/canvas3d.html</a></li>
</ul>
</div>
<div id="patch">
<h3>Diff</h3>
<a id="trunkLayoutTestsChangeLog"></a>
<div class="modfile"><h4>Modified: trunk/LayoutTests/ChangeLog (214805 => 214806)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/LayoutTests/ChangeLog 2017-04-03 16:54:49 UTC (rev 214805)
+++ trunk/LayoutTests/ChangeLog 2017-04-03 16:59:41 UTC (rev 214806)
</span><span class="lines">@@ -1,3 +1,13 @@
</span><ins>+2017-04-03 Youenn Fablet <youenn@apple.com>
+
+ captureStream is getting black frames with webgl canvas
+ https://bugs.webkit.org/show_bug.cgi?id=170325
+
+ Reviewed by Dean Jackson.
+
+ * fast/mediastream/captureStream/canvas3d-expected.txt: Added.
+ * fast/mediastream/captureStream/canvas3d.html: Added.
+
</ins><span class="cx"> 2017-04-03 Per Arne Vollan <pvollan@apple.com>
</span><span class="cx">
</span><span class="cx"> Implement stroke-miterlimit.
</span></span></pre></div>
<a id="trunkLayoutTestsfastmediastreamcaptureStreamcanvas3dexpectedtxt"></a>
<div class="addfile"><h4>Added: trunk/LayoutTests/fast/mediastream/captureStream/canvas3d-expected.txt (0 => 214806)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/LayoutTests/fast/mediastream/captureStream/canvas3d-expected.txt (rev 0)
+++ trunk/LayoutTests/fast/mediastream/captureStream/canvas3d-expected.txt 2017-04-03 16:59:41 UTC (rev 214806)
</span><span class="lines">@@ -0,0 +1,7 @@
</span><ins>+CONSOLE MESSAGE: line 23: Turning drawing buffer preservation for the WebGL canvas being captured
+
+
+
+PASS captureStream with 3d context drawing - not buffered initially
+PASS captureStream with 3d context drawing - buffered initially
+
</ins></span></pre></div>
<a id="trunkLayoutTestsfastmediastreamcaptureStreamcanvas3dhtml"></a>
<div class="addfile"><h4>Added: trunk/LayoutTests/fast/mediastream/captureStream/canvas3d.html (0 => 214806)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/LayoutTests/fast/mediastream/captureStream/canvas3d.html (rev 0)
+++ trunk/LayoutTests/fast/mediastream/captureStream/canvas3d.html 2017-04-03 16:59:41 UTC (rev 214806)
</span><span class="lines">@@ -0,0 +1,81 @@
</span><ins>+<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML//EN">
+<html>
+ <body>
+ <canvas id="canvas1" width=400px height=400px></canvas>
+ <video id="video1" controls autoplay width=400px height=400px></video>
+ <br>
+ <canvas id="canvas2" width=400px height=400px></canvas>
+ <video id="video2" controls autoplay width=400px height=400px></video>
+ <script src="../../../resources/testharness.js"></script>
+ <script src="../../../resources/testharnessreport.js"></script>
+ <script>
+
+var canvas1 = document.getElementById("canvas1");
+var canvas2 = document.getElementById("canvas2");
+var video1 = document.getElementById("video1");
+var video2 = document.getElementById("video2");
+var gl1 = canvas1.getContext('webgl', { preserveDrawingBuffer: false } );
+var gl2 = canvas2.getContext('webgl', { preserveDrawingBuffer: true } );
+
+function modifyCanvas(gl, green)
+{
+ gl.clearColor(0, green ? 1 : 0, 0, 1);
+ gl.clear(gl.COLOR_BUFFER_BIT);
+
+ setTimeout(() => { modifyCanvas(gl, !green) }, 500);
+}
+
+function checkGreenPixel(value, cptr)
+{
+ var index = cptr % 4;
+ if (index == 0)
+ return value == 0;
+ if (index == 1)
+ return value == 255;
+ if (index == 2)
+ return value == 0;
+ if (index == 3)
+ return value == 255;
+}
+
+promise_test((test) => {
+ var stream = canvas1.captureStream();
+ video1.srcObject = stream;
+
+ if (!window.internals)
+ var promise = Promise.resolve();
+ else {
+ internals.observeMediaStreamTrack(stream.getVideoTracks()[0]);
+ var promise = internals.grabNextMediaStreamTrackFrame().then((data) => {
+ data.data.forEach((value, cptr) => {
+ assert_true(checkGreenPixel(value, cptr), "expecting value " + cptr + " to be part of a green pixel");
+ });
+ })
+ }
+
+ modifyCanvas(gl1, true);
+ return promise;
+}, "captureStream with 3d context drawing - not buffered initially");
+
+promise_test((test) => {
+ var stream = canvas2.captureStream();
+ video2.srcObject = stream;
+
+ if (!window.internals)
+ var promise = Promise.resolve();
+ else {
+ internals.observeMediaStreamTrack(stream.getVideoTracks()[0]);
+ var promise = internals.grabNextMediaStreamTrackFrame().then((data) => {
+ data.data.forEach((value, cptr) => {
+ assert_true(checkGreenPixel(value, cptr), "expecting value " + cptr + " to be part of a green pixel");
+ });
+ })
+ }
+
+ modifyCanvas(gl2, true);
+ return promise;
+}, "captureStream with 3d context drawing - buffered initially");
+
+ </script>
+ </body>
+</html>
</ins></span></pre></div>
<a id="trunkSourceWebCoreChangeLog"></a>
<div class="modfile"><h4>Modified: trunk/Source/WebCore/ChangeLog (214805 => 214806)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/WebCore/ChangeLog 2017-04-03 16:54:49 UTC (rev 214805)
+++ trunk/Source/WebCore/ChangeLog 2017-04-03 16:59:41 UTC (rev 214806)
</span><span class="lines">@@ -1,3 +1,60 @@
</span><ins>+2017-04-03 Youenn Fablet <youenn@apple.com>
+
+ captureStream is getting black frames with webgl canvas
+ https://bugs.webkit.org/show_bug.cgi?id=170325
+
+ Reviewed by Dean Jackson.
+
+ Test: fast/mediastream/captureStream/canvas3d.html
+
+ Changing the webgl context to save buffers in case the canvas is captured.
+ Adding a canvas changed notification in case of clear.
+ In the future, we might want to change this notification and do it when endPaint or similar is called.
+
+ Adding an Internals API to grab the RGBA equivalent of the next track frame.
+ For that purpose, adding a bunch of WEBCORE_EXPORT.
+
+ * Modules/mediastream/CanvasCaptureMediaStreamTrack.cpp:
+ (WebCore::CanvasCaptureMediaStreamTrack::Source::Source): Adding constraints support so that track settings
+ getter actually transmits the width and height of the source.
+ (WebCore::CanvasCaptureMediaStreamTrack::Source::canvasChanged): ensuring webgl canvas context keep their drawing buffer.
+ * Modules/mediastream/MediaStreamTrack.h:
+ * bindings/js/JSDOMGuardedObject.h:
+ * bindings/js/JSDOMPromise.h:
+ (WebCore::DeferredPromise::resolve):
+ (WebCore::DeferredPromise::reject):
+ * dom/ActiveDOMCallback.h:
+ * html/HTMLCanvasElement.cpp:
+ (WebCore::HTMLCanvasElement::captureStream):
+ * html/ImageData.h:
+ * html/ImageData.idl:
+ * html/canvas/WebGLRenderingContext.cpp:
+ (WebCore::WebGLRenderingContext::clear): ensuring canvas observers get notified in case of clear calls.
+ * html/canvas/WebGLRenderingContextBase.h:
+ (WebCore::WebGLRenderingContextBase::preserveDrawingBuffer): Added to allow canvas capture to update this property.
+ * platform/MediaSample.h:
+ (WebCore::MediaSample::getRGBAImageData): Added for internals API.
+ * platform/graphics/avfoundation/MediaSampleAVFObjC.h:
+ * platform/graphics/avfoundation/objc/MediaSampleAVFObjC.mm:
+ (WebCore::MediaSampleAVFObjC::getRGBAImageData):
+ * platform/graphics/cv/PixelBufferConformerCV.cpp:
+ (WebCore::PixelBufferConformerCV::convert): Helper routine for getRGBAImageData.
+ * platform/graphics/cv/PixelBufferConformerCV.h:
+ * platform/mediastream/RealtimeMediaSourceSettings.h:
+ (WebCore::RealtimeMediaSourceSettings::setSupportedConstraints):
+ (WebCore::RealtimeMediaSourceSettings::setSupportedConstraits): Deleted.
+ * platform/mediastream/mac/AVMediaCaptureSource.mm:
+ (WebCore::AVMediaCaptureSource::initializeSettings):
+ * platform/mediastream/openwebrtc/RealtimeAudioSourceOwr.h:
+ * platform/mediastream/openwebrtc/RealtimeVideoSourceOwr.h:
+ * platform/mock/MockRealtimeMediaSource.cpp:
+ (WebCore::MockRealtimeMediaSource::initializeSettings):
+ * testing/Internals.cpp:
+ (WebCore::Internals::grabNextMediaStreamTrackFrame):
+ (WebCore::Internals::videoSampleAvailable):
+ * testing/Internals.h:
+ * testing/Internals.idl:
+
</ins><span class="cx"> 2017-04-03 Per Arne Vollan <pvollan@apple.com>
</span><span class="cx">
</span><span class="cx"> Implement stroke-miterlimit.
</span></span></pre></div>
<a id="trunkSourceWebCoreModulesmediastreamCanvasCaptureMediaStreamTrackcpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/WebCore/Modules/mediastream/CanvasCaptureMediaStreamTrack.cpp (214805 => 214806)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/WebCore/Modules/mediastream/CanvasCaptureMediaStreamTrack.cpp 2017-04-03 16:54:49 UTC (rev 214805)
+++ trunk/Source/WebCore/Modules/mediastream/CanvasCaptureMediaStreamTrack.cpp 2017-04-03 16:59:41 UTC (rev 214806)
</span><span class="lines">@@ -26,6 +26,7 @@
</span><span class="cx"> #include "CanvasCaptureMediaStreamTrack.h"
</span><span class="cx">
</span><span class="cx"> #include "GraphicsContext.h"
</span><ins>+#include "WebGLRenderingContextBase.h"
</ins><span class="cx">
</span><span class="cx"> #if ENABLE(MEDIA_STREAM)
</span><span class="cx">
</span><span class="lines">@@ -67,6 +68,10 @@
</span><span class="cx"> {
</span><span class="cx"> m_settings.setWidth(canvas.width());
</span><span class="cx"> m_settings.setHeight(canvas.height());
</span><ins>+ RealtimeMediaSourceSupportedConstraints constraints;
+ constraints.setSupportsWidth(true);
+ constraints.setSupportsHeight(true);
+ m_settings.setSupportedConstraints(constraints);
</ins><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> void CanvasCaptureMediaStreamTrack::Source::startProducingData()
</span><span class="lines">@@ -124,6 +129,17 @@
</span><span class="cx"> {
</span><span class="cx"> ASSERT_UNUSED(canvas, m_canvas == &canvas);
</span><span class="cx">
</span><ins>+ // FIXME: We need to preserve drawing buffer as we are currently grabbing frames asynchronously.
+ // We should instead add an anchor point for both 2d and 3d contexts where canvas will actually paint.
+ // And call canvas observers from that point.
+ if (canvas.renderingContext() && canvas.renderingContext()->isWebGL()) {
+ auto& context = static_cast<WebGLRenderingContextBase&>(*canvas.renderingContext());
+ if (!context.isPreservingDrawingBuffer()) {
+ canvas.document().addConsoleMessage(MessageSource::JS, MessageLevel::Warning, ASCIILiteral("Turning drawing buffer preservation for the WebGL canvas being captured"));
+ context.setPreserveDrawingBuffer(true);
+ }
+ }
+
</ins><span class="cx"> // FIXME: We should try to generate the frame at the time the screen is being updated.
</span><span class="cx"> if (m_canvasChangedTimer.isActive())
</span><span class="cx"> return;
</span></span></pre></div>
<a id="trunkSourceWebCoreModulesmediastreamMediaStreamTrackh"></a>
<div class="modfile"><h4>Modified: trunk/Source/WebCore/Modules/mediastream/MediaStreamTrack.h (214805 => 214806)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/WebCore/Modules/mediastream/MediaStreamTrack.h 2017-04-03 16:54:49 UTC (rev 214805)
+++ trunk/Source/WebCore/Modules/mediastream/MediaStreamTrack.h 2017-04-03 16:59:41 UTC (rev 214806)
</span><span class="lines">@@ -89,7 +89,7 @@
</span><span class="cx"> String deviceId;
</span><span class="cx"> String groupId;
</span><span class="cx"> };
</span><del>- TrackSettings getSettings() const;
</del><ins>+ WEBCORE_EXPORT TrackSettings getSettings() const;
</ins><span class="cx">
</span><span class="cx"> struct TrackCapabilities {
</span><span class="cx"> std::optional<LongRange> width;
</span></span></pre></div>
<a id="trunkSourceWebCorebindingsjsJSDOMGuardedObjecth"></a>
<div class="modfile"><h4>Modified: trunk/Source/WebCore/bindings/js/JSDOMGuardedObject.h (214805 => 214806)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/WebCore/bindings/js/JSDOMGuardedObject.h 2017-04-03 16:54:49 UTC (rev 214805)
+++ trunk/Source/WebCore/bindings/js/JSDOMGuardedObject.h 2017-04-03 16:59:41 UTC (rev 214806)
</span><span class="lines">@@ -33,7 +33,7 @@
</span><span class="cx">
</span><span class="cx"> namespace WebCore {
</span><span class="cx">
</span><del>-class DOMGuardedObject : public RefCounted<DOMGuardedObject>, public ActiveDOMCallback {
</del><ins>+class WEBCORE_EXPORT DOMGuardedObject : public RefCounted<DOMGuardedObject>, public ActiveDOMCallback {
</ins><span class="cx"> public:
</span><span class="cx"> ~DOMGuardedObject();
</span><span class="cx">
</span></span></pre></div>
<a id="trunkSourceWebCorebindingsjsJSDOMPromiseh"></a>
<div class="modfile"><h4>Modified: trunk/Source/WebCore/bindings/js/JSDOMPromise.h (214805 => 214806)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/WebCore/bindings/js/JSDOMPromise.h 2017-04-03 16:54:49 UTC (rev 214805)
+++ trunk/Source/WebCore/bindings/js/JSDOMPromise.h 2017-04-03 16:59:41 UTC (rev 214806)
</span><span class="lines">@@ -88,7 +88,7 @@
</span><span class="cx"> void reject();
</span><span class="cx"> void reject(std::nullptr_t);
</span><span class="cx"> void reject(Exception&&);
</span><del>- void reject(ExceptionCode, const String& = { });
</del><ins>+ WEBCORE_EXPORT void reject(ExceptionCode, const String& = { });
</ins><span class="cx"> void reject(const JSC::PrivateName&);
</span><span class="cx">
</span><span class="cx"> template<typename Callback>
</span><span class="lines">@@ -122,7 +122,7 @@
</span><span class="cx">
</span><span class="cx"> JSC::JSPromiseDeferred* deferred() const { return guarded(); }
</span><span class="cx">
</span><del>- void callFunction(JSC::ExecState&, JSC::JSValue function, JSC::JSValue resolution);
</del><ins>+ WEBCORE_EXPORT void callFunction(JSC::ExecState&, JSC::JSValue function, JSC::JSValue resolution);
</ins><span class="cx"> void resolve(JSC::ExecState& state, JSC::JSValue resolution) { callFunction(state, deferred()->resolve(), resolution); }
</span><span class="cx"> void reject(JSC::ExecState& state, JSC::JSValue resolution) { callFunction(state, deferred()->reject(), resolution); }
</span><span class="cx"> };
</span><span class="lines">@@ -212,7 +212,7 @@
</span><span class="cx"> void fulfillPromiseWithJSON(Ref<DeferredPromise>&&, const String&);
</span><span class="cx"> void fulfillPromiseWithArrayBuffer(Ref<DeferredPromise>&&, ArrayBuffer*);
</span><span class="cx"> void fulfillPromiseWithArrayBuffer(Ref<DeferredPromise>&&, const void*, size_t);
</span><del>-void rejectPromiseWithExceptionIfAny(JSC::ExecState&, JSDOMGlobalObject&, JSC::JSPromiseDeferred&);
</del><ins>+WEBCORE_EXPORT void rejectPromiseWithExceptionIfAny(JSC::ExecState&, JSDOMGlobalObject&, JSC::JSPromiseDeferred&);
</ins><span class="cx"> JSC::EncodedJSValue createRejectedPromiseWithTypeError(JSC::ExecState&, const String&);
</span><span class="cx">
</span><span class="cx"> using PromiseFunction = void(JSC::ExecState&, Ref<DeferredPromise>&&);
</span></span></pre></div>
<a id="trunkSourceWebCoredomActiveDOMCallbackh"></a>
<div class="modfile"><h4>Modified: trunk/Source/WebCore/dom/ActiveDOMCallback.h (214805 => 214806)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/WebCore/dom/ActiveDOMCallback.h 2017-04-03 16:54:49 UTC (rev 214805)
+++ trunk/Source/WebCore/dom/ActiveDOMCallback.h 2017-04-03 16:59:41 UTC (rev 214806)
</span><span class="lines">@@ -46,7 +46,7 @@
</span><span class="cx"> ActiveDOMCallback(ScriptExecutionContext* context);
</span><span class="cx"> virtual ~ActiveDOMCallback();
</span><span class="cx">
</span><del>- bool canInvokeCallback() const;
</del><ins>+ WEBCORE_EXPORT bool canInvokeCallback() const;
</ins><span class="cx"> };
</span><span class="cx">
</span><span class="cx"> } // namespace WebCore
</span></span></pre></div>
<a id="trunkSourceWebCorehtmlHTMLCanvasElementcpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/WebCore/html/HTMLCanvasElement.cpp (214805 => 214806)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/WebCore/html/HTMLCanvasElement.cpp 2017-04-03 16:54:49 UTC (rev 214805)
+++ trunk/Source/WebCore/html/HTMLCanvasElement.cpp 2017-04-03 16:59:41 UTC (rev 214806)
</span><span class="lines">@@ -603,7 +603,7 @@
</span><span class="cx">
</span><span class="cx"> if (frameRequestRate && frameRequestRate.value() < 0)
</span><span class="cx"> return Exception(NOT_SUPPORTED_ERR, ASCIILiteral("frameRequestRate is negative"));
</span><del>-
</del><ins>+
</ins><span class="cx"> auto track = CanvasCaptureMediaStreamTrack::create(context, *this, WTFMove(frameRequestRate));
</span><span class="cx"> auto stream = MediaStream::create(context);
</span><span class="cx"> stream->addTrack(track);
</span></span></pre></div>
<a id="trunkSourceWebCorehtmlImageDatah"></a>
<div class="modfile"><h4>Modified: trunk/Source/WebCore/html/ImageData.h (214805 => 214806)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/WebCore/html/ImageData.h 2017-04-03 16:54:49 UTC (rev 214805)
+++ trunk/Source/WebCore/html/ImageData.h 2017-04-03 16:59:41 UTC (rev 214806)
</span><span class="lines">@@ -34,7 +34,7 @@
</span><span class="cx">
</span><span class="cx"> namespace WebCore {
</span><span class="cx">
</span><del>-class ImageData : public RefCounted<ImageData> {
</del><ins>+class WEBCORE_EXPORT ImageData : public RefCounted<ImageData> {
</ins><span class="cx"> public:
</span><span class="cx"> static ExceptionOr<Ref<ImageData>> create(unsigned sw, unsigned sh);
</span><span class="cx"> static RefPtr<ImageData> create(const IntSize&);
</span></span></pre></div>
<a id="trunkSourceWebCorehtmlImageDataidl"></a>
<div class="modfile"><h4>Modified: trunk/Source/WebCore/html/ImageData.idl (214805 => 214806)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/WebCore/html/ImageData.idl 2017-04-03 16:54:49 UTC (rev 214805)
+++ trunk/Source/WebCore/html/ImageData.idl 2017-04-03 16:59:41 UTC (rev 214806)
</span><span class="lines">@@ -31,6 +31,7 @@
</span><span class="cx"> Constructor(unsigned long sw, unsigned long sh),
</span><span class="cx"> ConstructorMayThrowException,
</span><span class="cx"> CustomToJSObject,
</span><ins>+ ExportMacro=WEBCORE_EXPORT,
</ins><span class="cx"> Exposed=(Window,Worker),
</span><span class="cx"> ImplementationLacksVTable,
</span><span class="cx"> ] interface ImageData {
</span></span></pre></div>
<a id="trunkSourceWebCorehtmlcanvasWebGLRenderingContextcpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/WebCore/html/canvas/WebGLRenderingContext.cpp (214805 => 214806)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/WebCore/html/canvas/WebGLRenderingContext.cpp 2017-04-03 16:54:49 UTC (rev 214805)
+++ trunk/Source/WebCore/html/canvas/WebGLRenderingContext.cpp 2017-04-03 16:59:41 UTC (rev 214806)
</span><span class="lines">@@ -470,7 +470,7 @@
</span><span class="cx"> }
</span><span class="cx"> if (!clearIfComposited(mask))
</span><span class="cx"> m_context->clear(mask);
</span><del>- markContextChanged();
</del><ins>+ markContextChangedAndNotifyCanvasObserver();
</ins><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> WebGLAny WebGLRenderingContext::getParameter(GC3Denum pname)
</span></span></pre></div>
<a id="trunkSourceWebCorehtmlcanvasWebGLRenderingContextBaseh"></a>
<div class="modfile"><h4>Modified: trunk/Source/WebCore/html/canvas/WebGLRenderingContextBase.h (214805 => 214806)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/WebCore/html/canvas/WebGLRenderingContextBase.h 2017-04-03 16:54:49 UTC (rev 214805)
+++ trunk/Source/WebCore/html/canvas/WebGLRenderingContextBase.h 2017-04-03 16:59:41 UTC (rev 214806)
</span><span class="lines">@@ -207,6 +207,9 @@
</span><span class="cx"> WebGLAny getVertexAttrib(GC3Duint index, GC3Denum pname);
</span><span class="cx"> long long getVertexAttribOffset(GC3Duint index, GC3Denum pname);
</span><span class="cx">
</span><ins>+ bool isPreservingDrawingBuffer() const { return m_attributes.preserveDrawingBuffer; }
+ void setPreserveDrawingBuffer(bool value) { m_attributes.preserveDrawingBuffer = value; }
+
</ins><span class="cx"> virtual void hint(GC3Denum target, GC3Denum mode) = 0;
</span><span class="cx"> GC3Dboolean isBuffer(WebGLBuffer*);
</span><span class="cx"> bool isContextLost() const;
</span></span></pre></div>
<a id="trunkSourceWebCoreplatformMediaSampleh"></a>
<div class="modfile"><h4>Modified: trunk/Source/WebCore/platform/MediaSample.h (214805 => 214806)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/WebCore/platform/MediaSample.h 2017-04-03 16:54:49 UTC (rev 214805)
+++ trunk/Source/WebCore/platform/MediaSample.h 2017-04-03 16:59:41 UTC (rev 214806)
</span><span class="lines">@@ -27,6 +27,7 @@
</span><span class="cx"> #define MediaSample_h
</span><span class="cx">
</span><span class="cx"> #include "FloatSize.h"
</span><ins>+#include <runtime/TypedArrays.h>
</ins><span class="cx"> #include <wtf/MediaTime.h>
</span><span class="cx"> #include <wtf/RefCounted.h>
</span><span class="cx"> #include <wtf/text/AtomicString.h>
</span><span class="lines">@@ -69,6 +70,8 @@
</span><span class="cx"> virtual std::pair<RefPtr<MediaSample>, RefPtr<MediaSample>> divide(const MediaTime& presentationTime) = 0;
</span><span class="cx"> virtual Ref<MediaSample> createNonDisplayingCopy() const = 0;
</span><span class="cx">
</span><ins>+ virtual RefPtr<JSC::Uint8ClampedArray> getRGBAImageData() const { return nullptr; }
+
</ins><span class="cx"> enum SampleFlags {
</span><span class="cx"> None = 0,
</span><span class="cx"> IsSync = 1 << 0,
</span></span></pre></div>
<a id="trunkSourceWebCoreplatformgraphicsavfoundationMediaSampleAVFObjCh"></a>
<div class="modfile"><h4>Modified: trunk/Source/WebCore/platform/graphics/avfoundation/MediaSampleAVFObjC.h (214805 => 214806)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/WebCore/platform/graphics/avfoundation/MediaSampleAVFObjC.h 2017-04-03 16:54:49 UTC (rev 214805)
+++ trunk/Source/WebCore/platform/graphics/avfoundation/MediaSampleAVFObjC.h 2017-04-03 16:59:41 UTC (rev 214806)
</span><span class="lines">@@ -41,6 +41,8 @@
</span><span class="cx"> static RefPtr<MediaSampleAVFObjC> createImageSample(Ref<JSC::Uint8ClampedArray>&&, unsigned long width, unsigned long height);
</span><span class="cx"> static RefPtr<MediaSampleAVFObjC> createImageSample(Vector<uint8_t>&&, unsigned long width, unsigned long height);
</span><span class="cx">
</span><ins>+ RefPtr<JSC::Uint8ClampedArray> getRGBAImageData() const final;
+
</ins><span class="cx"> private:
</span><span class="cx"> MediaSampleAVFObjC(CMSampleBufferRef sample)
</span><span class="cx"> : m_sample(sample)
</span></span></pre></div>
<a id="trunkSourceWebCoreplatformgraphicsavfoundationobjcMediaSampleAVFObjCmm"></a>
<div class="modfile"><h4>Modified: trunk/Source/WebCore/platform/graphics/avfoundation/objc/MediaSampleAVFObjC.mm (214805 => 214806)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/WebCore/platform/graphics/avfoundation/objc/MediaSampleAVFObjC.mm 2017-04-03 16:54:49 UTC (rev 214805)
+++ trunk/Source/WebCore/platform/graphics/avfoundation/objc/MediaSampleAVFObjC.mm 2017-04-03 16:59:41 UTC (rev 214806)
</span><span class="lines">@@ -26,6 +26,8 @@
</span><span class="cx"> #import "config.h"
</span><span class="cx"> #import "MediaSampleAVFObjC.h"
</span><span class="cx">
</span><ins>+#import "PixelBufferConformerCV.h"
+#import <runtime/TypedArrayInlines.h>
</ins><span class="cx"> #import <wtf/PrintStream.h>
</span><span class="cx">
</span><span class="cx"> #import "CoreMediaSoftLink.h"
</span><span class="lines">@@ -265,4 +267,28 @@
</span><span class="cx"> return MediaSampleAVFObjC::create(adoptCF(newSampleBuffer).get(), m_id);
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+RefPtr<JSC::Uint8ClampedArray> MediaSampleAVFObjC::getRGBAImageData() const
+{
+ const OSType imageFormat = kCVPixelFormatType_32RGBA;
+ RetainPtr<CFNumberRef> imageFormatNumber = adoptCF(CFNumberCreate(nullptr, kCFNumberIntType, &imageFormat));
+
+ RetainPtr<CFMutableDictionaryRef> conformerOptions = adoptCF(CFDictionaryCreateMutable(0, 1, &kCFTypeDictionaryKeyCallBacks, &kCFTypeDictionaryValueCallBacks));
+ CFDictionarySetValue(conformerOptions.get(), kCVPixelBufferPixelFormatTypeKey, imageFormatNumber.get());
+ PixelBufferConformerCV pixelBufferConformer(conformerOptions.get());
+
+ auto pixelBuffer = static_cast<CVPixelBufferRef>(CMSampleBufferGetImageBuffer(m_sample.get()));
+ auto rgbaPixelBuffer = pixelBufferConformer.convert(pixelBuffer);
+ auto status = CVPixelBufferLockBaseAddress(rgbaPixelBuffer.get(), kCVPixelBufferLock_ReadOnly);
+ ASSERT(status == noErr);
+
+ void* data = CVPixelBufferGetBaseAddressOfPlane(rgbaPixelBuffer.get(), 0);
+ size_t byteLength = CVPixelBufferGetHeight(pixelBuffer) * CVPixelBufferGetWidth(pixelBuffer) * 4;
+ auto result = JSC::Uint8ClampedArray::create(JSC::ArrayBuffer::create(data, byteLength), 0, byteLength);
+
+ status = CVPixelBufferUnlockBaseAddress(rgbaPixelBuffer.get(), kCVPixelBufferLock_ReadOnly);
+ ASSERT(status == noErr);
+
+ return result;
</ins><span class="cx"> }
</span><ins>+
+}
</ins></span></pre></div>
<a id="trunkSourceWebCoreplatformgraphicscvPixelBufferConformerCVcpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/WebCore/platform/graphics/cv/PixelBufferConformerCV.cpp (214805 => 214806)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/WebCore/platform/graphics/cv/PixelBufferConformerCV.cpp 2017-04-03 16:54:49 UTC (rev 214805)
+++ trunk/Source/WebCore/platform/graphics/cv/PixelBufferConformerCV.cpp 2017-04-03 16:59:41 UTC (rev 214806)
</span><span class="lines">@@ -72,6 +72,24 @@
</span><span class="cx"> CFRelease(pixelBuffer);
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+RetainPtr<CVPixelBufferRef> PixelBufferConformerCV::convert(CVPixelBufferRef rawBuffer)
+{
+#if USE(VIDEOTOOLBOX)
+ RetainPtr<CVPixelBufferRef> buffer { rawBuffer };
+
+ if (!VTPixelBufferConformerIsConformantPixelBuffer(m_pixelConformer.get(), buffer.get())) {
+ CVPixelBufferRef outputBuffer = nullptr;
+ OSStatus status = VTPixelBufferConformerCopyConformedPixelBuffer(m_pixelConformer.get(), buffer.get(), false, &outputBuffer);
+ if (status != noErr || !outputBuffer)
+ return nullptr;
+ return adoptCF(outputBuffer);
+ }
+#else
+ UNUSED_PARAM(rawBuffer);
+#endif
+ return nullptr;
+}
+
</ins><span class="cx"> RetainPtr<CGImageRef> PixelBufferConformerCV::createImageFromPixelBuffer(CVPixelBufferRef rawBuffer)
</span><span class="cx"> {
</span><span class="cx"> RetainPtr<CVPixelBufferRef> buffer { rawBuffer };
</span></span></pre></div>
<a id="trunkSourceWebCoreplatformgraphicscvPixelBufferConformerCVh"></a>
<div class="modfile"><h4>Modified: trunk/Source/WebCore/platform/graphics/cv/PixelBufferConformerCV.h (214805 => 214806)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/WebCore/platform/graphics/cv/PixelBufferConformerCV.h 2017-04-03 16:54:49 UTC (rev 214805)
+++ trunk/Source/WebCore/platform/graphics/cv/PixelBufferConformerCV.h 2017-04-03 16:59:41 UTC (rev 214806)
</span><span class="lines">@@ -37,6 +37,7 @@
</span><span class="cx"> class PixelBufferConformerCV {
</span><span class="cx"> public:
</span><span class="cx"> PixelBufferConformerCV(CFDictionaryRef attributes);
</span><ins>+ RetainPtr<CVPixelBufferRef> convert(CVPixelBufferRef);
</ins><span class="cx"> RetainPtr<CGImageRef> createImageFromPixelBuffer(CVPixelBufferRef);
</span><span class="cx">
</span><span class="cx"> private:
</span></span></pre></div>
<a id="trunkSourceWebCoreplatformmediastreamRealtimeMediaSourceSettingsh"></a>
<div class="modfile"><h4>Modified: trunk/Source/WebCore/platform/mediastream/RealtimeMediaSourceSettings.h (214805 => 214806)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/WebCore/platform/mediastream/RealtimeMediaSourceSettings.h 2017-04-03 16:54:49 UTC (rev 214805)
+++ trunk/Source/WebCore/platform/mediastream/RealtimeMediaSourceSettings.h 2017-04-03 16:59:41 UTC (rev 214806)
</span><span class="lines">@@ -92,7 +92,7 @@
</span><span class="cx"> const AtomicString& groupId() const { return m_groupId; }
</span><span class="cx"> void setGroupId(const AtomicString& groupId) { m_groupId = groupId; }
</span><span class="cx">
</span><del>- void setSupportedConstraits(const RealtimeMediaSourceSupportedConstraints& supportedConstraints) { m_supportedConstraints = supportedConstraints; }
</del><ins>+ void setSupportedConstraints(const RealtimeMediaSourceSupportedConstraints& supportedConstraints) { m_supportedConstraints = supportedConstraints; }
</ins><span class="cx">
</span><span class="cx"> template<class Encoder> void encode(Encoder&) const;
</span><span class="cx"> template<class Decoder> static bool decode(Decoder&, RealtimeMediaSourceSettings&);
</span></span></pre></div>
<a id="trunkSourceWebCoreplatformmediastreammacAVMediaCaptureSourcemm"></a>
<div class="modfile"><h4>Modified: trunk/Source/WebCore/platform/mediastream/mac/AVMediaCaptureSource.mm (214805 => 214806)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/WebCore/platform/mediastream/mac/AVMediaCaptureSource.mm 2017-04-03 16:54:49 UTC (rev 214805)
+++ trunk/Source/WebCore/platform/mediastream/mac/AVMediaCaptureSource.mm 2017-04-03 16:59:41 UTC (rev 214806)
</span><span class="lines">@@ -204,7 +204,7 @@
</span><span class="cx"> void AVMediaCaptureSource::initializeSettings()
</span><span class="cx"> {
</span><span class="cx"> if (m_currentSettings.deviceId().isEmpty())
</span><del>- m_currentSettings.setSupportedConstraits(supportedConstraints());
</del><ins>+ m_currentSettings.setSupportedConstraints(supportedConstraints());
</ins><span class="cx">
</span><span class="cx"> m_currentSettings.setDeviceId(id());
</span><span class="cx"> updateSettings(m_currentSettings);
</span></span></pre></div>
<a id="trunkSourceWebCoreplatformmediastreamopenwebrtcRealtimeAudioSourceOwrh"></a>
<div class="modfile"><h4>Modified: trunk/Source/WebCore/platform/mediastream/openwebrtc/RealtimeAudioSourceOwr.h (214805 => 214806)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/WebCore/platform/mediastream/openwebrtc/RealtimeAudioSourceOwr.h 2017-04-03 16:54:49 UTC (rev 214805)
+++ trunk/Source/WebCore/platform/mediastream/openwebrtc/RealtimeAudioSourceOwr.h 2017-04-03 16:59:41 UTC (rev 214806)
</span><span class="lines">@@ -59,7 +59,7 @@
</span><span class="cx"> protected:
</span><span class="cx"> void initializeSettings() final {
</span><span class="cx"> if (m_currentSettings.deviceId().isEmpty())
</span><del>- m_currentSettings.setSupportedConstraits(supportedConstraints());
</del><ins>+ m_currentSettings.setSupportedConstraints(supportedConstraints());
</ins><span class="cx">
</span><span class="cx"> m_currentSettings.setDeviceId(id());
</span><span class="cx"> }
</span></span></pre></div>
<a id="trunkSourceWebCoreplatformmediastreamopenwebrtcRealtimeVideoSourceOwrh"></a>
<div class="modfile"><h4>Modified: trunk/Source/WebCore/platform/mediastream/openwebrtc/RealtimeVideoSourceOwr.h (214805 => 214806)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/WebCore/platform/mediastream/openwebrtc/RealtimeVideoSourceOwr.h 2017-04-03 16:54:49 UTC (rev 214805)
+++ trunk/Source/WebCore/platform/mediastream/openwebrtc/RealtimeVideoSourceOwr.h 2017-04-03 16:59:41 UTC (rev 214806)
</span><span class="lines">@@ -59,7 +59,7 @@
</span><span class="cx"> protected:
</span><span class="cx"> void initializeSettings() final {
</span><span class="cx"> if (m_currentSettings.deviceId().isEmpty())
</span><del>- m_currentSettings.setSupportedConstraits(supportedConstraints());
</del><ins>+ m_currentSettings.setSupportedConstraints(supportedConstraints());
</ins><span class="cx">
</span><span class="cx"> m_currentSettings.setDeviceId(id());
</span><span class="cx">
</span></span></pre></div>
<a id="trunkSourceWebCoreplatformmockMockRealtimeMediaSourcecpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/WebCore/platform/mock/MockRealtimeMediaSource.cpp (214805 => 214806)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/WebCore/platform/mock/MockRealtimeMediaSource.cpp 2017-04-03 16:54:49 UTC (rev 214805)
+++ trunk/Source/WebCore/platform/mock/MockRealtimeMediaSource.cpp 2017-04-03 16:59:41 UTC (rev 214806)
</span><span class="lines">@@ -98,7 +98,7 @@
</span><span class="cx"> void MockRealtimeMediaSource::initializeSettings()
</span><span class="cx"> {
</span><span class="cx"> if (m_currentSettings.deviceId().isEmpty()) {
</span><del>- m_currentSettings.setSupportedConstraits(supportedConstraints());
</del><ins>+ m_currentSettings.setSupportedConstraints(supportedConstraints());
</ins><span class="cx"> m_currentSettings.setDeviceId(id());
</span><span class="cx"> }
</span><span class="cx">
</span></span></pre></div>
<a id="trunkSourceWebCoretestingInternalscpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/WebCore/testing/Internals.cpp (214805 => 214806)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/WebCore/testing/Internals.cpp 2017-04-03 16:54:49 UTC (rev 214805)
+++ trunk/Source/WebCore/testing/Internals.cpp 2017-04-03 16:59:41 UTC (rev 214806)
</span><span class="lines">@@ -85,6 +85,7 @@
</span><span class="cx"> #include "InstrumentingAgents.h"
</span><span class="cx"> #include "IntRect.h"
</span><span class="cx"> #include "InternalSettings.h"
</span><ins>+#include "JSImageData.h"
</ins><span class="cx"> #include "Language.h"
</span><span class="cx"> #include "LibWebRTCProvider.h"
</span><span class="cx"> #include "MainFrame.h"
</span><span class="lines">@@ -3895,6 +3896,33 @@
</span><span class="cx"> m_track = &track;
</span><span class="cx"> m_track->source().addObserver(*this);
</span><span class="cx"> }
</span><ins>+
+void Internals::grabNextMediaStreamTrackFrame(TrackFramePromise&& promise)
+{
+ m_nextTrackFramePromise = WTFMove(promise);
+}
+
+void Internals::videoSampleAvailable(MediaSample& sample)
+{
+ m_trackVideoSampleCount++;
+ if (!m_nextTrackFramePromise)
+ return;
+
+ auto videoSettings = m_track->getSettings();
+ if (!videoSettings.width || !videoSettings.height)
+ return;
+
+ auto rgba = sample.getRGBAImageData();
+ if (!rgba)
+ return;
+ auto imageData = ImageData::create(rgba.releaseNonNull(), *videoSettings.width, *videoSettings.height);
+ if (!imageData.hasException())
+ m_nextTrackFramePromise->resolve(imageData.releaseReturnValue().releaseNonNull());
+ else
+ m_nextTrackFramePromise->reject(imageData.exception().code());
+ m_nextTrackFramePromise = std::nullopt;
+}
+
</ins><span class="cx"> #endif
</span><span class="cx">
</span><span class="cx"> } // namespace WebCore
</span></span></pre></div>
<a id="trunkSourceWebCoretestingInternalsh"></a>
<div class="modfile"><h4>Modified: trunk/Source/WebCore/testing/Internals.h (214805 => 214806)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/WebCore/testing/Internals.h 2017-04-03 16:54:49 UTC (rev 214805)
+++ trunk/Source/WebCore/testing/Internals.h 2017-04-03 16:59:41 UTC (rev 214806)
</span><span class="lines">@@ -29,9 +29,11 @@
</span><span class="cx"> #include "CSSComputedStyleDeclaration.h"
</span><span class="cx"> #include "ContextDestructionObserver.h"
</span><span class="cx"> #include "ExceptionOr.h"
</span><ins>+#include "JSDOMPromise.h"
</ins><span class="cx"> #include "PageConsoleClient.h"
</span><span class="cx"> #include "RealtimeMediaSource.h"
</span><span class="cx"> #include <runtime/Float32Array.h>
</span><ins>+#include <wtf/Optional.h>
</ins><span class="cx">
</span><span class="cx"> #if ENABLE(MEDIA_SESSION)
</span><span class="cx"> #include "MediaSessionInterruptionProvider.h"
</span><span class="lines">@@ -54,6 +56,7 @@
</span><span class="cx"> class HTMLLinkElement;
</span><span class="cx"> class HTMLMediaElement;
</span><span class="cx"> class HTMLSelectElement;
</span><ins>+class ImageData;
</ins><span class="cx"> class InspectorStubFrontend;
</span><span class="cx"> class InternalSettings;
</span><span class="cx"> class MallocStatistics;
</span><span class="lines">@@ -562,6 +565,8 @@
</span><span class="cx"> unsigned long trackAudioSampleCount() const { return m_trackAudioSampleCount; }
</span><span class="cx"> unsigned long trackVideoSampleCount() const { return m_trackVideoSampleCount; }
</span><span class="cx"> void observeMediaStreamTrack(MediaStreamTrack&);
</span><ins>+ using TrackFramePromise = DOMPromise<IDLInterface<ImageData>>;
+ void grabNextMediaStreamTrackFrame(TrackFramePromise&&);
</ins><span class="cx"> #endif
</span><span class="cx">
</span><span class="cx"> private:
</span><span class="lines">@@ -573,12 +578,13 @@
</span><span class="cx">
</span><span class="cx"> // RealtimeMediaSource::Observer API
</span><span class="cx"> #if ENABLE(MEDIA_STREAM)
</span><del>- void videoSampleAvailable(MediaSample&) final { m_trackVideoSampleCount++; }
</del><ins>+ void videoSampleAvailable(MediaSample&) final;
</ins><span class="cx"> void audioSamplesAvailable(const MediaTime&, const PlatformAudioData&, const AudioStreamDescription&, size_t) final { m_trackAudioSampleCount++; }
</span><span class="cx">
</span><span class="cx"> unsigned long m_trackVideoSampleCount { 0 };
</span><span class="cx"> unsigned long m_trackAudioSampleCount { 0 };
</span><span class="cx"> RefPtr<MediaStreamTrack> m_track;
</span><ins>+ std::optional<TrackFramePromise> m_nextTrackFramePromise;
</ins><span class="cx"> #endif
</span><span class="cx">
</span><span class="cx"> std::unique_ptr<InspectorStubFrontend> m_inspectorFrontend;
</span></span></pre></div>
<a id="trunkSourceWebCoretestingInternalsidl"></a>
<div class="modfile"><h4>Modified: trunk/Source/WebCore/testing/Internals.idl (214805 => 214806)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/WebCore/testing/Internals.idl 2017-04-03 16:54:49 UTC (rev 214805)
+++ trunk/Source/WebCore/testing/Internals.idl 2017-04-03 16:59:41 UTC (rev 214806)
</span><span class="lines">@@ -526,6 +526,7 @@
</span><span class="cx"> [Conditional=WEBGL] void simulateWebGLContextChanged(WebGLRenderingContextBase context);
</span><span class="cx">
</span><span class="cx"> [Conditional=MEDIA_STREAM] void observeMediaStreamTrack(MediaStreamTrack track);
</span><ins>+ [Conditional=MEDIA_STREAM] Promise<ImageData> grabNextMediaStreamTrackFrame();
</ins><span class="cx"> [Conditional=MEDIA_STREAM] readonly attribute unsigned long trackAudioSampleCount;
</span><span class="cx"> [Conditional=MEDIA_STREAM] readonly attribute unsigned long trackVideoSampleCount;
</span><span class="cx"> };
</span></span></pre>
</div>
</div>
</body>
</html>