<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
"http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head><meta http-equiv="content-type" content="text/html; charset=utf-8" />
<title>[182555] trunk/Source/WebCore</title>
</head>
<body>
<style type="text/css"><!--
#msg dl.meta { border: 1px #006 solid; background: #369; padding: 6px; color: #fff; }
#msg dl.meta dt { float: left; width: 6em; font-weight: bold; }
#msg dt:after { content:':';}
#msg dl, #msg dt, #msg ul, #msg li, #header, #footer, #logmsg { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt; }
#msg dl a { font-weight: bold}
#msg dl a:link { color:#fc3; }
#msg dl a:active { color:#ff0; }
#msg dl a:visited { color:#cc6; }
h3 { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt; font-weight: bold; }
#msg pre { overflow: auto; background: #ffc; border: 1px #fa0 solid; padding: 6px; }
#logmsg { background: #ffc; border: 1px #fa0 solid; padding: 1em 1em 0 1em; }
#logmsg p, #logmsg pre, #logmsg blockquote { margin: 0 0 1em 0; }
#logmsg p, #logmsg li, #logmsg dt, #logmsg dd { line-height: 14pt; }
#logmsg h1, #logmsg h2, #logmsg h3, #logmsg h4, #logmsg h5, #logmsg h6 { margin: .5em 0; }
#logmsg h1:first-child, #logmsg h2:first-child, #logmsg h3:first-child, #logmsg h4:first-child, #logmsg h5:first-child, #logmsg h6:first-child { margin-top: 0; }
#logmsg ul, #logmsg ol { padding: 0; list-style-position: inside; margin: 0 0 0 1em; }
#logmsg ul { text-indent: -1em; padding-left: 1em; }#logmsg ol { text-indent: -1.5em; padding-left: 1.5em; }
#logmsg > ul, #logmsg > ol { margin: 0 0 1em 0; }
#logmsg pre { background: #eee; padding: 1em; }
#logmsg blockquote { border: 1px solid #fa0; border-left-width: 10px; padding: 1em 1em 0 1em; background: white;}
#logmsg dl { margin: 0; }
#logmsg dt { font-weight: bold; }
#logmsg dd { margin: 0; padding: 0 0 0.5em 0; }
#logmsg dd:before { content:'\00bb';}
#logmsg table { border-spacing: 0px; border-collapse: collapse; border-top: 4px solid #fa0; border-bottom: 1px solid #fa0; background: #fff; }
#logmsg table th { text-align: left; font-weight: normal; padding: 0.2em 0.5em; border-top: 1px dotted #fa0; }
#logmsg table td { text-align: right; border-top: 1px dotted #fa0; padding: 0.2em 0.5em; }
#logmsg table thead th { text-align: center; border-bottom: 1px solid #fa0; }
#logmsg table th.Corner { text-align: left; }
#logmsg hr { border: none 0; border-top: 2px dashed #fa0; height: 1px; }
#header, #footer { color: #fff; background: #636; border: 1px #300 solid; padding: 6px; }
#patch { width: 100%; }
#patch h4 {font-family: verdana,arial,helvetica,sans-serif;font-size:10pt;padding:8px;background:#369;color:#fff;margin:0;}
#patch .propset h4, #patch .binary h4 {margin:0;}
#patch pre {padding:0;line-height:1.2em;margin:0;}
#patch .diff {width:100%;background:#eee;padding: 0 0 10px 0;overflow:auto;}
#patch .propset .diff, #patch .binary .diff {padding:10px 0;}
#patch span {display:block;padding:0 10px;}
#patch .modfile, #patch .addfile, #patch .delfile, #patch .propset, #patch .binary, #patch .copfile {border:1px solid #ccc;margin:10px 0;}
#patch ins {background:#dfd;text-decoration:none;display:block;padding:0 10px;}
#patch del {background:#fdd;text-decoration:none;display:block;padding:0 10px;}
#patch .lines, .info {color:#888;background:#fff;}
--></style>
<div id="msg">
<dl class="meta">
<dt>Revision</dt> <dd><a href="http://trac.webkit.org/projects/webkit/changeset/182555">182555</a></dd>
<dt>Author</dt> <dd>jer.noble@apple.com</dd>
<dt>Date</dt> <dd>2015-04-08 11:12:56 -0700 (Wed, 08 Apr 2015)</dd>
</dl>
<h3>Log Message</h3>
<pre>[Mac][WebAudio] Update the AVAudioMix in the AudioSourceProviderAVFObjC when the list of enabled audio tracks change.
https://bugs.webkit.org/show_bug.cgi?id=143332
Reviewed by Eric Carlson.
Some media assets (notably, mp3s) will not have an enabled audio track when the AVAsset is
first loaded, so the AVAudioMix will have no trackID in it's parameters. Whenever the list
of enabled tracks change, recreate the AVAudioMix with the new first enabled audio trackID.
To facilitate this, add a new setter to AudioSourceProviderAVFObjC taking an AVAssetTrack to
use with the AVAudioMix. Whenever this parameter changes, the AVAudioMix is destroyed and
recreated.
* html/HTMLMediaElement.cpp:
(WebCore::HTMLMediaElement::mediaPlayerEngineUpdated): Drive-by fix: when the media
player switches engines, re-associate the audio source node with its provider.
* platform/graphics/avfoundation/AudioSourceProviderAVFObjC.h:
* platform/graphics/avfoundation/AudioSourceProviderAVFObjC.mm:
(WebCore::AudioSourceProviderAVFObjC::setPlayerItem): Only create the mix if there is a valid AVPlayerItem and AVAssetTrack
(WebCore::AudioSourceProviderAVFObjC::setAudioTrack): Ditto.
(WebCore::AudioSourceProviderAVFObjC::createMix): Don't iterate over the AVPlayerItem's tracks,
just use the one passed in through setAudioTrack().
* platform/graphics/avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.mm:
(WebCore::MediaPlayerPrivateAVFoundationObjC::cancelLoad): Clear the provider's track.
(WebCore::MediaPlayerPrivateAVFoundationObjC::createAVPlayerItem): Set the provider's track.
(WebCore::MediaPlayerPrivateAVFoundationObjC::tracksChanged): Ditto.
(WebCore::MediaPlayerPrivateAVFoundationObjC::audioSourceProvider): Ditto.</pre>
<h3>Modified Paths</h3>
<ul>
<li><a href="#trunkSourceWebCoreChangeLog">trunk/Source/WebCore/ChangeLog</a></li>
<li><a href="#trunkSourceWebCorehtmlHTMLMediaElementcpp">trunk/Source/WebCore/html/HTMLMediaElement.cpp</a></li>
<li><a href="#trunkSourceWebCoreplatformgraphicsavfoundationAudioSourceProviderAVFObjCh">trunk/Source/WebCore/platform/graphics/avfoundation/AudioSourceProviderAVFObjC.h</a></li>
<li><a href="#trunkSourceWebCoreplatformgraphicsavfoundationAudioSourceProviderAVFObjCmm">trunk/Source/WebCore/platform/graphics/avfoundation/AudioSourceProviderAVFObjC.mm</a></li>
<li><a href="#trunkSourceWebCoreplatformgraphicsavfoundationobjcMediaPlayerPrivateAVFoundationObjCmm">trunk/Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.mm</a></li>
</ul>
</div>
<div id="patch">
<h3>Diff</h3>
<a id="trunkSourceWebCoreChangeLog"></a>
<div class="modfile"><h4>Modified: trunk/Source/WebCore/ChangeLog (182554 => 182555)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/WebCore/ChangeLog 2015-04-08 18:06:58 UTC (rev 182554)
+++ trunk/Source/WebCore/ChangeLog 2015-04-08 18:12:56 UTC (rev 182555)
</span><span class="lines">@@ -1,3 +1,33 @@
</span><ins>+2015-04-08 Jer Noble <jer.noble@apple.com>
+
+ [Mac][WebAudio] Update the AVAudioMix in the AudioSourceProviderAVFObjC when the list of enabled audio tracks change.
+ https://bugs.webkit.org/show_bug.cgi?id=143332
+
+ Reviewed by Eric Carlson.
+
+ Some media assets (notably, mp3s) will not have an enabled audio track when the AVAsset is
+ first loaded, so the AVAudioMix will have no trackID in it's parameters. Whenever the list
+ of enabled tracks change, recreate the AVAudioMix with the new first enabled audio trackID.
+
+ To facilitate this, add a new setter to AudioSourceProviderAVFObjC taking an AVAssetTrack to
+ use with the AVAudioMix. Whenever this parameter changes, the AVAudioMix is destroyed and
+ recreated.
+
+ * html/HTMLMediaElement.cpp:
+ (WebCore::HTMLMediaElement::mediaPlayerEngineUpdated): Drive-by fix: when the media
+ player switches engines, re-associate the audio source node with its provider.
+ * platform/graphics/avfoundation/AudioSourceProviderAVFObjC.h:
+ * platform/graphics/avfoundation/AudioSourceProviderAVFObjC.mm:
+ (WebCore::AudioSourceProviderAVFObjC::setPlayerItem): Only create the mix if there is a valid AVPlayerItem and AVAssetTrack
+ (WebCore::AudioSourceProviderAVFObjC::setAudioTrack): Ditto.
+ (WebCore::AudioSourceProviderAVFObjC::createMix): Don't iterate over the AVPlayerItem's tracks,
+ just use the one passed in through setAudioTrack().
+ * platform/graphics/avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.mm:
+ (WebCore::MediaPlayerPrivateAVFoundationObjC::cancelLoad): Clear the provider's track.
+ (WebCore::MediaPlayerPrivateAVFoundationObjC::createAVPlayerItem): Set the provider's track.
+ (WebCore::MediaPlayerPrivateAVFoundationObjC::tracksChanged): Ditto.
+ (WebCore::MediaPlayerPrivateAVFoundationObjC::audioSourceProvider): Ditto.
+
</ins><span class="cx"> 2015-04-08 Anders Carlsson <andersca@apple.com>
</span><span class="cx">
</span><span class="cx"> Move some ApplicationCache static member functions to ApplicationCacheStorage
</span></span></pre></div>
<a id="trunkSourceWebCorehtmlHTMLMediaElementcpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/WebCore/html/HTMLMediaElement.cpp (182554 => 182555)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/WebCore/html/HTMLMediaElement.cpp 2015-04-08 18:06:58 UTC (rev 182554)
+++ trunk/Source/WebCore/html/HTMLMediaElement.cpp 2015-04-08 18:12:56 UTC (rev 182555)
</span><span class="lines">@@ -4273,6 +4273,14 @@
</span><span class="cx">
</span><span class="cx"> m_mediaSession->applyMediaPlayerRestrictions(*this);
</span><span class="cx">
</span><ins>+#if ENABLE(WEB_AUDIO)
+ if (m_audioSourceNode && audioSourceProvider()) {
+ m_audioSourceNode->lock();
+ audioSourceProvider()->setClient(m_audioSourceNode);
+ m_audioSourceNode->unlock();
+ }
+#endif
+
</ins><span class="cx"> #if PLATFORM(IOS)
</span><span class="cx"> if (!m_player)
</span><span class="cx"> return;
</span></span></pre></div>
<a id="trunkSourceWebCoreplatformgraphicsavfoundationAudioSourceProviderAVFObjCh"></a>
<div class="modfile"><h4>Modified: trunk/Source/WebCore/platform/graphics/avfoundation/AudioSourceProviderAVFObjC.h (182554 => 182555)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/WebCore/platform/graphics/avfoundation/AudioSourceProviderAVFObjC.h 2015-04-08 18:06:58 UTC (rev 182554)
+++ trunk/Source/WebCore/platform/graphics/avfoundation/AudioSourceProviderAVFObjC.h 2015-04-08 18:12:56 UTC (rev 182555)
</span><span class="lines">@@ -34,6 +34,7 @@
</span><span class="cx"> #include <wtf/RefPtr.h>
</span><span class="cx"> #include <wtf/RetainPtr.h>
</span><span class="cx">
</span><ins>+OBJC_CLASS AVAssetTrack;
</ins><span class="cx"> OBJC_CLASS AVPlayerItem;
</span><span class="cx"> OBJC_CLASS AVMutableAudioMix;
</span><span class="cx">
</span><span class="lines">@@ -54,6 +55,7 @@
</span><span class="cx"> virtual ~AudioSourceProviderAVFObjC();
</span><span class="cx">
</span><span class="cx"> void setPlayerItem(AVPlayerItem *);
</span><ins>+ void setAudioTrack(AVAssetTrack *);
</ins><span class="cx">
</span><span class="cx"> private:
</span><span class="cx"> AudioSourceProviderAVFObjC(AVPlayerItem *);
</span><span class="lines">@@ -78,6 +80,7 @@
</span><span class="cx"> void process(CMItemCount numberFrames, MTAudioProcessingTapFlags flagsIn, AudioBufferList *bufferListInOut, CMItemCount *numberFramesOut, MTAudioProcessingTapFlags *flagsOut);
</span><span class="cx">
</span><span class="cx"> RetainPtr<AVPlayerItem> m_avPlayerItem;
</span><ins>+ RetainPtr<AVAssetTrack> m_avAssetTrack;
</ins><span class="cx"> RetainPtr<AVMutableAudioMix> m_avAudioMix;
</span><span class="cx"> RetainPtr<MTAudioProcessingTapRef> m_tap;
</span><span class="cx"> RetainPtr<AudioConverterRef> m_converter;
</span></span></pre></div>
<a id="trunkSourceWebCoreplatformgraphicsavfoundationAudioSourceProviderAVFObjCmm"></a>
<div class="modfile"><h4>Modified: trunk/Source/WebCore/platform/graphics/avfoundation/AudioSourceProviderAVFObjC.mm (182554 => 182555)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/WebCore/platform/graphics/avfoundation/AudioSourceProviderAVFObjC.mm 2015-04-08 18:06:58 UTC (rev 182554)
+++ trunk/Source/WebCore/platform/graphics/avfoundation/AudioSourceProviderAVFObjC.mm 2015-04-08 18:12:56 UTC (rev 182555)
</span><span class="lines">@@ -38,7 +38,6 @@
</span><span class="cx"> #import <AVFoundation/AVAudioMix.h>
</span><span class="cx"> #import <AVFoundation/AVMediaFormat.h>
</span><span class="cx"> #import <AVFoundation/AVPlayerItem.h>
</span><del>-#import <AVFoundation/AVPlayerItemTrack.h>
</del><span class="cx"> #import <objc/runtime.h>
</span><span class="cx"> #import <wtf/MainThread.h>
</span><span class="cx">
</span><span class="lines">@@ -151,10 +150,24 @@
</span><span class="cx">
</span><span class="cx"> m_avPlayerItem = avPlayerItem;
</span><span class="cx">
</span><del>- if (m_client && m_avPlayerItem)
</del><ins>+ if (m_client && m_avPlayerItem && m_avAssetTrack)
</ins><span class="cx"> createMix();
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+void AudioSourceProviderAVFObjC::setAudioTrack(AVAssetTrack *avAssetTrack)
+{
+ if (m_avAssetTrack == avAssetTrack)
+ return;
+
+ if (m_avAudioMix)
+ destroyMix();
+
+ m_avAssetTrack = avAssetTrack;
+
+ if (m_client && m_avPlayerItem && m_avAssetTrack)
+ createMix();
+}
+
</ins><span class="cx"> void AudioSourceProviderAVFObjC::destroyMix()
</span><span class="cx"> {
</span><span class="cx"> if (m_avPlayerItem)
</span><span class="lines">@@ -190,15 +203,8 @@
</span><span class="cx"> RetainPtr<AVMutableAudioMixInputParameters> parameters = adoptNS([allocAVMutableAudioMixInputParametersInstance() init]);
</span><span class="cx"> [parameters setAudioTapProcessor:m_tap.get()];
</span><span class="cx">
</span><del>- CMPersistentTrackID firstEnabledAudioTrackID = kCMPersistentTrackID_Invalid;
- NSArray* tracks = [m_avPlayerItem tracks];
- for (AVPlayerItemTrack* track in tracks) {
- if ([track.assetTrack hasMediaCharacteristic:AVMediaCharacteristicAudible] && track.enabled) {
- firstEnabledAudioTrackID = track.assetTrack.trackID;
- break;
- }
- }
- [parameters setTrackID:firstEnabledAudioTrackID];
</del><ins>+ CMPersistentTrackID trackID = m_avAssetTrack.get().trackID;
+ [parameters setTrackID:trackID];
</ins><span class="cx">
</span><span class="cx"> [m_avAudioMix setInputParameters:@[parameters.get()]];
</span><span class="cx"> [m_avPlayerItem setAudioMix:m_avAudioMix.get()];
</span></span></pre></div>
<a id="trunkSourceWebCoreplatformgraphicsavfoundationobjcMediaPlayerPrivateAVFoundationObjCmm"></a>
<div class="modfile"><h4>Modified: trunk/Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.mm (182554 => 182555)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.mm 2015-04-08 18:06:58 UTC (rev 182554)
+++ trunk/Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.mm 2015-04-08 18:12:56 UTC (rev 182555)
</span><span class="lines">@@ -349,6 +349,7 @@
</span><span class="cx"> static NSArray *itemKVOProperties();
</span><span class="cx"> static NSArray *assetTrackMetadataKeyNames();
</span><span class="cx"> static NSArray *playerKVOProperties();
</span><ins>+static AVAssetTrack* firstEnabledTrack(NSArray* tracks);
</ins><span class="cx">
</span><span class="cx"> #if !LOG_DISABLED
</span><span class="cx"> static const char *boolString(bool val)
</span><span class="lines">@@ -566,8 +567,10 @@
</span><span class="cx"> m_cachedTracks = nullptr;
</span><span class="cx">
</span><span class="cx"> #if ENABLE(WEB_AUDIO) && USE(MEDIATOOLBOX)
</span><del>- if (m_provider)
</del><ins>+ if (m_provider) {
</ins><span class="cx"> m_provider->setPlayerItem(nullptr);
</span><ins>+ m_provider->setAudioTrack(nullptr);
+ }
</ins><span class="cx"> #endif
</span><span class="cx">
</span><span class="cx"> setIgnoreLoadStateChanges(false);
</span><span class="lines">@@ -998,8 +1001,10 @@
</span><span class="cx"> #endif
</span><span class="cx">
</span><span class="cx"> #if ENABLE(WEB_AUDIO) && USE(MEDIATOOLBOX)
</span><del>- if (m_provider)
</del><ins>+ if (m_provider) {
</ins><span class="cx"> m_provider->setPlayerItem(m_avPlayerItem.get());
</span><ins>+ m_provider->setAudioTrack(firstEnabledTrack([m_avAsset tracksWithMediaCharacteristic:AVMediaCharacteristicAudible]));
+ }
</ins><span class="cx"> #endif
</span><span class="cx">
</span><span class="cx"> setDelayCallbacks(false);
</span><span class="lines">@@ -1917,6 +1922,11 @@
</span><span class="cx"> if (primaryAudioTrackLanguage != languageOfPrimaryAudioTrack())
</span><span class="cx"> characteristicsChanged();
</span><span class="cx">
</span><ins>+#if ENABLE(WEB_AUDIO) && USE(MEDIATOOLBOX)
+ if (m_provider)
+ m_provider->setAudioTrack(firstEnabledTrack([m_avAsset tracksWithMediaCharacteristic:AVMediaCharacteristicAudible]));
+#endif
+
</ins><span class="cx"> setDelayCharacteristicsChangedNotification(false);
</span><span class="cx"> }
</span><span class="cx">
</span><span class="lines">@@ -2127,8 +2137,11 @@
</span><span class="cx"> #if ENABLE(WEB_AUDIO) && USE(MEDIATOOLBOX)
</span><span class="cx"> AudioSourceProvider* MediaPlayerPrivateAVFoundationObjC::audioSourceProvider()
</span><span class="cx"> {
</span><del>- if (!m_provider)
</del><ins>+ if (!m_provider) {
</ins><span class="cx"> m_provider = AudioSourceProviderAVFObjC::create(m_avPlayerItem.get());
</span><ins>+ m_provider->setAudioTrack(firstEnabledTrack([m_avAsset tracksWithMediaCharacteristic:AVMediaCharacteristicAudible]));
+ }
+
</ins><span class="cx"> return m_provider.get();
</span><span class="cx"> }
</span><span class="cx"> #endif
</span></span></pre>
</div>
</div>
</body>
</html>