As the audio clock would only move forward when it consumes real sample,
if we don't keep sending data when there is a gap between audio and
video, then the audio clock would stall.
Therefore, we need to create fake audio data to cover the gap between
audio and video if the audio track is shorter. But we don't need to
worry about the opposite situation, where the video track is shorter
than the audio track. Because the video sink would simply display the
last video frame to cover the gap.
Differential Revision: https://phabricator.services.mozilla.com/D184272
When the audio sink shutdown, we should also reset the audible state for
the listeners. Otherwise, MediaDecoderStateMachine would possibly rely
on the incorrect audible state that is produced by previous sink, which
would cause showing the sound indicator even if no sound is being playing.
Eg. Considering following case without the reset of the audible state
1. Sink A is audible -> MDSM set mIsAudioDataAudible=true
2. Stop sink
3. Recreate sink B and it's playing silence -> MDSM's mIsAudioDataAudible=true is still true, this is wrong!
Differential Revision: https://phabricator.services.mozilla.com/D113550
This patch is used to create a consistent clock timeline among
different media sinks. Having a consistent clock timeline can help the
newly created media sink to understand audio data's "adjusted" time,
which can prevent the sink from inserting silence if the sink
incorrectly think there is a gap between data and sink start time.
However, once seeking completed, we should use the media time that
indicates to the new position, instead of using the previous clock time.
Eg. Audio duration = 10, and has seamlessly looped 5 times. Then the
start time of the sample in 2s in the 6th looping would be 2 + 5*10 = 52.
If we switch to the new sink, and use media time (which is 2s) as a
start time to start a new sink, then the new sink would think there is a gap between 2s and 52s, then push a lot of silence. [1]
[1] https://searchfox.org/mozilla-central/rev/9b430bb1a11d7152cab2af4574f451ffb906b052/dom/media/mediasink/AudioSink.cpp#390-426
Differential Revision: https://phabricator.services.mozilla.com/D113548
The original change was introduced when implementing seamless audio
looping, and it's no longer needed. Because now we would properly
adjust the offset of the media queue when entering the looping decoding
state to make the media time and the clock time consistent.
Differential Revision: https://phabricator.services.mozilla.com/D182526
If we seek directly to EOS, it's possible that we still can't know the
track duration because we don't have the last decoded data. Therefore,
we remove the assertion and make it as condition.
Differential Revision: https://phabricator.services.mozilla.com/D182417
This allows video playback to continue uninterrupted.
The AudioSinkWrapper now maintains the correct playback position in the media
data. Previously the replacement AudioSinkWrapper started its clock at the
"current playback position" from the MediaDecoderStateMachine, which did not
include additional time from looping, so silence would be played until the
missing time had passed.
The AudioSinkWrapper now continues to periodically attempt to open a new
output device if not initially available. Previously such attempts were only
performed on state changes such as seeking or unmuting.
The promise returned from MediaDecoderStateMachine::InvokeSetSink() now
resolves regardless of whether the new device can be opened successfully.
Previously the promise could be rejected if the device was necessary for audio
output at the time of the call but no check for device availability (beyond
that of device enumeration) was performed if playback was suspended or muted
or if a subsequent device change had already been queued. Even previously the
underlying device was changed even when the promise was rejected and could be
used if a state change occurred.
Differential Revision: https://phabricator.services.mozilla.com/D181975
This allows video playback to continue uninterrupted.
The AudioSinkWrapper now maintains the correct playback position in the media
data. Previously the replacement AudioSinkWrapper started its clock at the
"current playback position" from the MediaDecoderStateMachine, which did not
include additional time from looping, so silence would be played until the
missing time had passed.
The AudioSinkWrapper now continues to periodically attempt to open a new
output device if not initially available. Previously such attempts were only
performed on state changes such as seeking or unmuting.
The promise returned from MediaDecoderStateMachine::InvokeSetSink() now
resolves regardless of whether the new device can be opened successfully.
Previously the promise could be rejected if the device was necessary for audio
output at the time of the call but no check for device availability (beyond
that of device enumeration) was performed if playback was suspended or muted
or if a subsequent device change had already been queued. Even previously the
underlying device was changed even when the promise was rejected and could be
used if a state change occurred.
Differential Revision: https://phabricator.services.mozilla.com/D181975
This shortens the time it takes to establish the connections for canonicals
living on the main thread, from the thread hop chain:
main thread (MediaDecoderStateMachine::Init)
-> state machine task queue (Mirror::Connect)
-> main thread (AbstractCanonical::AddMirror)
-> state machine task queue (AbstractMirror::UpdateValue)
to the thread hop chain:
main thread (MediaDecoderStateMachine::Init + Canonical::ConnectMirror)
-> state machine task queue (Mirror::AbstractUpdateValue)
No races exist here as MediaDecoderStateMachine initiates its shutdown on main
thread and continues on the task queue (where the mirrors live).
Differential Revision: https://phabricator.services.mozilla.com/D181086
This shortens the time it takes to establish the connections for canonicals
living on the main thread, from the thread hop chain:
main thread (MediaDecoderStateMachine::Init)
-> state machine task queue (Mirror::Connect)
-> main thread (AbstractCanonical::AddMirror)
-> state machine task queue (AbstractMirror::UpdateValue)
to the thread hop chain:
main thread (MediaDecoderStateMachine::Init + Canonical::ConnectMirror)
-> state machine task queue (Mirror::AbstractUpdateValue)
No races exist here as MediaDecoderStateMachine initiates its shutdown on main
thread and continues on the task queue (where the mirrors live).
Differential Revision: https://phabricator.services.mozilla.com/D181086
If we are not going to loop, then there is no need to keep waiting the
data from the start position if we have had all data for the track.
Differential Revision: https://phabricator.services.mozilla.com/D177440
If one of the track reaches EOS before starting the playback, playback
would never start due to the those condition checks. That causes
`test_NoAudioLoopBackData.html` failed on the try server as well.
But those checks should only be used for the situation where the media
queue has been closed before entering the looping state. Therefore,
update them in order to fix the failure.
Differential Revision: https://phabricator.services.mozilla.com/D176706
In MSE seamless looping, it's possible that the data from the start
position isn't available because website won't append them until the
media time has seeked back to the start position.
In this situation, if the audio track is shorter, when all queued audio
get consumed already, the audio underrun would happen. As we still need
the audio clock to be driven in order to play remaining video frames, we
need to treat audio underrun as slient frames. Otherwise, audio clock
would stall, resulting in whole playback stall as well.
Differential Revision: https://phabricator.services.mozilla.com/D176591
During MSE seamless looping, if we can't get the data from the start
position, those data might not be appended yet. Entering buffering
state won't help on this situation, because accessing data from the
start position is our internal implementation.
Therefore, we should fallback to the normal looping state, which would
trigger seeking to the start position. That is visible to the website so
that they can start appending data again.
Differential Revision: https://phabricator.services.mozilla.com/D176367
This patch fixes the incorrect condition for the single track playback,
because the EOS flag for the nonexistent track would always be true.
Eg. for audio-only playback, `mIsReachingVideoEOS` is always true.
Differential Revision: https://phabricator.services.mozilla.com/D162025
