Since I want to use more agent's functions in MediaElement and I don't think these functions need to be exposed on IDL level. (for other languages binding)
Therefore, I want to use AudioChannelAgent directly in MediaElement.
MozReview-Commit-ID: 43FvDeLpZPt
--HG--
extra : rebase_source : 78741d791eb66cfb6223866ce823f217e3438fdb
Regression by bug1262053 because of removing the current playing checking.
It would cause that the media control on Fennec shows too early and some media
elements would also generate the media control even it doens't have available
source.
Therefore, I think we still need to add this checking back.
MozReview-Commit-ID: 1m1ywmLmpSe
--HG--
extra : rebase_source : 30741b0e7bf1bb15349caaf1ecbcdad8cd92dcc3
8eef5d2cc850 in Bug 1300654 incorrectly resolved a merge conflict and backed
out part of the fix for Bug 1280829. Relanding.
MozReview-Commit-ID: 6DIkUrc9R4t
Also replace the check for IsPotentiallyPlaying() with !mPaused because mReadyState >= HAVE_FUTURE_DATA guarantees playback is not ended.
MozReview-Commit-ID: HetaVDgYEW0
--HG--
extra : rebase_source : 94dd2156db419240ad4462208540e9f98f1c8050
Previously, we'd block loading of non-MSE content when there was a MediaKeys
attached, that is, we'd assume that all content was EME content if a MediaKeys
was attached. But some sites attach a MediaKeys and then load non-MSE non-EME
content, and that (despite being a bit silly) shouldn't fail.
MozReview-Commit-ID: 9LupWaehXim
--HG--
extra : rebase_source : 6db281feb27a61b49dc565d4d4562a3d2366fc89
There's no such step defined in the spec. Despite, this code can no longer be called when readyState is HAVE_NOTHING
MozReview-Commit-ID: 2fDoNHt1COp
--HG--
extra : rebase_source : d8b79acee85b00331f55153626038e26cd4a72c7
There's no such step defined in the spec. Despite, this code can no longer be called when readyState is HAVE_NOTHING
MozReview-Commit-ID: 2fDoNHt1COp
--HG--
extra : rebase_source : baa542cdf3144db8b941c20f22548c67bfc06346
Instead of "not visible", "approximately visible", and "visible" (in display port) we now have "approximately not visible", and "approximately visible" which includes "visible".
Instead of "not visible", "approximately visible", and "visible" (in display port) we now have "approximately not visible", and "approximately visible" which includes "visible".
Enable nsAttrValue::EnumTable to be initialized with enum. So, we could get rid
of the castings in EnumTable. Fix EnumTable initialization comment.
For those untyped enumerations, declare them with uint8_t, as to other typed
enumerations with type size larger than int16_t, force casting to int16_t.
Use {nullptr,0} instead of {0} to represent the last entry.
MozReview-Commit-ID: 7Dma3Apkmxj
--HG--
extra : rebase_source : b2289866c4c33d80c8e170727bf109d018d92f67
In ancient degisn, we would only register audio channel after the media element has audio track and enoguh data to playback,
that is because the "audio-playback" event would be dispatched with the registration, and then shows the tab audio indicator.
However, now the event dispatching doesn't follow with the registration, it would be triggered when the media element has
really audible data which would be notified from media decoder.
Therefore, the media element without audio track or without enough data can also register audio channel agent, it won't affect
the display of tab audio indicator. The reason we need to do that is for blocking autoplay media in the non-visited tab.
The autoplay can be adding "autoplay" keyword or playing by the script, and we don't want to dispatch dom event for blocked
media. Therefore, we should register audio channel agent to know whether it needs to be blocked immediately even the media
element doesn't have any enough data which can let us to distinguish it have any audio track or not (this information can
be known from metadata).
First, we must check whether the media is blocked which is notified by audio channel agent, and then we can decide whether
need to dispatch the event. If we don't register audio channel agent, that we can't get blocking information.
MozReview-Commit-ID: HLLkOuecql1
--HG--
extra : rebase_source : 99c34f0185276ecd5b70ae09959b47c584d1564e
If the media was blocked, we would postpone the dom event and dispatch them after media is resumed.
MozReview-Commit-ID: LcdJtH16qQn
--HG--
extra : rebase_source : 209d18925a88a9e0dac82f2c0695ec1b60f54f63
This adds support for HTMLMediaElement.mozCaptureStream() and
mozCaptureStreamUntilEnded() for a HTMLMediaElement playing a MediaStream.
This is up to spec, while capturing a HTMLMediaElement playing a file is not.
This incompatibility means we cannot mix sources for the returned MediaStream.
As such, a MediaStream returned while the HTMLMediaElement was playing a file
will only have content while the element is playing files. If the src changes
to a MediaStream, the stream will be empty.
It works the same way if a MediaStream was captured while the HTMLMediaElement
was playing another MediaStream.
This is due to TrackID management - MediaDecoder doesn't care, and creates new
tracks when you seek, so users are unable to keep track, while for MediaStream
we control everything from main thread and keep track of the TrackIDs used
previously.
This also adds a separate path from MediaElementAudioSourceNode so that we don't
forward video tracks when the returned MediaStream is only used internally for
WebAudio. We should in that case not require a DOMMediaStream but just forwarding
tracks to a TrackUnionStream should be enough, and will save us some cpu cycles.
This is however fine for now as it's simpler.
MozReview-Commit-ID: Bg8hESDISDU
--HG--
extra : rebase_source : 83885a73ec8cfc5fbe3c30a9330a52cd6b6dff12
extra : source : f1aec79078869c0a6435a1c06957c649d7a40dd9
Sometimes a track is added to a stream synchronously (before the stream is
exposed to script), and sometimes asynchronously (see the mediacapture-main spec
on the "addtrack" event).
In the latter case we might still need to create the MediaStreamTrack object
synchronously for tracking purposes. CaptureStream of Media element playing a
MediaStream wants this.
MozReview-Commit-ID: 7me8xzN7rwj
--HG--
extra : rebase_source : 4f129b127b855e47aad2ae9ab3981ffde057412d
This prepares HTMLMediaElement for having a separate MediaStreamTrackSource for
MediaStreams, StreamCaptureTrackSource.
MozReview-Commit-ID: FVrYxFgvXgA
--HG--
extra : rebase_source : 5c162a0e861fa693fea0ba6b94b8e45446c0c13c
extra : source : a9151ac77a81573b8dbd9fee9c8aa09ba8dc7812
On Linux x64 PGO try, HTMLMediaElement was reliably invoking
decoder->NotifyOwnerActivityChanged() after SetVisible(false) was
called. This caused the pending suspend to be cancelled and the test
waits for an event that never arrives.
Fixed by adding 'forced hidden' to MediaDecoder that overrides the
element visibility that comes from HTMLMediaElement.
MozReview-Commit-ID: 5aRhxxZ5cZd
--HG--
extra : rebase_source : 5a4e1c44ddd2265eab545f8fe19c4ae47cebf7bf
Bubble information from SamplesWaitingForKey to an HTMLMediaElement so that we
can emit a waitingForKey event. If we are unable to decode more samples due to
needing a key the event will be signalled. See
http://w3c.github.io/encrypted-media/#dom-evt-waitingforkey for more information
on this event.
The code in place before this patch handles updating readyState when we are
waiting for a key, this patch adds the event which should be emitted in such a
case. The spec defines certain preconditions should be the case before running
the algo to emit this event. For example, the element should potentially be
playing, and it should have at least HAVE_FUTURE_DATA ready state. This is not
strictly true for when the new code is run, due how existing code handles ready
state. We are honoring the spirit of the spec, though the letter of the spec is
lightly gone against in the handling of the preconditions.
MozReview-Commit-ID: LKlDd4wkRSE
--HG--
extra : rebase_source : 2c61fc41636e430afa23240ad5d26c886124d87f
We will now record '0' in VIDEO_INTER_KEYFRAME_MAX_MS when only one keyframe is
found when playing a video (played for at least enough time to be eligible for
video-decode-suspend feature.)
MozReview-Commit-ID: ALDmSYxs2f1
--HG--
extra : rebase_source : 70b19e9c5c976f764a744dc4bf7a0760e80dd834
This patch makes most Run() declarations in subclasses of nsIRunnable have the
same form: |NS_IMETHOD Run() override|.
As a result of these changes, I had to add |override| to a couple of other
functions to satisfy clang's -Winconsistent-missing-override warning.
--HG--
extra : rebase_source : 815d0018b0b13329bb5698c410f500dddcc3ee12
After a video has been playing while hidden for a certain time, count the time
until it is not hidden anymore (or it has finished playing), to test-drive how
much decoding time would have been saved by the video-decode-suspend feature.
Note that this is done inside HTMLMediaElement by simulating what should happen
in the MDSM, because instrumenting the MDSM itself and friends would have been
harder and more intrusive.
MozReview-Commit-ID: LdxhPtmoXeA
--HG--
extra : rebase_source : 151e1f1383ab5c445eb8c957be8363340cdc4ab1
After a video has been playing while hidden for a certain time, count the time
until it is not hidden anymore (or it has finished playing), to test-drive how
much decoding time would have been saved by the video-decode-suspend feature.
Note that this is done inside HTMLMediaElement by simulating what should happen
in the MDSM, because instrumenting the MDSM itself and friends would have been
harder and more intrusive.
MozReview-Commit-ID: LdxhPtmoXeA
--HG--
extra : rebase_source : c4063d7c39b56e62e4f397bc21ef889ed14307c8
In this patch, we first deal with the case of MediaElement. Now we replace |PlayVideo| with |VideoFrameContainer::SetCurrentFrames| in |SourceMediaStream::AppendToTrack|. The MSG use TimeStamp::Now() for the TimeStamp of each video frame in most of case except MediaElement case. Becasue the MediaElement has its own VideoQueue, we need to calucalte the correct Timestamp based on the StartTimeStamp of this MediaStream and the elpased time of the video frame in DecodedStream.
MozReview-Commit-ID: 2bm2AHkFXHu
--HG--
extra : transplant_source : %3D%AA%00%CE%A3SV5%8F%84%96%AC%E2%D9%10%EC%85%07N%DF
Replace the pointer of VideoFrameContainer with the pointer of MediaStreamVideoSink.
MozReview-Commit-ID: 5bqEMpemwuR
--HG--
extra : transplant_source : %008z%D8W%EE%87%8E%E9/%2CT%26%EBvo%AE%099%A6
In this patch, we first deal with the case of MediaElement. Now we replace |PlayVideo| with |VideoFrameContainer::SetCurrentFrames| in |SourceMediaStream::AppendToTrack|. The MSG use TimeStamp::Now() for the TimeStamp of each video frame in most of case except MediaElement case. Becasue the MediaElement has its own VideoQueue, we need to calucalte the correct Timestamp based on the StartTimeStamp of this MediaStream and the elpased time of the video frame in DecodedStream.
MozReview-Commit-ID: 2bm2AHkFXHu
--HG--
extra : amend_source : 7b9c51dcb4046c6c807d1cf1e48bef301e45fa8e
Replace the pointer of VideoFrameContainer with the pointer of MediaStreamVideoSink.
MozReview-Commit-ID: 5bqEMpemwuR
--HG--
extra : amend_source : 7eb1e87fdcbc61f2f9831fa3a6d803cc50306604
In this patch, we first deal with the case of MediaElement. Now we replace |PlayVideo| with |VideoFrameContainer::SetCurrentFrames| in |SourceMediaStream::AppendToTrack|. The MSG use TimeStamp::Now() for the TimeStamp of each video frame in most of case except MediaElement case. Becasue the MediaElement has its own VideoQueue, we need to calucalte the correct Timestamp based on the StartTimeStamp of this MediaStream and the elpased time of the video frame in DecodedStream.
MozReview-Commit-ID: 2bm2AHkFXHu
--HG--
extra : transplant_source : %C4n%D7a%10%CFK%D5%F2%DC%10%08%C2%24%EC%11%13J%DB%5D
Replace the pointer of VideoFrameContainer with the pointer of MediaStreamVideoSink.
MozReview-Commit-ID: 5bqEMpemwuR
--HG--
extra : transplant_source : %9D%86%93%A6%DF%D5%9Ep%20%DF%FD%C1%E2%BA%A3Gq%1A%7E%A3
Notify audible-changing when resuming audio by MediaElement::play() instead of control interface.
MozReview-Commit-ID: 10a1npQ46Th
--HG--
extra : rebase_source : 16cce6782a132a2aab4ad66dc216d6237fc2ba9f
Use the new FrameStatistics members to report telemetry about inter-keyframe
timings.
MozReview-Commit-ID: 1ZWU2qpSWyC
--HG--
extra : rebase_source : f13fffc2bc81ad6090c3f494cc99e1d198f3256d
HTMLVideoElement can expose its thread-safe FrameStatistics object, so that
HTMLMediaElement can access more adequate data for its telemetry, without
having to use an intermediary (and potentially less accurate)
VideoPlaybackQuality object.
This will also help with accessing other/new FrameStatistics members later on.
MozReview-Commit-ID: AT7mEGy0zGr
--HG--
extra : rebase_source : 35bf6673cc0acd9d4e6e1a58c573749689614d43
Removed test blocking execution of ReportTelemetry in non-MSE case.
EME currently depends on MSE, so ReportEMETelemetry won't do anything in the
non-MSE case anyway. But I think it's good to remove the MSE test as well, in
case we change that limitation in the future.
MozReview-Commit-ID: 8sTg5F13KMs
--HG--
extra : rebase_source : f4acd3011f34be11b37c221ffd8439f6396fd37d
These numbers are only used to calculate a percentage, so they don't really
need to be 64-bit long.
MozReview-Commit-ID: FfdyStjGITL
--HG--
extra : rebase_source : 704d7664a3462f68f2c495e3b5d66f71180d290b
Note that this is a simple duration counter based on the existing (non-hidden)
play-time, but it only counts when video is playing while hidden.
There is no bucketing yet, future probes may add finer-grained information.
MozReview-Commit-ID: DUfryXjGBAN
--HG--
extra : rebase_source : 4d0bd7f60329dba6c6a6ab039cc733315c50b686
This is so that if media autoplay is disabled, a site can capture user intent
to play in their click handlers by calling load() on an empty video element to
"bless" the video element with the ability to play later when the video has
loaded.
Large video sites typically have a catalog view of videos to chose from on
their main page, and when the user clicks on one, with this patch, the site can
bless a video element so that it will play by beginning the load of that video
in the click handler. Often these sites do a bunch of asynchronous things
before they actually get around to playing the video, they don't call play in
the click handler, so we need another way to bless the videos so they can play.
We allow seeks in a click handler to capture user intent to play, so I don't
see why we should not also allow a load() to capture user intent.
MozReview-Commit-ID: KzjNcn3s6od
--HG--
extra : rebase_source : 6424e04ecb5220b765b69e815fc7b15c49ea6cb8
This can reduce the include header dependency. MediaStreamVideoSink will inherit from DirectMediaStreamTrackListener. But we can't use forward declaration on MediaStreamListener because the usage of nsTArray<RefPtr<MediaStreamVideoSink>>.
MozReview-Commit-ID: 328s4Kw9NvW
--HG--
extra : transplant_source : %D2%18%E3%3B%0C%D8%F04%F3%EB%EB%A0%A7%8B%B1%A9%AB%97rY
1. If mHasUserInteraction MediaElement is false, don't run the TimeMarchesOn because the element is not played. 2. Update the activeCueList only in TimeMarchesOn(). 3. Run TimeMarchesOn() at the beginning of play. r=rillian
MozReview-Commit-ID: BhwsIfRm3B2
--HG--
extra : rebase_source : 9713d4f467f1d708f65a25e54435d0c6e8ff1816
Rename those two function to better name alignment with AddDirectListener and AddDirectTrackListener.
MozReview-Commit-ID: 6QY08oyih1X
--HG--
extra : rebase_source : e0f2ac5de75d54a870f5a99f08505e40aa0696d9
For plain media playback, the buffered range will always be empty if readyState is HAVE_NOTHING has we need to decode the metadata to determine the duration and eventually decode the first frame to determine the start time. With MSE however, the buffered range is per spec directly related to the source buffer buffered range.
So we can always simply query the MediaDecoder to determine the buffered range regardless of the readyState value.
MozReview-Commit-ID: BQs8iuUCiNw
--HG--
extra : rebase_source : 28fea8a89eab38b481a15d76061debacce86274f
This function is no longer MSE-specific, so use a more general
ReportTelemetry() name for it.
MozReview-Commit-ID: JU7S4pEFLWk
--HG--
extra : rebase_source : 2a8a9e2e3d07df51c32c98796ebedb6f64c9fef4
Rename the VIDEO_MSE_UNLOAD_STATE telemetry probe to just
VIDEO_UNLOAD STATE and reactivate it. We were using this
to track MSE deployment success and would now like to
generalize it to all media playback.
We can remove this probe from histogram-whitelists.json because
we now define alert emails and bug numbers.
MozReview-Commit-ID: Jwmdkgl8CC3
Rename the VIDEO_MSE_PLAY_TIME_MS telemetry probe to just
VIDEO_PLAY_TIME_MS and make it active for all video playback.
We were using this to track MSE deployment success. Now we'd
like to do something similar for video playback in general,
regardless of the origin. This allows us to simplify the
collection code somewhat.
MozReview-Commit-ID: 7s8pOQWipf4
It looks like VC++ doesn't like comparisons of nsCOMPtr to 0 after this
change, but those are bad style anyway, so I removed them from
TestCOMPtr.cpp instead of trying to make them work.
Under some circumstances, the MP3Demuxer is unable to calculate the buffered range which would prevent the readyState value to be recalculated. To get around this we force recalculation when the progress event is fired.
MozReview-Commit-ID: DTaxJjOX4JU
--HG--
extra : rebase_source : 83389805b14c88061e492525abaada1d63b4d5ea
Some refactoring, clean-ups, etc.
The main change is that the can-play status is passed when diagnostics are
finally recorded. This will help when introducing different types of
diagnostics in future patches (e.g., Key System issues).
MozReview-Commit-ID: 182ePlrMqn4
The Widevine CDM, and I believe the Adobe CDM too probably, do not mark a
session's keys as unusable when a key session is closed. This means that
the CDMCaps' copy of the keys that are usable is wrong, and so if the media
with the same keys is reloaded with the same MediaKeys, Firefox will assume
that the keys are usable before the license for the keys has been
re-negotiated, and so decryption will fail.
MozReview-Commit-ID: 1kTDzwSD8PE
--HG--
extra : rebase_source : 89c480f36bfaec11ceb5ca1e8d798b0ae795eaea
DecoderDoctorDiagnostics are now used at places where Firefox Chrome and/or
websites checks whether some media formats may be played:
- audio|video.canPlayType()
- audio|video resource loader
- MediaSource.IsTypeSupported()
- MediaSource.AddSourceBuffer()
MozReview-Commit-ID: B1KdjXODq9j
Pass declared-but-yet-undefined DecoderDoctorDiagnostics pointer to various
routines that contribute to deciding if a media format can be played, and
those that create decoders.
Points where a DecoderDoctorDiagnostics can be injected are currently marked
with "/* DecoderDoctorDiagnostics* */ nullptr", and some will be used in
following patches.
MozReview-Commit-ID: 7u37bvY4CpW
HTMLMediaElement needs special protection when playing a stream since its
ImageContainer can outlive the video track of a stream.
Consider for instance when a (cross-origin) video track is removed from a
DOMMediaStream by a user and the remaining video track (non-CORS) does not yet
contain any actual video frames. The HTMLMediaElement will display a frame from
the removed track but the DOMMediaStream's principal has been updated to not
include the principal from the removed track.
With this patch we handle this by letting VideoFrameContainer notify
HTMLMediaElement when it has flushed out all video frames belonging to a
certain PrincipalHandle. I.e., when a new PrincipalHandle has been applied to the
underlying ImageContainer.
MozReview-Commit-ID: LvIZPl6Rdgj
--HG--
extra : rebase_source : cfbad5e5e7f43af4da4bfc213494b7b8e22cde17
This makes it consistent with PrincipalChangeObserver.
MozReview-Commit-ID: 91PtqFZRcW6
--HG--
extra : rebase_source : e39abb668be7fbd0dae0a701ec17b048c8761879
This allows DOMMediaStream to assign MediaStreamTrackSources to
dynamically created MediaStreamTracks.
MozReview-Commit-ID: 3v91zLiqfl7
--HG--
extra : rebase_source : 4fe67e92b9c833fe5ba964fc5f3b11ba21447f36
This lets a MediaStreamTrack communicate with its source/producer on the
main thread. It's for now used for stopping a track at the source and
retrieving some metadata, but it could also be a link between actual
sinks of a track and the source, to for instance let the source optimize
by scaling down the resolution when all sinks want lowres-video.
MozReview-Commit-ID: D4SJLr0aqhJ
--HG--
extra : rebase_source : ea511b5c86ca4836bfa980825f04617fef498261
MediaDecoder previously had 3 states within GetSeekable(), media is either
seekable, seekable but not supported by transport, or not seekable. Due to
changes to make cueless webms playable, a 4th option is needed: a file that is
not fully seekable, but may support seeking from the transport, such as these
webms, should only be seekable in the buffered range.
MozReview-Commit-ID: ISeFkngtrGU
The spec requires us to await a stable state and pause when a media element is removed from a document.
Additionally, before this change we would pause in UnbindFromTree only if HTMLMediaElement::mPaused is false.
However if the media element is set to autoplay, and we unbind before autoplay has had a chance to activate,
we will still continue loading and activate autoplay later. Calling pause, as required by the spec, means
we'll cancel autoplaying.
--HG--
extra : amend_source : 47f89e1a1e855f9304fbd12959f636c973ddf356
