When OffscreenCanvas::CommitFrameToCompositor uses the non-remote
texture canvas path with Skia, it uses ImageBridgeChild for compositing.
When ImageContainer::SetCurrentImages is called, there was an
intermediate state where the relevant textures were not yet marked as
read only for the compositor's consumption, because the event to do so
was dispatched asynchronously to the ImageBridgeChild thread. If the
owning thread of the canvas (main or DOM worker) ran immediately after
CommitFrameToCompositor, then we could run into texture reuse since
nothing marked the texture yet as being used for compositing. This had
the end result of sometimes displaying back buffer textures currently
being used for drawing on the display pipeline.
This patch makes it so that we mark OffscreenCanvas textures as read
only for the compositor before dispatching, and releasing the lock
either when we swap the images in the ImageContainer (winning the race
with ImageBridgeChild), or after the compositor has finished with it
(losing the race, if any, with ImageBridgeChild).
Additionally, to handle better the case where we run out of buffers, we
need to implement ImageBridgeChild::SyncWithCompositor, to be analogous
to how WebRenderBridgeChild::SyncWithCompositor works. We achieve this
by calling from ImageBridgeChild back into the appropriate
WebRenderBridgeChild based on the window ID associated with the canvas,
It also adds a new pref, gfx.offscreencanvas.shared-provider, which
allows one to switch between PersistentBufferProviderShared and Basic.
The latter of which is used if we fallback from using shared buffers if
it takes too long to get the shared buffers back from the compositor.
Differential Revision: https://phabricator.services.mozilla.com/D200991
UnregisterTextureOwner, if called before any use of UseRemoteTexture, can cause UseRemoteTexture to wait
for the texture owner to be created, since the texture owner does not exist, and there is no evidence it
was previously unregistered.
This patch attempts to address the issue by delaying the actual UnregisterTextureOwner until all such
UseRemoteTexture instances are processed. This is accomplished by noting that UseRemoteTexture ops come
in via transactions from a CompositableForwarder and so all are associated with a forwarder transaction
with a FwdTransactionId. RecordedTextureData on destruction reports the last FwdTransactionId associated
with its final UseRemoteTexture before it attempts to call UnregisterTextureOwner. If RemoteTextureMap
has not been notified of a given FwdTransactionId yet, then the UnregisterTextureOwner call will be
deferred until it has seen this FwdTransactionId.
This adds a RemoteTextureTxnScheduler to track the issuing of dependencies and waiting for FwdTransactionIds.
This patch also cleans up the issuing of FwdTransactionIds themselves to be associated with a given
top-level protocol so that all sub-protocols have transaction numbers that can be safely compared amongst
each other. This makes dependency expiration more robust since any advancement of the transaction number
from any source can help retire expired dependencies.
Differential Revision: https://phabricator.services.mozilla.com/D197895
UnregisterTextureOwner, if called before any use of UseRemoteTexture, can cause UseRemoteTexture to wait
for the texture owner to be created, since the texture owner does not exist, and there is no evidence it
was previously unregistered.
This patch attempts to address the issue by delaying the actual UnregisterTextureOwner until all such
UseRemoteTexture instances are processed. This is accomplished by noting that UseRemoteTexture ops come
in via transactions from a CompositableForwarder and so all are associated with a forwarder transaction
with a FwdTransactionId. RecordedTextureData on destruction reports the last FwdTransactionId associated
with its final UseRemoteTexture before it attempts to call UnregisterTextureOwner. If RemoteTextureMap
has not been notified of a given FwdTransactionId yet, then the UnregisterTextureOwner call will be
deferred until it has seen this FwdTransactionId.
This adds a RemoteTextureTxnScheduler to track the issuing of dependencies and waiting for FwdTransactionIds.
This patch also cleans up the issuing of FwdTransactionIds themselves to be associated with a given
top-level protocol so that all sub-protocols have transaction numbers that can be safely compared amongst
each other. This makes dependency expiration more robust since any advancement of the transaction number
from any source can help retire expired dependencies.
Differential Revision: https://phabricator.services.mozilla.com/D197895
The idea of trying to obsolete the presented texture from the last frame when
creating a new texture, if we believe it wasn't used, seems problematic and
may not actually address the fundamental issue at hand.
It seems more like there are use-cases where the recorded texture is locked
and unlocked many times in succession, whereas we actually only intend to
present it once.
This decouples the TextureLock/TextureUnlock events from a new PresentTexture
event which handles the actual creation of remote texture ids, so that regardless
of how many times we lock/unlock the recorded texture, we don't actually churn
presentation unless we truly need to. This seems like a better way of addressing
the memory reuse issue than trying to remove textures since we don't create them
in the first place.
Differential Revision: https://phabricator.services.mozilla.com/D197453
This patch makes the CanvasManagerChild creates/manager the
ActiveResourceTracker instead of the WebRenderBridgeChild. Since PCanvas
is now managed by PCanvasManager, and ActiveResourceTracker is only used
by the PCanvas recording plumbing, we need it on every thread that the
CanvasManagerChild can be created.
Differential Revision: https://phabricator.services.mozilla.com/D189529
This patch makes the CanvasManagerChild creates/manager the
ActiveResourceTracker instead of the WebRenderBridgeChild. Since PCanvas
is now managed by PCanvasManager, and ActiveResourceTracker is only used
by the PCanvas recording plumbing, we need it on every thread that the
CanvasManagerChild can be created.
Differential Revision: https://phabricator.services.mozilla.com/D189529
This adds the necessary infrastructure for CanvasTranslator to allocate DrawTargetWebgl
instead of just allocating TextureData, and to use RemoteTextureMap to handle sending
the DrawTargetWebgl frames to the compositor.
This optimizes snapshot transport to use fewer copies to and from shmems when we know
the snapshot contents can be sourced from a shmem.
This adds a blocking mechanism separate from deactivation so that existing DrawTargetWebgls
can continue processing events while denying further ones from being created in the event
that allocating further DrawTargetWebgls might cause problems, but so that we don't disrupt
canvases that are already in flight.
PersistentBufferProviderAccelerated still remains the buffer provider for the new setup,
but just allocates a single RecordedTextureData internally. Since DrawTargetWebgl already
does its own swap chain management internally, we do not want to use the multiple texture
client strategy that PersistentBufferProviderShared does.
This adds a fallback mechanism such that if DrawTargetWebgl allocation fails, a TextureData
is allocated instead that still sends results to RemoteTextureMap. This has the advantage
that we don't need to synchronously block in the content process to verify if acceleration
succeeded, as the costs of such blocking are rather extreme and we must still produce the
rendered frame to ensure the user sees the correct result if the speculative acceleration
failed. It then notifies the content process asynchronously via the refresh mechanism to
try to recreate a non-accelerated buffer provider when it is ready.
There is one additional hitch in RemoteTextureMap that we need to add a mechanism to deal
with the setup of the RemoteTextureOwner. When display list building initially needs to get
the remote texture, the RemoteTextureOwner might not exist yet. In this case, we need to add
a block to ensure we wait for that to occur so that we do not render an erroneous result.
Previously, this block was handled in ClientWebGLContext. Since that is no longer used,
the block must be reinstated somewhere else until a non-blocking mechanism for display list
building to proceed with a stub texture host wrapper can be implemented.
Currently this leaves the gfx.canvas.remote and gfx.canvas.accelerated prefs as separate
toggles rather than trying to lump everything into one mechanism. While this may be desirable
in the future, currently Direct2D remote canvas is a separate acceleration mechanism that
needs to co-exist with the WebGL acceleration, and so being able to toggle both on or off
for testing is desirable.
Differential Revision: https://phabricator.services.mozilla.com/D194352
This adds the necessary infrastructure for CanvasTranslator to allocate DrawTargetWebgl
instead of just allocating TextureData, and to use RemoteTextureMap to handle sending
the DrawTargetWebgl frames to the compositor.
This optimizes snapshot transport to use fewer copies to and from shmems when we know
the snapshot contents can be sourced from a shmem.
This adds a blocking mechanism separate from deactivation so that existing DrawTargetWebgls
can continue processing events while denying further ones from being created in the event
that allocating further DrawTargetWebgls might cause problems, but so that we don't disrupt
canvases that are already in flight.
PersistentBufferProviderAccelerated still remains the buffer provider for the new setup,
but just allocates a single RecordedTextureData internally. Since DrawTargetWebgl already
does its own swap chain management internally, we do not want to use the multiple texture
client strategy that PersistentBufferProviderShared does.
This adds a fallback mechanism such that if DrawTargetWebgl allocation fails, a TextureData
is allocated instead that still sends results to RemoteTextureMap. This has the advantage
that we don't need to synchronously block in the content process to verify if acceleration
succeeded, as the costs of such blocking are rather extreme and we must still produce the
rendered frame to ensure the user sees the correct result if the speculative acceleration
failed. It then notifies the content process asynchronously via the refresh mechanism to
try to recreate a non-accelerated buffer provider when it is ready.
There is one additional hitch in RemoteTextureMap that we need to add a mechanism to deal
with the setup of the RemoteTextureOwner. When display list building initially needs to get
the remote texture, the RemoteTextureOwner might not exist yet. In this case, we need to add
a block to ensure we wait for that to occur so that we do not render an erroneous result.
Previously, this block was handled in ClientWebGLContext. Since that is no longer used,
the block must be reinstated somewhere else until a non-blocking mechanism for display list
building to proceed with a stub texture host wrapper can be implemented.
Currently this leaves the gfx.canvas.remote and gfx.canvas.accelerated prefs as separate
toggles rather than trying to lump everything into one mechanism. While this may be desirable
in the future, currently Direct2D remote canvas is a separate acceleration mechanism that
needs to co-exist with the WebGL acceleration, and so being able to toggle both on or off
for testing is desirable.
Differential Revision: https://phabricator.services.mozilla.com/D194352
This adds the necessary infrastructure for CanvasTranslator to allocate DrawTargetWebgl
instead of just allocating TextureData, and to use RemoteTextureMap to handle sending
the DrawTargetWebgl frames to the compositor.
This optimizes snapshot transport to use fewer copies to and from shmems when we know
the snapshot contents can be sourced from a shmem.
This adds a blocking mechanism separate from deactivation so that existing DrawTargetWebgls
can continue processing events while denying further ones from being created in the event
that allocating further DrawTargetWebgls might cause problems, but so that we don't disrupt
canvases that are already in flight.
PersistentBufferProviderAccelerated still remains the buffer provider for the new setup,
but just allocates a single RecordedTextureData internally. Since DrawTargetWebgl already
does its own swap chain management internally, we do not want to use the multiple texture
client strategy that PersistentBufferProviderShared does.
This adds a fallback mechanism such that if DrawTargetWebgl allocation fails, a TextureData
is allocated instead that still sends results to RemoteTextureMap. This has the advantage
that we don't need to synchronously block in the content process to verify if acceleration
succeeded, as the costs of such blocking are rather extreme and we must still produce the
rendered frame to ensure the user sees the correct result if the speculative acceleration
failed. It then notifies the content process asynchronously via the refresh mechanism to
try to recreate a non-accelerated buffer provider when it is ready.
There is one additional hitch in RemoteTextureMap that we need to add a mechanism to deal
with the setup of the RemoteTextureOwner. When display list building initially needs to get
the remote texture, the RemoteTextureOwner might not exist yet. In this case, we need to add
a block to ensure we wait for that to occur so that we do not render an erroneous result.
Previously, this block was handled in ClientWebGLContext. Since that is no longer used,
the block must be reinstated somewhere else until a non-blocking mechanism for display list
building to proceed with a stub texture host wrapper can be implemented.
Currently this leaves the gfx.canvas.remote and gfx.canvas.accelerated prefs as separate
toggles rather than trying to lump everything into one mechanism. While this may be desirable
in the future, currently Direct2D remote canvas is a separate acceleration mechanism that
needs to co-exist with the WebGL acceleration, and so being able to toggle both on or off
for testing is desirable.
Differential Revision: https://phabricator.services.mozilla.com/D194352
Device reset could cause front TextureClient lost in PersistentBufferProviderShared::SetKnowsCompositor(). In this case, ShareableCanvasRenderer::UpdateCompositableClient() does not have TextureClient for compositor.
Differential Revision: https://phabricator.services.mozilla.com/D176748
Device reset in test_device_reset.html replaces CompositorBridgeChild to a new one. And shutting down old CompositorBridgeChild also destroys TextureClients.
Differential Revision: https://phabricator.services.mozilla.com/D164175
By default, BorrowSnapshot is pessimistic and forces DrawTargetWebgl to return a data snapshot on
the assumption that the snapshot might be used off thread. However, if we actually know the DrawTarget
we're going to be drawing the snapshot to, then we can check if they're both DrawTargetWebgls with
the same internal SharedContext. In that case, we can use a SourceSurfaceWebgl snapshot which can
pass through a GPU texture to the target. This requires us to plumb the DrawTarget down through
SurfaceFromElement all the way to DrawTargetWebgl to make this decision.
Differential Revision: https://phabricator.services.mozilla.com/D162176
BorrowSnapshot can be called by OffScreenCanvas in various places that may send
a SourceSurfaceWebgl to the main thread. If it did not originate from the main
thread, then this can cause multiple threads to use it. In general we want to
avoid this. For now, override BorrowSnapshot and make it always force a Skia
snapshot that can be safely shared between threads instead of SourceSurfaceWebgl.
Differential Revision: https://phabricator.services.mozilla.com/D152417
With async present we can now rely on being able to do readbacks from WebGL
in the GPU process, rather than needing CopySnapshotTo to accelerate this in
the content process. Just remove CopySnapshotTo since it doesn't help anymore.
Differential Revision: https://phabricator.services.mozilla.com/D150721
Since we're going to be keeping around empty cache pages, which can further
increase memory pressure, it would be nice if we respond to memory pressure
events by evicting the empty cache pages.
Differential Revision: https://phabricator.services.mozilla.com/D145499
This adds OnEvent hooks to DrawTargetWebgl for various events so that
profile counters may be maintained each frame. These profile counters
try to determine if a software fallback happened or an uncacheable event
occurs that requires either uploading data to the GPU or reading back
from the GPU, events which can cause substantial slowdown if they happen
repeatedly even without an explicit fallback to software rasterization.
When it is determined a threshold has been reached (as controlled by
some prefs), RequiresRefresh() in PersistentBufferProvider is used to
signal that we should recreate the PersistentBufferProvider of a different
type and thus disable acceleration.
Differential Revision: https://phabricator.services.mozilla.com/D142646
Currently within DrawTargetWebGL, the Skia framebuffer and external software
surfaces are in BGRA, while the WebGL framebuffer is in RGBA. This requires
swizzling all software surfaces to RGBA on upload and swizzling surfaces
back to BGRA on readback. This can either require intermediate surfaces or
extra costly processing.
Now that CopyToSwapChain is available with support for a BGRA blit, we can
remove these complications by just treating the WebGL framebuffer as if it
was BGRA directly, so that any uploads or readbacks do not require a swizzle.
Differential Revision: https://phabricator.services.mozilla.com/D139066
Most of the support for presenting a WebGLFramebuffer to a swap chain existed as part of the
mechanism for opaque WebXR framebuffer support. However, such "opaque" framebuffer are meant
to be opaque in the sense that their attachments can't be inspected or changed, which does
not provide the requisite level of control for efficiently implementing Canvas2D snapshots.
To this end, the existing Present mechanism is slightly extended to allow presenting to the
swap chain already present in WebGLFramebuffer without the existence of a corresponding
MozFramebuffer.
This also fixes a bug in that AsWebgl() was no longer being utilized in CanvasRenderer, such
that a new mechanism that routed GetFrontBuffer() was needed to fix the code rot.
There are also some efforts to remove a couple redundant copies I noticed in profiles along
the way.
Differential Revision: https://phabricator.services.mozilla.com/D138119
Most of the support for presenting a WebGLFramebuffer to a swap chain existed as part of the
mechanism for opaque WebXR framebuffer support. However, such "opaque" framebuffer are meant
to be opaque in the sense that their attachments can't be inspected or changed, which does
not provide the requisite level of control for efficiently implementing Canvas2D snapshots.
To this end, the existing Present mechanism is slightly extended to allow presenting to the
swap chain already present in WebGLFramebuffer without the existence of a corresponding
MozFramebuffer.
This also fixes a bug in that AsWebgl() was no longer being utilized in CanvasRenderer, such
that a new mechanism that routed GetFrontBuffer() was needed to fix the code rot.
There are also some efforts to remove a couple redundant copies I noticed in profiles along
the way.
Differential Revision: https://phabricator.services.mozilla.com/D138119
This removes some of the changes that meant we started using
mPermanentBackBuffer straight away and we now wait until we actually try and
lock a read locked texture.
While this might still give a very small risk of contention, it gives
improvements in the following two circumstances.
* If a canvas texture is never forwarded and never read locked, it means we will
only use one texture with no copies.
* If a canvas is always fully overwritten at the start of the frame (and a
snapshot is not taken between frames), then we avoid a copy on each frame.
This also adds back in code so that on an OPEN_READ_WRITE lock we cache the data
surface if required, because that texture will be the new front buffer and we
won't be using mPermanentBackBuffer at that point.
Depends on D132601
Differential Revision: https://phabricator.services.mozilla.com/D132602
This measure was originally put in to help with what was believed to be an issue
with ClearCachedResources, but we now think it was down to textures being
re-forwarded on tab switch when already read locked.
A change in bug 1717209 fixed this, so I think we can safely remove
mTextureLockIsUnreliable, which would cause some compilcations with the
following patch.
Differential Revision: https://phabricator.services.mozilla.com/D132601
This mainly provides DrawTargetWebgl, which implements the subset of the DrawTarget
API necessary for integration with CanvasRenderingContext2D. It translates them to
suitable commands for its internal ClientWebGLContext, which then manages remoting
WebGL requests to the parent/GPU process.
Currently two shaders are used for drawing Canvas2D primitives, but can be expanded
in the future. These are an image shader and a solid color shader.
The core of this implementation revolves around TexturePacker and TextureHandle,
which cope with the necessity of frequently uploading SourceSurfaces for use with
WebGL. TexturePacker implements a bin-packing algorithm for packing these uploads
into texture pages, which can either be SharedTextures if they are reasonably small,
or StandaloneTextures if they are too big to pack in a SharedTexture. Each upload
is assigned a TextureHandle which is used to manage it in a move-to-front cache,
so that we can easily eject TextureHandles from the back of the cache if we have
too many. These TextureHandles are associated with the SourceSurface that spawned
them to more easily manage their lifetimes.
There are further dependent caches for dealing with blurred shadows and with text.
Shadows are cached in an uploaded texture bound to the SourceSurface that generated
them. Text is handled by caching entire runs in the GlyphCache (keyed by both their
rendering parameters and their glyphs). The text is first rasterized to a surface
and then uploaded to a texture in the GlyphCache which can be reused should the
text be encountered again.
To deal with commands we can't accelerate, a separate internal DrawTargetSkia is
also maintained. The content of the WebGL framebuffer is copied into it so that
drawing can then proceed in software from there. It remains in this fallover state
until the next frame, when it resets back to using the WebGL framebuffer again.
This acceleration is disabled by default. To enable it, you must toggle the pref
"gfx.canvas.accelerated" to true. This should be suitably different from the naming
of the previous SkiaGL prefs to not alias with them. There are a few dependent prefs
that follow from the previous SkiaGL prefs for setting the size limitations for
acceleration and also limitations for the internal texture cache.
Differential Revision: https://phabricator.services.mozilla.com/D130388
This mainly provides DrawTargetWebgl, which implements the subset of the DrawTarget
API necessary for integration with CanvasRenderingContext2D. It translates them to
suitable commands for its internal ClientWebGLContext, which then manages remoting
WebGL requests to the parent/GPU process.
Currently two shaders are used for drawing Canvas2D primitives, but can be expanded
in the future. These are an image shader and a solid color shader.
The core of this implementation revolves around TexturePacker and TextureHandle,
which cope with the necessity of frequently uploading SourceSurfaces for use with
WebGL. TexturePacker implements a bin-packing algorithm for packing these uploads
into texture pages, which can either be SharedTextures if they are reasonably small,
or StandaloneTextures if they are too big to pack in a SharedTexture. Each upload
is assigned a TextureHandle which is used to manage it in a move-to-front cache,
so that we can easily eject TextureHandles from the back of the cache if we have
too many. These TextureHandles are associated with the SourceSurface that spawned
them to more easily manage their lifetimes.
There are further dependent caches for dealing with blurred shadows and with text.
Shadows are cached in an uploaded texture bound to the SourceSurface that generated
them. Text is handled by caching entire runs in the GlyphCache (keyed by both their
rendering parameters and their glyphs). The text is first rasterized to a surface
and then uploaded to a texture in the GlyphCache which can be reused should the
text be encountered again.
To deal with commands we can't accelerate, a separate internal DrawTargetSkia is
also maintained. The content of the WebGL framebuffer is copied into it so that
drawing can then proceed in software from there. It remains in this fallover state
until the next frame, when it resets back to using the WebGL framebuffer again.
This acceleration is disabled by default. To enable it, you must toggle the pref
"gfx.canvas.accelerated" to true. This should be suitably different from the naming
of the previous SkiaGL prefs to not alias with them. There are a few dependent prefs
that follow from the previous SkiaGL prefs for setting the size limitations for
acceleration and also limitations for the internal texture cache.
Differential Revision: https://phabricator.services.mozilla.com/D130388
If we are re-forwarding a texture that has not been updated then it might still
be read locked. We rely on the read lock for us to know when the Compositor or
Renderer has finished with the texture. So this change copies it to a new
texture, which we can safely read lock.
If the texture is not read locked then we ensure that it is set as updated,
which means we will read lock it as we forward it.
Differential Revision: https://phabricator.services.mozilla.com/D119546
