APZ now ensures that GetHandledResult() is Unhandled when appropriate,
so the widget code doesn't need to special-case eIgnore any more.
Differential Revision: https://phabricator.services.mozilla.com/D164585
This effectively backs out the fix for bug 1746336, while keeping the test
added in that bug (PanZoomControllerTest#touchActionWithWheelListener)
passing using a different fix approach (remembering the eager status
and using it in the computation of the delayed result).
Differential Revision: https://phabricator.services.mozilla.com/D164114
On Android when SWGL is enabled, occasionally when switching tabs it
refuses to render the new tab. The user will instead be left with the
contents of the previous tab, or if they have just switched to firefox
they will be left with a black screen.
This occurs when SyncResumeResizeCompositor is called from java code
prior to the gecko compositor having been created. This results in us
delaying the call to UiCompositorController::ResumeAndResize().
Instead, once the compositor eventually gets created we call
UiCompositorController::Resume() (without the resize).
This means that the compositor widget is left with a zero size, and
RenderCompositorLayersSWGL bails early on compositing as a result.
To fix this, we cache the X, Y, width, and height arguments passed to
SyncResumeResizeCompositor. Then to resume the compositor when it
eventually gets created, we call ResumeAndResize() with the cached
values, rather than just calling Resume.
Differential Revision: https://phabricator.services.mozilla.com/D161046
In bug 1772839 we were seeing a large number of crashes due to
encountering a webrender error after exhausting all fallback
configurations. At least in some cases, this was due to the compositor
being resumed with an Android Surface that was already in an abandoned
state, meaning we can never succeed in creating an EGL Surface.
We added a check for this condition, and a workaround, to the
GeckoView java code. However, we are still seeing crash reports
matching this signature. To help determine whether these are also due
to the Surface being abandoned, or due to some other reason, this
patch adds a deliberate crash much earlier in the pipeline if we
detect an abandoned Surface.
Differential Revision: https://phabricator.services.mozilla.com/D160042
The android magnifier widget does not work when using our
SurfaceControl rendering path, as we no longer render in to the
Surface provided by the SurfaceView, but instead into a child Surface
we have created and attached the SurfaceControl.
To fix this, we create a SurfaceView subclass, MagnifiableSurfaceView,
which allows us to set an override Surface to be used by the
magnifier. This class works by overriding getHolder() to return a
custom SurfaceHolder, which returns our override Surface rather than
the default one when called by the Magnifier class.
Depends on D157308
Differential Revision: https://phabricator.services.mozilla.com/D157309
In order to fix the magnifier widget being broken, the previous patch
in this bug added a mechanism to disable and enable the SurfaceControl
rendering path. This caused some glitches to occur, so we removed the
calls to that code in bug 1783542, but the code remained.
As we now have an alternative solution to fix the magnifier widget, we
no longer require this code. This patch therefore reverts the original
patch, to lead the way for the new solution in the next patch.
Differential Revision: https://phabricator.services.mozilla.com/D157308
When turning on fission, OOP child frame doesn't receive touch events such as
`touchstart`.
When dispatching touch event from GeckoView side, some informations such as
`mLayersId` are missing since copy constructor of `MultiTouchInput` doesn't
copy all information.
Since the structure of Input data has a lot of members, we should use
`std::move` instead of copy constructor of `InputData`. It can avoid
unnecessary copy.
Differential Revision: https://phabricator.services.mozilla.com/D154242
On Android we previously added a rendering path using the
SurfaceControl API, in order to work around an Android OS bug when
recovering from a GPU process crash. Unfortunately the Magnifier
widget (shown when moving a text selection caret) does not work when
rendering using SurfaceControl.
This patch makes it so that we temporarily disable the SurfaceControl
path when a text selection is active, allowing the Magnifier to work.
Unfortunately this means that if a GPU process crash occurs while
there is an active selection we will be unable to recover. Hopefully
this turns out to be a relatively rare occurence.
Differential Revision: https://phabricator.services.mozilla.com/D153454
When the session storage prefs are enabled, GeckoSession updateSessionState will provide the bundle of information, including zoom, scroll, and form data, to the delegate. Currently works for Fission and on Fenix.
Differential Revision: https://phabricator.services.mozilla.com/D148215
When the session storage prefs are enabled, GeckoSession updateSessionState will provide the bundle of information, including zoom, scroll, and form data, to the delegate. Currently works for Fission and on Fenix.
Differential Revision: https://phabricator.services.mozilla.com/D148215
When the session storage prefs are enabled, GeckoSession updateSessionState will provide the bundle of information, including zoom, scroll, and form data, to the delegate. Currently works for Fission and on Fenix.
Differential Revision: https://phabricator.services.mozilla.com/D148215
When the session storage prefs are enabled, GeckoSession updateSessionState will provide the bundle of information, including zoom, scroll, and form data, to the delegate. Currently works for Fission and on Fenix.
Differential Revision: https://phabricator.services.mozilla.com/D148215
In bug 1762424 we introduced a rendering path on Android using the
SurfaceControl API, in order to work around a bug preventing recovery
from a GPU process crash. However, the initial implementation caused
this bug: repeatedly sending the same SurfaceControl objects over AIDL
to the GPU process resulted in them being leaked, eventually causing
severe display issues. Not only were we duplicating the SurfaceControl
for each widget, but each time a widget was resized too.
This patch reworks our usage of the SurfaceControl API to avoid ever
having to send them cross-process. Instead, we create a single child
SurfaceControl object for each SurfaceControl that is attached to a
widget. (Typically there will be a single one shared between all
widgets, changing only when the app is paused and resumed, which is
much fewer than one per widget per resize.)
In the parent process we obtain the Surfaces that will be rendered in
to from the child SurfaceControls, and only send those Surfaces to the
GPU process. Thankfully unlike SurfaceControls, sending Surfaces
cross-process does not cause leaks. When the GPU process dies we
simply destroy all of the child SurfaceControls, and recreate them
again on-demand.
Differential Revision: https://phabricator.services.mozilla.com/D147437
Adds a GeckoView save to PDF API that can be used to save the current session’s page content to a PDF. The API returns a Java InputStream that can be used by the consumer to process the PDF.
Differential Revision: https://phabricator.services.mozilla.com/D146245
In bug 1762424 we introduced a compositing path using SurfaceControl
to work around an issue on Android 12 where the app would freeze when
attempting to recover from a GPU process crash. However, this caused
bug 1767128. Until we have a proper solution to that, we should
disable the SurfaceControl functionality.
This means on Android 12 the browser will be once again be unable to
recover from a GPU process crash. However, as part of bug 1762424 we
made it so the parent process crashes in this situation rather than
freezing, meaning Android 12 users will be no worse off than having
the GPU process disabled.
Differential Revision: https://phabricator.services.mozilla.com/D146114
This member is no longer used on Windows, and having it in the
base class doesn't provide a real abstraction anyway since
the child classes will break if anything about it is changed.
Differential Revision: https://phabricator.services.mozilla.com/D145132
On Android we have long since calculated the compositor widget size
from the Surface rather than using the main thread widget size. This
is to avoid a trip via the main thread in response to a
ResumeAndResize event on the UI thread.
AndroidCompositorWidget::mClientSize therefore gets calculated when
the compositor is resumed.
However, it is possible in some circumstances for the compositor to
receive a display list prior to it being resumed. In this bug's case,
SyncResumeAndResize is getting called before the UI thread has been
notified that the compositor has been initialized. It therefore cannot
resume the compositor, and we instead resume the compositor on the
subsequent NotifyCompositorCreated call. This starts a race between
the main thread paint and NotifyCompositorCreated being scheduled on
the UI thread then resuming the compositor via
PUiCompositorController. If we receive
WebRenderBridgeParent::RecvSetDisplayList prior to
UiCompositorControllerParent::RecvResumeAndResize, then
AndroidCompositorWidget::mClientSize will be zero. This results in us
setting zero-sized webrender scene rect, leading to webrender exiting
early during rendering, resulting in a black screen.
To fix this, allow the main thread to set the AndroidCompositorWidget
size, in addition to the UI thread being able to set it. We do so by
adding a NotifyClientSizeChanged method to
PlatformCompositorWidgetDelegate, called from
nsWindow::OnSizeChanged. The cross-process implementation of this uses
an IPDL call on PCompositorWidget, which shares a top-level protocol
with PWebRenderBridge, meaning calls are guaranteed to occur in
order. This means a resize event on the main thread is guaranteed to
set the CompositorWidget size before the display list from the
subsequent paint is received.
Differential Revision: https://phabricator.services.mozilla.com/D144594
This adds new version of the GeckoView API
GeckoDisplay.surfaceChanged(), which takes a single argument of a new
type GeckoDisplay.SurfaceInfo. As well as containing fields for each
the the existing surfaceChanged() arguments, this has an additional
SurfaceControl field. This must be provided when rendering in to a
SurfaceView on SDK level 29 or greater. On earlier SDK levels, or when
rendering in to a TextureView or SurfaceTexture, this can be
null. SurfaceViewWrapper and GeckoView classes are updated to handle
this correctly. The old surfaceChanged() methods have been deprecated,
and tests have been updated to use the new version.
When provided, the SurfaceControl is passed along with the Surface
through to the widget and, when enabled, over to the GPU process. The
compositor widget then creates a child Surface from that
SurfaceControl, and renders in to that child Surface rather than the
parent one.
This works around a bug on Android 12 where following the GPU process
dying the Surface was left in an unusable state, meaning subsequent
attempts to initialize a compositor would fail. Because the Surface is
now created by the GPU process it gets destroyed when the process
dies, therefore a new Surface can successfully be created when we
reinitialize the compositor.
Differential Revision: https://phabricator.services.mozilla.com/D143485
For example, the SDK class android.media.MediaDrm$KeyStatus will now
be defined as MediaDrm::KeyStatus rather than just KeyStatus.
Not only does this avoid polluting the top-level namespace, but it
also avoids a bug where invalid type names were generated if the
nested class contains a method with a parameter or return of the outer
class' type.
Differential Revision: https://phabricator.services.mozilla.com/D143043
In bug 1756700 we delayed initializing the compositor on android. A
consequence of this is that when
LayerViewSupport::SetDefaultClearColor() gets called
mUiCompositorControllerChild is still null, meaning we skip setting
the compositor's clear color. As a result, dark-mode fenix users have
once again started seeing white flashes while waiting for the page to
load.
Additionally, when the compositor is re-initialized (following a
fallback to software rendering, or a GPU process restart) we do not
set the clear color for the new compositor, which again can lead to
white flashes.
To fix both of these issues, this patch makes us cache the color value
passed to LayerViewSupport::SetDefaultClearColor(). Then whenever
LayerViewSupport::NotifyCompositorCreated() is called, we call
UiCompositorControllerChild::SetDefaultClearColor() to ensure the new
compositor uses the correct clear color.
Differential Revision: https://phabricator.services.mozilla.com/D141730
In bug 1756700 we delayed initializing the compositor on android. A
consequence of this is that when
LayerViewSupport::SetDefaultClearColor() gets called
mUiCompositorControllerChild is still null, meaning we skip setting
the compositor's clear color. As a result, dark-mode fenix users have
once again started seeing white flashes while waiting for the page to
load.
Additionally, when the compositor is re-initialized (following a
fallback to software rendering, or a GPU process restart) we do not
set the clear color for the new compositor, which again can lead to
white flashes.
To fix both of these issues, this patch makes us cache the color value
passed to LayerViewSupport::SetDefaultClearColor(). Then whenever
LayerViewSupport::NotifyCompositorCreated() is called, we call
UiCompositorControllerChild::SetDefaultClearColor() to ensure the new
compositor uses the correct clear color.
Differential Revision: https://phabricator.services.mozilla.com/D141730
We noticed a cold_view_nav_start regression on Fenix from enabling the
GPU process, and profiles showed time spent synchronously waiting for
the GPU process to launch. This occured because the compositor was
being created in nsWindow::Create, and as it requires the GPU process
to be running it had to block until launch completed. The process is
launched when the gfxPlatform is first initialized, but that was only
occuring immediately prior to creating the compositor, which did not
give it enough time to complete asynchronously.
This patch makes it so that we initialize the gfxPlatform slightly
earlier, and importantly delay creating the compositor until it is
actually required. This gives the process enough time to launch
asynchronously meaning we do not have to block.
We started deliberately creating the compositor early on Android
because of bug 1453501, to avoid a race condition where the compositor
didn't exist when RemoteLayerTreeOwner::Initialize was called, causing
us to use a basic layer manager. However, since bug 1741156 landed we
now create the compositor on-demand, meaning this is no longer a
possibility.
Delaying compositor creation can, however, uncover another race
condition. If the UICompositorControllerChild is opened on the UI
thread before the main thread is able to set its pointer to the
widget, then the java GeckoSession will never be notified that the
compositor has been opened, and composition will never be
resumed. This patch fixes this issue by setting the
UiCompositorControllerChild's widget pointer in its constructor rather
than immediately afterwards.
Differential Revision: https://phabricator.services.mozilla.com/D139842
With the later patch in this series to delay initializing the
compositor, we started crashing in ScreenshotTest#giantScreenshot due
to webrender's window size sanity check.
This check panics early if we attempt to render an area larger than
webrender can handle (rather than panicing internally in
webrender). However, this test deliberately creates a 999999x999999
sized window, to ensure that attempting to allocate a bitmap this size
for a screenshot results in an out of memory exception.
Previously this test only succeeded because we created the compositor
early with a default size of 0x0, whereas now we create it after the
widget has its very large size. Additionally, the test completes
before we have a chance to render anything, otherwise it would indeed
have crashed.
To ensure users do not hit the panic in the wild, in bug 1653649 we
added the necessary limit to the default widget size constraints,
ensuring we never create widgets that are too large. On Android,
however, we do not use the size constraints, so this had no effect.
This patch starts applying size constraints to android widgets,
meaning we do not attempt to render too large an area, and webrender
does not panic. The test still attempts to allocate a large bitmap,
and therefore still throws an out of memory exception and passes.
Differential Revision: https://phabricator.services.mozilla.com/D140050
We noticed a cold_view_nav_start regression on Fenix from enabling the
GPU process, and profiles showed time spent synchronously waiting for
the GPU process to launch. This occured because the compositor was
being created in nsWindow::Create, and as it requires the GPU process
to be running it had to block until launch completed. The process is
launched when the gfxPlatform is first initialized, but that was only
occuring immediately prior to creating the compositor, which did not
give it enough time to complete asynchronously.
This patch makes it so that we initialize the gfxPlatform slightly
earlier, and importantly delay creating the compositor until it is
actually required. This gives the process enough time to launch
asynchronously meaning we do not have to block.
We started deliberately creating the compositor early on Android
because of bug 1453501, to avoid a race condition where the compositor
didn't exist when RemoteLayerTreeOwner::Initialize was called, causing
us to use a basic layer manager. However, since bug 1741156 landed we
now create the compositor on-demand, meaning this is no longer a
possibility.
Delaying compositor creation can, however, uncover another race
condition. If the UICompositorControllerChild is opened on the UI
thread before the main thread is able to set its pointer to the
widget, then the java GeckoSession will never be notified that the
compositor has been opened, and composition will never be
resumed. This patch fixes this issue by setting the
UiCompositorControllerChild's widget pointer in its constructor rather
than immediately afterwards.
Differential Revision: https://phabricator.services.mozilla.com/D139842
If the android system kills the GPU process to free memory while the
app is in the background, then we want to avoid immediately restarting
the GPU process.
To achieve this, we make GPUProcessManager keep track of whether it is
in the foreground or background. If HandleProcessLost() gets called
while in the background then we destroy the existing compositor
sessions as before, but return early instead of immediately
relaunching the process. If the process has not been launched when the
app later gets foregrounded then we do so then.
The final part of HandleProcessLost(), which reinitializes the content
bridges and emits the "compositor-reinitialized" signal, has been
moved to a new function ReinitializeRendering(). If the GPU process
has been disabled, this gets called as-before at the end of
HandleProcessLost(). When the GPU process is enabled, however, we now
call it from OnProcessLaunchComplete(), so that it gets called
regardless of whether the process is launched immediately or after a
delay.
While we're here, rename the functions RebuildRemoteSessions() and
RebuildInProcessSessions() to DestroyRemoteCompositorSessions() and
DestroyInProcessCompositorSessions(), to better reflect what they
actually do: the "rebuilding" part occurs later on. Also update the
mega-comment documenting the restart sequence, as it was somewhat
outdated.
In case a caller of EnsureGPUReady() gets called before the foreground
signal arrives (eg in nsBaseWidget::CreateCompositorSession() due to a
refresh tick paint), make EnsureGPUReady() launch the GPU process
itself if the GPU process is enabled but not yet launched. As a
consequence, to avoid launching the GPU process unnecessarily, change
a couple callers of EnsureGPUReady() to simply check whether the
process is enabled instead.
Additionally, guard against a null pointer deref if the compositor has
been destroyed when the widget receives a memory pressure event. This
is now more likely to occur as there may be a gap between the
compositor being destroyed and recreated.
Differential Revision: https://phabricator.services.mozilla.com/D139042
Rather than using a separate function
APZCTreeManager::AddInputBlockCallback(), allow an optional callback
argument to be passed to APZInputBridge::ReceiveInputEvent().
This avoids a race condition where the input block is processed before
the callback has a chance to be added to the input queue, meaning the
callback will never be fired. With the GPU process enabled the added
latency of adding the callback cross-process meant this occured
frequently, causing intermittent junit test failures.
This works similarily to before, with the in-process APZCTreeManager
implementation simply calling InputQueue::AddInputBlockCallback() and
the InputQueue will fire the callback when the input block has been
processed. For the cross-process implementation, APZInputBridgeChild
maintains a local map of the callbacks, and APZInputBridgeParent adds
an intermediate callback to its local InputQueue. This intermediate
callback will call APZInputBridgeParent::SendCallInputBlockCallback(),
which tells the APZInputBridgeChild to fire the real callback.
Care must be taken in both APZInputBridgeChild and Parent to only add
their respective callbacks if we determine that it is required, eg not
already handled by the root APZ.
Differential Revision: https://phabricator.services.mozilla.com/D138801
In bug 1750569 we attempted to ensure that following a GPU process
crash outstanding screen pixels requests would be fulfilled. While
this usually worked there was a race condition between sending the
request to the new compositor and the content process sending the
display list to the new compositor, which meant that sometimes we
would screenshot an empty screen instead of the page content.
As a GPU process crash is an extraordinary circumstance and
screenshots are non-critical, the best solution is to simply return an
error if a GPU process crash occurs while there is an outstanding
request (or if a new request is made whilst the GPU process is
restarting). This patch also updates the junit test to check for this
error rather than expecting a screenshot to be returned.
Differential Revision: https://phabricator.services.mozilla.com/D138323
WebRender retains about 50MBs of memory for every window. When switching
between lots of tabs on Android (where 1 tab = 1 window), this can cause
problems as the app will consume a significant amount of memory.
To avoid this problem, we send a memory pressure event whenever a session is
deactivated, which signals to WebRender that it should deallocate the memory.
This message is sent on a delay of 10s to avoid interfering with tab switching,
and we cancel the message if the tab becomes active again before we fire the
memory pressure event.
Co-Authored-By: Cathy Lu <calu@mozilla.com>
Co-Authored-By: Jonathan Almeida [:jonalmeida] <jonalmeida942@gmail.com>
Differential Revision: https://phabricator.services.mozilla.com/D136965
If the GPU process crashes while a screen pixels request is
outstanding, then currently that request will never be completed. This
patch makes it so that we check for outstanding requests when the
compositor is (re)initialized, and send those requests to the new
compositor if they exist.
This includes a refactoring of how the LayerViewSupport class accesses
the UiCompositorControllerChild from the UI thread. Previously it
looked it up through the nsWindow's mCompositorSession whenever it was
required. However, since the compositor session can now be destroyed
and created we instead notify the UI thread whenever that occurs with
LayerViewSupport::NotifyCompositorSessionLost and
LayerViewSupport::NotifyCompositorCreated. The latter of these takes a
reference to the UiCompositorControllerChild, which can safely be used
by the LayerViewSupport on the UI thread.
Differential Revision: https://phabricator.services.mozilla.com/D136151
Since bug 1747116 landed, if the compositor is reinitialized whilst
the Android Surface is invalid, we avoid crashing when querying the
window size and instead keep the compositor in a paused state.
However, in this case we will believe the widget size is 0x0 until the
compositor is eventually resumed. If webrender receives a display list
during this time, it will set an empty view rect. This means when the
compositor is subsequently resumed webrender believes it has nothing
to render, and we get stuck in a state where nothing is ever rendered
to the screen.
This patch initializes the AndroidCompositorWidget with an initial
size, which avoids the problem.
Differential Revision: https://phabricator.services.mozilla.com/D135711
If AndroidCompositorWidget's surface reference points to a surface
that has been destroyed, we can end up with a null ANativeWindow
pointer. This can result in crashes when using it to query the window
size.
This patch makes it so that we use the native window to query the size
only when the surface has changed rather than for every call to
GetClientSize(). This allows us to guard against a null pointer in a
single place. If we have a null pointer then return false from
OnCompositorSurfaceChanged(). CompositorBridgeParent::ResumeComposition()
will handle that by not resuming the compositor, like it already does
if WebRenderBridgeParent::Resume() fails.
Additonally, when we receive a pause event from GeckoView ensure that
we always set the mCompositorPaused flag to true, even if the
UiCompositorController is null. This avoids a possible cause of the
situation described above - if we receive a pause event (eg the app is
minimised) during compositor reinitialization (while the
UiCompositorController is temporarily null) we would not set that flag
to true, and would therefore resume compositing in to an invalid
surface.
Depends on D135117
Differential Revision: https://phabricator.services.mozilla.com/D135118
This patch ensures that, following a GPU process crash, we
re-initialize the compositor and resume painting on Android.
nsWindow::GetWindowRenderer() is made to always reinitialize the
window renderer if there is none, like on other platforms. We
therefore no longer need to track whether webrender is being disabled,
as this is no longer a special case.
Previously we started the compositor as initially paused in
nsBaseWidget::CreateCompositorSession only if the widget did not yet
have a surface. Now we must unconditionally (re)start it as initially
paused, as even though the widget in the parent process may have a
surface, we will not have been able to send it to the GPU process
yet. We will send the surface to the compositor once control flow
returns to nsWindow::CreateLayerManager, where we will also now resume
the compositor if required.
Finally, we must ensure that we manually trigger a paint, both in the
parent and content processes. On other platforms this occurs
automatically following a GPU process loss through various refresh
driver events. On Android, however, nothing causes the refresh driver
to paint by itself, and we cannot receive input without first
initializing our APZ controllers, which does not happen until the
compositor receives a display list. We therefore must manually
schedule a paint. We do so from nsWindow::NotifyCompositorSessionLost
for the parent process, and BrowserChild::ReinitRendering for content
processes.
Differential Revision: https://phabricator.services.mozilla.com/D131232
Declare a GPU process and corresponding Service in the
AndroidManifest. This is of a new class GeckoServiceGpuProcess which
inherits from GeckoServiceChildProcess, and provides a binder
interface ICompositorSurfaceManager which allows the parent process to
set the compositor Surface for a given widget ID, and the compositor
in the GPU process to look up the Surface for a widget ID. The
ICompositorSurfaceManager interface is exposed to the parent process
through a new method getCompositorSurfaceManager() in the
IChildProcess interface.
Add a new connection type for GPU processes to GeckoProcessManager,
along with a function to look up the GPU process connection and fetch
the ICompositorSurfaceManager binder. When the GPU process is launched
we store the returned binder in the GPUProcessHost, and when each
widget's compositor is created we store a reference to the binder in
the UiCompositorControllerChild.
Each nsWindow is given a unique ID, and whenever the Surface changes
due to an Android resume event, it sends the new surface for that ID
to the GPU process (if enabled) by calling
ICompositorSurfaceManager.onSurfaceChanged().
Stop inheriting AndroidCompositorWidget from InProcessCompositorWidget
and instead inherit from CompositorWidget directly. This class holds a
reference to the Surface that will be rendered in to. The
CompositorBridgeParent notifies the CompositorWidget whenever it has
been resumed, allowing it to fetch the new Surface. For the
cross-process CompositorWidgetParent implementation it fetches that
Surface from the CompositorSurfaceManagerService, whereas the
InProcessAndroidCompositorWidget can read it directly from the real
widget.
AndroidCompositorWidget::GetClientSize() can now calculate its size
from the Surface, rather than racily reading the value from the
nsWindow. This means RenderCompositorEGL and RenderCompositorOGLSWGL
can now use GetClientSize() again rather than querying their own size
from the Surface.
With this patch, setting layers.gpu-process.enabled to true will cause
us to launch a GPU process and render from it. We do not yet
gracefully recover from a GPU process crash, nor can we render
anything using SurfaceTextures (eg video or webgl). Those will come in
future patches.
Differential Revision: https://phabricator.services.mozilla.com/D131231
