This adds telemetry to UrlbarView that records the following things related to
the zero-prefix view (i.e., the topsites view):
* Exposures: How many times the ZP view was shown
* Engagements: How many times a result was picked in the ZP view
* Abandonments: How many times the user abandoned the ZP view
* Dwell time: How long the user was shown the ZP view
I considered adding telemetry specifically for topsites instead of the ZP view
as a whole, but since we have plans to start showing other types of results in
the ZP view, I don't think it's a good idea to rely on one specific type of
result as a proxy for the view itself. What DS and Product want to know about is
the view itself: how many times it was shown, for how long, etc.
This also adds one related scalar related to weather suggestions that counts the
number of times it's shown. This is the same scalar I added in D164778 using a
different, more complex approach.
Depends on D164615
Differential Revision: https://phabricator.services.mozilla.com/D165253
This adds the following scalars:
* `impression_weather`
* `click_weather`
* `help_weather`
* `block_weather`
And these histograms:
* `FX_URLBAR_MERINO_LATENCY_WEATHER_MS`
* `FX_URLBAR_MERINO_RESPONSE_WEATHER`
The histograms are updated in addition to the existing general Merino latency
and response histograms. I also modified the existing response histogram by
adding a new `no_suggestion` category so we can tell the difference between a
successful fetch with suggestions and a successful fetch without suggestions.
There's other telemetry in https://mozilla-hub.atlassian.net/browse/SNT-333 that
this doesn't add. I didn't want to do it all here since some of it is very
different. I'll file new bugs as necessary.
Other changes this makes:
* Factor out weather initialization from test_weather.js into MerinoTestUtils so
it can also be used in the new browser_telemetry_weather.js
* Copy `updateTopSites()` from the main urlbar head.js to quicksuggest's head.js
* Add some more `info()` logging to the telemetry helpers in head.js
Differential Revision: https://phabricator.services.mozilla.com/D164615
This makes the weather suggestion blockable. It depends on the new `isBlockable`
payload property added in D163766. The suggestion's row in the view will
automatically get a block button by setting that property.
We don't have an existing browser-chrome test for the block button by itself to
make sure picking it removes the row, so I added a task to
browser_remove_match.js.
The change to test_weather.js makes sure calling `blockResult()` on the provider
correctly disables the `suggest.weather` pref.
Depends on D163766
Differential Revision: https://phabricator.services.mozilla.com/D164120
This makes a couple of large changes:
(1) "Generic" buttons (the ones added by `UrlbarView.#addRowButton()`) are now
supported in all row types. The help button that's currently included in some
types of rows when `result.payload.helpUrl` is defined is now supported for all
row types, and two additional button types are now supported too: block buttons
and labeled buttons. A row will get a block button if its
`result.payload.isBlockable` is defined. It will get a labeled button if
`result.payload.buttons` is defined and non-empty. A button can include a `url`
property that is then added as an attribute on the button's element, and
`UrlbarInput.pickResult()` will use this attribute to load the URL when the
button is picked.
(2) The reason I added labeled buttons is because it lets us support tip buttons
without much more effort, which then lets us get rid of the special row type
used for tips. With this patch, tips are now standard rows that use generic
buttons.
This approach should be compatible with the result menu, when we switch over to
it, because we can include the help and block commands in the menu when
`helpUrl` and `isBlockable` are defined, instead of creating buttons for them.
Labeled buttons -- the ones used in tips -- would still be created. The result
menu button itself can continue to be a generic button.
It should also be compatible with including the result menu button inside the
row selection. We'll still add buttons to `.urlbarView-row`, separate from
`.urlbarView-row-inner`, so that the buttons can continue to be on the right
side of the row. We can color the background of the row instead of the
row-inner.
As with D163630, my motivation for this change is to support generic buttons in
dynamic result rows so that help and block buttons can be easily added to
weather suggestions. Here too the larger changes of supporting generic labeled
buttons and removing special rows for tips aren't strictly necessary, but I took
the opportunity to rework things.
Finally, this makes a few other changes:
* It includes some of the more minor improvements to selection that I made in
D163630.
* It removes the help URL code from quick actions since it was decided not to
show a help button. Currently, the button is hidden in CSS, but now that a
generic help button is added for dynamic result rows when
`result.payload.helpUrl` is defined, `helpUrl` needs to be removed from the
payload to prevent a button from being added.
* I removed the special tip wrapping behavior, where the tip button and help
button would wrap below the tip's text. Instead, now the text wraps inside
row-inner and the buttons always remain on the same horizontal as the text. I
don't think it's worth the extra complication.
Differential Revision: https://phabricator.services.mozilla.com/D163766
This makes a couple of large changes:
(1) "Generic" buttons (the ones added by `UrlbarView.#addRowButton()`) are now
supported in all row types. The help button that's currently included in some
types of rows when `result.payload.helpUrl` is defined is now supported for all
row types, and two additional button types are now supported too: block buttons
and labeled buttons. A row will get a block button if its
`result.payload.isBlockable` is defined. It will get a labeled button if
`result.payload.buttons` is defined and non-empty. A button can include a `url`
property that is then added as an attribute on the button's element, and
`UrlbarInput.pickResult()` will use this attribute to load the URL when the
button is picked.
(2) The reason I added labeled buttons is because it lets us support tip buttons
without much more effort, which then lets us get rid of the special row type
used for tips. With this patch, tips are now standard rows that use generic
buttons.
This approach should be compatible with the result menu, when we switch over to
it, because we can include the help and block commands in the menu when
`helpUrl` and `isBlockable` are defined, instead of creating buttons for them.
Labeled buttons -- the ones used in tips -- would still be created. The result
menu button itself can continue to be a generic button.
It should also be compatible with including the result menu button inside the
row selection. We'll still add buttons to `.urlbarView-row`, separate from
`.urlbarView-row-inner`, so that the buttons can continue to be on the right
side of the row. We can color the background of the row instead of the
row-inner.
As with D163630, my motivation for this change is to support generic buttons in
dynamic result rows so that help and block buttons can be easily added to
weather suggestions. Here too the larger changes of supporting generic labeled
buttons and removing special rows for tips aren't strictly necessary, but I took
the opportunity to rework things.
Finally, this makes a few other changes:
* It includes some of the more minor improvements to selection that I made in
D163630.
* It removes the help URL code from quick actions since it was decided not to
show a help button. Currently, the button is hidden in CSS, but now that a
generic help button is added for dynamic result rows when
`result.payload.helpUrl` is defined, `helpUrl` needs to be removed from the
payload to prevent a button from being added.
* I removed the special tip wrapping behavior, where the tip button and help
button would wrap below the tip's text. Instead, now the text wraps inside
row-inner and the buttons always remain on the same horizontal as the text. I
don't think it's worth the extra complication.
Differential Revision: https://phabricator.services.mozilla.com/D163766
This makes a couple of large changes:
(1) "Generic" buttons (the ones added by `UrlbarView.#addRowButton()`) are now
supported in all row types. The help button that's currently included in some
types of rows when `result.payload.helpUrl` is defined is now supported for all
row types, and two additional button types are now supported too: block buttons
and labeled buttons. A row will get a block button if its
`result.payload.isBlockable` is defined. It will get a labeled button if
`result.payload.buttons` is defined and non-empty. A button can include a `url`
property that is then added as an attribute on the button's element, and
`UrlbarInput.pickResult()` will use this attribute to load the URL when the
button is picked.
(2) The reason I added labeled buttons is because it lets us support tip buttons
without much more effort, which then lets us get rid of the special row type
used for tips. With this patch, tips are now standard rows that use generic
buttons.
This approach should be compatible with the result menu, when we switch over to
it, because we can include the help and block commands in the menu when
`helpUrl` and `isBlockable` are defined, instead of creating buttons for them.
Labeled buttons -- the ones used in tips -- would still be created. The result
menu button itself can continue to be a generic button.
It should also be compatible with including the result menu button inside the
row selection. We'll still add buttons to `.urlbarView-row`, separate from
`.urlbarView-row-inner`, so that the buttons can continue to be on the right
side of the row. We can color the background of the row instead of the
row-inner.
As with D163630, my motivation for this change is to support generic buttons in
dynamic result rows so that help and block buttons can be easily added to
weather suggestions. Here too the larger changes of supporting generic labeled
buttons and removing special rows for tips aren't strictly necessary, but I took
the opportunity to rework things.
Finally, this makes a few other changes:
* It includes some of the more minor improvements to selection that I made in
D163630.
* It removes the help URL code from quick actions since it was decided not to
show a help button. Currently, the button is hidden in CSS, but now that a
generic help button is added for dynamic result rows when
`result.payload.helpUrl` is defined, `helpUrl` needs to be removed from the
payload to prevent a button from being added.
* I removed the special tip wrapping behavior, where the tip button and help
button would wrap below the tip's text. Instead, now the text wraps inside
row-inner and the buttons always remain on the same horizontal as the text. I
don't think it's worth the extra complication.
Differential Revision: https://phabricator.services.mozilla.com/D163766
This adds a weather feature to quick suggest. It periodically fetches a weather
suggestion from Merino. UrlbarProviderQuickSuggest shows the suggestion when the
search string is empty ("zero prefix").
The implementation of the UrlbarResult returned by UrlbarProviderQuickSuggest is
only temporary. Mandy is working on the final UI in
[SNT-323](https://mozilla-hub.atlassian.net/browse/SNT-323). Landing a temporary
implementation allows Mandy to base her patch on it and trigger real weather
suggestions from Merino to test her implementation against. It also lets people
on the team test weather suggestions without having to wait for the final UI to
land.
I added the following prefs and Nimbus variable to control the feature. It will
initially be rolled out in an experiment, so we need a Nimbus variable. In the
initial experiment, users will be able to dismiss the suggestion but not toggle
a checkbox in about:preferences.
* `weatherFeatureGate` - Nimbus variable that controls the whole feature
* `browser.urlbar.weather.featureGate` - Fallback pref for the Nimbus variable
* `browser.urlbar.suggest.weather` - When the feature gate pref is true, this
determines whether the suggestion should be shown. In a future patch, we'll
flip this to false when users dismiss the suggestion.
I set the fetch interval to 30 minutes. That seems reasonable considering that
the suggestion contains the current temperature and weather. Merino will set
caching headers appropriately so that Firefox won't actually make a new network
request if the previously fetched suggestion is new enough.
Depends on D161368
Differential Revision: https://phabricator.services.mozilla.com/D161410
This is a follow up to D161866 and effectively reverts and replaces it with a
different approach. Please see bug 1800810 for background. In short, engagement
telemetry should be based on the result that's visible in the view, not on the
result the provider added last.
D161866 fixed the case where the last-added result is in the view but hidden.
There's another case we need to handle, when the last-added result is not in the
view at all. That can happen when you type something and hit enter really
quickly, before the view can update. I was able to trigger it several times.
When that happens, there are two possibilities:
* No quick suggest result is visible in the view. `result.rowIndex` is its
initial value of -1, so we record an engagement for a result that is not
visible and that doesn't have a valid index.
* A quick suggest result from a previous query is visible in the view. Here
again, `result.rowIndex` is -1 so we record an engagement for a result that is
not visible and that doesn't have a valid index, and we miss recording an
engagement for the result that's actually visible.
Right now it's not easy to fix this inside the provider because providers don't
know anything about the view(s). I could record this telemetry in the view but
that's not its role. I think it makes sense to include the view in the query
context, so that's what this does. I added `view.visibleResults` to make getting
the visible results easy.
Daisuke's new Glean telemetry in D160193 uses `context.results`, which has this
same problem of not being based on actually visible results. We'll need to use
`context.view.visibleResults` there too, and there may be other existing cases
as well.
Depends on D161866
Differential Revision: https://phabricator.services.mozilla.com/D162182
This adds a weather feature to quick suggest. It periodically fetches a weather
suggestion from Merino. UrlbarProviderQuickSuggest shows the suggestion when the
search string is empty ("zero prefix").
The implementation of the UrlbarResult returned by UrlbarProviderQuickSuggest is
only temporary. Mandy is working on the final UI in
[SNT-323](https://mozilla-hub.atlassian.net/browse/SNT-323). Landing a temporary
implementation allows Mandy to base her patch on it and trigger real weather
suggestions from Merino to test her implementation against. It also lets people
on the team test weather suggestions without having to wait for the final UI to
land.
I added the following prefs and Nimbus variable to control the feature. It will
initially be rolled out in an experiment, so we need a Nimbus variable. In the
initial experiment, users will be able to dismiss the suggestion but not toggle
a checkbox in about:preferences.
* `weatherFeatureGate` - Nimbus variable that controls the whole feature
* `browser.urlbar.weather.featureGate` - Fallback pref for the Nimbus variable
* `browser.urlbar.suggest.weather` - When the feature gate pref is true, this
determines whether the suggestion should be shown. In a future patch, we'll
flip this to false when users dismiss the suggestion.
I set the fetch interval to 30 minutes. That seems reasonable considering that
the suggestion contains the current temperature and weather. Merino will set
caching headers appropriately so that Firefox won't actually make a new network
request if the previously fetched suggestion is new enough.
Depends on D161368
Differential Revision: https://phabricator.services.mozilla.com/D161410
This adds `result.isVisible`. Like `rowIndex`, it's updated by the view.
UrlbarProviderQuickSuggest will skip impression telemetry updates (and
impression stats updates too) when it's false.
Please see bug 1800184 for background.
Differential Revision: https://phabricator.services.mozilla.com/D161866
This adds a new `BaseFeature` class that quick suggest features can extend.
`BaseFeature` encapsulates the logic for deciding whether a feature should be
enabled. Subclasses override a few methods and getters to describe their own
logic and to do initialization and uninitialization.
Any time a Nimbus variable (or a variable fallback pref) changes, QuickSuggest
iterates over each feature and enables or disables it appropriately.
As a first step, I ported impression caps to `BaseFeature`. Later patches in
this stack will port other features.
Please see bug 1799264 for details.
Differential Revision: https://phabricator.services.mozilla.com/D161366
This does the following:
* Get rid of `QUICK_SUGGEST_SOURCE` since it's only used in a couple of
places. Its simpler to use the string literals directly.
* Set `source: "merino"` on Merino suggesions in the Merino client instead of
doing it in UrlbarProviderQuickSuggest, similar to how the remote settings
client sets `source: "remote-settings"`
* Export `ONBOARDING_CHOICE` and `ONBOARDING_URI` on the QuickSuggest object for
consistency with other consts
* Remove unnecessary consts from QuickSuggestTestUtils that are already defined
on QuickSuggest
Please see bug 1798595 for details.
Depends on D160986
Differential Revision: https://phabricator.services.mozilla.com/D160987
This does the following:
* Moves quick suggest initialization from UrlbarQuickSuggest to QuickSuggest
* Renames UrlbarQuickSuggest.sys.mjs to QuickSuggestRemoteSettingsClient.sys.mjs, so now this file is focused only on remote settings
* Makes QuickSuggest create an instance of QuickSuggestRemoteSettingsClient and keep it as `QuickSuggest.remoteSettings`
* Moves latency telemetry from UrlbarProviderQuickSuggest into QuickSuggestRemoteSettingsClient
* Changes the ad hoc logger used in QuickSuggestRemoteSettingsClient to a proper urlbar-style logger
* Updates consumers to use `QuickSuggest.remoteSettings` instead of UrlbarQuickSuggest
Please see bug 1798595 for details.
Depends on D160985
Differential Revision: https://phabricator.services.mozilla.com/D160986
This moves the help URL const out of UrlbarProviderQuickSuggest in order to make
UrlbarProviderQuickSuggest focused on being a urlbar provider.
It also changes how QuickSuggestTestUtils is initialized to be more similar to
MerinoTestUtils. There's no need for a separate `init()` method when the
constructor will do. I'd like to make this change in a different revision but I
did it here because I want to include all changes to about:preferences in one
revision for easier review.
Please see bug 1798595 for details.
Depends on D160984
Differential Revision: https://phabricator.services.mozilla.com/D160985
This moves Nimbus exposure event recording out of UrlbarQuickSuggest in
preparation for making UrlbarQuickSuggest strictly related to remote settings.
Please see bug 1798595 for details.
Depends on D160982
Differential Revision: https://phabricator.services.mozilla.com/D160983
This does the following:
* Copies UrlbarProviderQuickSuggest.sys.mjs to a new QuickSuggest.sys.mjs file
* Removes everything related to UrlbarProvider from QuickSuggest
* Removes most everything not related to UrlbarProvider from
UrlbarProviderQuickSuggest
* Updates consumers to use the new QuickSuggest module
Please see bug 1798595 for details.
Differential Revision: https://phabricator.services.mozilla.com/D160982
This factors out the Merino client code from UrlbarProviderQuickSuggest into
MerinoClient. It also adds MerinoTestUtils.
There are a few reasons for this:
1. Weather suggestions will be prefetched from Merino, and it may make sense to
do the prefetching outside of UrlbarProviderQuickSuggest, I'm not sure
yet.
2. Weather suggestions will be fetched from a slightly different URL from other
suggestions. They'll need to include `providers=weather` as a search param.
In the future there may be other types of suggestions that also require
different parameters or whole new endpoints. Supporting options like this is
a little nicer with a dedicated Merino client class.
3. In general this code is substantial enough that it makes sense to encapsulate
it in its own file and class, even if the provider is the only thing that
will use it.
This is a large patch but a lot of it moves code around. Summary of changes:
* Factor out Merino client code into MerinoClient
* Also include some new useful helpers in MerinoClient
* Consolidate and factor out Merino test code into MerinoTestUtils
* Also include some new useful helpers in MerinoTestUtils and make other improvements
* Add a MockMerinoServer class (in MerinoTestUtils.sys.mjs) to help with setting up a mock Merino server
* Add test_merinoClient.js to test the client in isolation. This file started as a copy of test_quicksuggest_merino.js because it's so similar, and some of the tests in the latter are moved to the former.
* Add test_merinoClient_sessions.js to test Merino sessions in isolation. Similar to the previous point, this file started as a copy of test_quicksuggest_merinoSessions.js.
* Modify existing Merino tests so they use MerinoTestUtils
Differential Revision: https://phabricator.services.mozilla.com/D160449
This adds a Nimbus variable corresponding to `browser.urlbar.merino.providers`.
I noticed that when we added a Nimbus variable for client variants (bug
1743685), we didn't actually use it in the provider. Instead we use its fallback
pref. I fixed that too.
Differential Revision: https://phabricator.services.mozilla.com/D159557
On second thought, I don't think we should use `syncIfEmpty: false`. I played
around with it more today, and it can take more than five minutes for a sync to
happen after startup on a new profile. If you're using the urlbar during that
time and expecting Suggest suggestions, it's frustrating, and it's probably not
what we or our partners want. The delay is just too big I think, and there
doesn't seem to be a hard guarantee about when it will happen, although I might
be wrong about that.
Instead I moved quick suggest initialization to a startup idle task in
BrowserGlue. That will allow startup and session store to finish before we try
to sync while still performing sync in a timely manner. There may be a better
compromise, like implementing a guarantee that sync will happen no later than
one minute after startup, and/or after the user starts using the urlbar, but
this at least slightly improves on the status quo before bug 1773777.
I haven't pushed this to try yet, so I might have regressed the main intention
behind bug 1773777, preventing unrelated tests from triggering failing syncs for
the collection. The test I mentioned in that bug does pass for me locally
though.
Currently we use `Services.tm.idleDispatchToMainThread()` to do the inital setup
of the RS client, but I removed that since it's now redundant, now that we're
using a startup idle task.
Differential Revision: https://phabricator.services.mozilla.com/D154036
