In feature-session we had some logic to filter out certain URIs. This
patch moves this logic into the storage layer, so that it has a broader
coverage - whenever we attempt to write some URL into the storage, we'll
now go through this filter.
Before, we'd only go through this filter for writes that were routed via
the HistoryDelegate.
After this change, direct writes (such as for metadata) are also
covered.
This shouldn't change all that much - the storage implementation is
likely to have rejected these URIs. Now, instead of letting it through
exceptions at us (which we catch and report via sentry), we simply don't
even call into the storage if we don't expect these calls to succeed.
This should reduce volume of "url without base" type of 'info' events we
see in Sentry.
Most of the time redirectSource isn't set, so this makes that field
optional, and switches away from a special enum (NOT_A_SOURCE) to just a
simple nullable.
Not using a write scope means we could have concurrent writes, leading
to sqlite throwing "database locked" exceptions.
All other writers here use the writeScope.
* Add Places calls for recently added bookmarks.
Changelog addition for bookmarks places api additions.
* Test for fetching list of recently added bookmarks
* Reword kdoc and changelog
Running ./gradlew ktlint locally reports a lot of stuff like:
/home/emilio/src/moz/android-components/components/support/migration/src/test/java/mozilla/components/support/migration/MigrationIntentProcessorTest.kt:1:1: File must end with a newline (\n)
This fixes it with:
for f in $(cat files | cut -d : -f 1); do echo "" >> $f; done
Making ./gradlew ktlint pass.
This adds a new Nimbus component that will act as a wrapper around the uniffi generated Kotlin code from mozilla/nimbus-sdk, as well as be a point of Glean integration, at least initially.
- Integrate service-nimbus with samples-glean for testing
- Set up default endpoint for debug and release, debug pointing at the dev Kinto endpoint and release at the production endpoint.
- Adds the documentation on how to set up the Kinto dev endpoint.
- Updates samples-glean README with Nimbus wording
Revert "Address even more lint errors."
This reverts commit e98ee991b34d318e671c644d63daec53902e68ff.
Revert "Update to Gradle 6.6.1."
This reverts commit cdda701c8f2be6c0486ddf1494885accf78aab00.
Revert "Address lint errors."
This reverts commit 26cb5b27bc2be672c89e7e0c63eeeda9f64c0d08.
Revert "Move JNA configuration to root project."
This reverts commit 694ef37cc30e727f3ec5d275c0e9583385980a43.
Revert "Issue https://github.com/mozilla-mobile/android-components/issues/8319: Update to Kotlin 1.4, Coroutines 1.3.9 and Android lint 27.0.1."
This reverts commit ad675c35e794050dbf8514a2d2fe7506ce0c546c.
Revert "Update Glean to 33.0.4."
This reverts commit 4731c4e2fa52d40f4062a47f02078ca0c85de632.
Revert "Update Glean to 33.0.0."
This reverts commit 352cf88a4d4bad64330d0ec997814fd8b13928c2.
Revert "Update Gradle to 6.1.1."
This reverts commit 18e88aa1a2528b10f7d81ff9dbf1df094071b6ce.
Revert "Update Android Gradle Plugin to 4.0.1."
This reverts commit f915a3b47b0f5584889ff62a80c9642bccfa36ce.
This allows consumers to ensure that underlying database connections are fully established.
After `warmUp` is called, there should be no additional overhead upon first access to the storage.
The tabs engine `sync` method signature needs to be different than
the one currently defined in `SyncableStore.sync`.
`StorageSync` is the only user of the interface method and is deprecated
already.
This patch removes the `StorageSync` class and makes the store `sync`
methods not inherited from any interface.
There's some confusion in `GeckoEngineSession` about redirect flags.
The `VISIT_REDIRECT_SOURCE` and `VISIT_REDIRECT_SOURCE_PERMANENT` flags
that we get from GeckoView's history delegate are for the redirect
_source_, not the visit type. They indicate if the URL passed to
`onVisited` is redirecting _to_ another URL, most likely because the
server returned an HTTP 3xy status code with a `Location` header.
Rust Places decides whether to mark the URL as hidden based on
these flags.
`VISIT_REDIRECT_{PERMANENT, TEMPORARY}`, however, indicate if the
URL passed to `onVisited` is the _target_ of a redirect (in other
words, the page that's _in_ the `Location` header). These get
translated into `VisitType` flags, which Rust Places stores as the
visit transition type. These two flags don't affect whether a URL
is hidden.
Note that, in a redirect chain, the middle links are both sources and
targets. For example, in "mozilla.org" -> "www.mozilla.org" ->
"www.mozilla.org/en-US", "www.mozilla.org" is both a redirect target
(since "mozilla.org" redirected to it), and a source (it redirected
to "www.mozilla.org/en-US").
See https://github.com/mozilla-mobile/fenix/issues/3526.
This allows us to use the same Glean definitions and processing logic
from multiple components. In our case, it'll be browser-storage-sync
(for non-AccountManager syncing) and services-firefox-accounts (SyncManager integration)
We will need to record sync telemetry from the sync manager. We already
have telemetry related code, but at the DB Connection level.
This patch refactors this code into a singleton object that may be
used from within any context in which there's a ping object available that
needs to be processed.
The docs are written, at build-time, to
'$project/docs/metrics.md'. This additionally
enables automatic doc generation for the following
components: lib-crash, storage-sync.
This includes exposing a new component for configuring the HTTP stack,
as to avoid forcing dependents who have no custom megazord to keep a-s
and a-c versions in sync.
This is required for the CWTS work, and has been coming for a while.
It's a breaking change for users of FxA/Sync/Push who don't megazord
(they must to init HTTP now, and should init rustlog as well). It's also
a breaking change for megazord users, but I'll file PRs for them
individually.
This includes other changes required to update a-s, in logins and fxa
specifically.
I've started pulling on one little thread, and ended up with a few more changes than initially anticipated.
Raison d'être for this PR - introducing access token caching for Sync.
- Some background on the issue: Rust FirefoxAccount object maintains an in-memory cache of access tokens, keyed by 'scope'. During every sync, we "rehydrate" an instance of FirefoxAccount, starting with a fresh cache. We then obtain an access token from it to sync; this performs a network request (since the internal cache is empty), which is quite costly at scale for our services. This creates a situation when we may overwhelm our own servers with a large enough, actively syncing user base.
- This PR adds a caching layer for sync authInfo objects. Sync workers no longer interact with the account directly, and instead look into the cache to obtain authentication info necessary for syncing. No more "talk to the FxA server before every sync".
Account manager is responsible for keeping the cache up-to-date, and resetting it when necessary. Cache is currently updated: on startup (but only if access token has expired), on authentication, and when we recover from auth problems.
And this is where the "thread pulling" begins! In order to "own" the access token for sync, account manager needs to be aware of the "sync scope".
Before, we just relied on the application to specify that scope. Instead, I've changed account manager's constructor to take a SyncConfig object which allows consuming application to configure how sync should behave (enabled at all?, periodic syncing enabled? how often to sync? which stores should be synced?).
Ownership of the "sync manager" moved down the stack, from the application layer into the account manager.
Application is now expected to interact with sync only via AccountManager's `sync` method, which exposes an internal SyncManager instance (if sync is enabled).
Above changes were a good reason to move support classes from feature-sync and into services-firefox-account. Note that since "sync" is part of our "storage" modules, this change doesn't mean that you need to take an extra native dependency on your classpath simply if you need to use FxA. Thanks to concept-sync, actual "Firefox Sync" machinery (within libplaces) is still fully decoupled from FxA. `feature-sync` has been removed entirely.
Since we're churning the public API anyway, I took the chance to introduce a few more simplifications at the API layer:
- 'SyncManager' interface was removed, since we're not expecting to have multiple implementations of it
- 'Config' was renamed to 'ServerConfig'
- 'DeviceTuple' was renamed to 'DeviceConfig'
- account manager grew a new public API, 'setSyncConfig', which allows application to re-configure how it wants sync to behave
- 'AuthInfo' was renamed to 'SyncAuthInfo', and a bunch of cleanup happened in that area
- 'AccountObservable'@'onError' method was removed. The only error that could have been passed into it (unable to restore account) wasn't actionable by the application anyway, and none of the integrations did anything with that call
Documentation of public APIs and classes was improved.
This commit adds two Glean pings to the Storage-Sync browser component:
one for history, and one for bookmarks. Both pings record the same set
of base metrics, including incoming and outgoing counts, sync duration,
the hashed FxA UID, and, most importantly, the failure reason if the
store fails to sync. The bookmarks ping records additional validation
data.
The Glean schema is a flattened version of the Sync ping that's
currently sent on Firefox Desktop and Firefox for iOS. Instead of
sending a single ping with multiple syncs, each having multiple
engines, we send one Glean ping per engine per sync.
The pings are recorded directly in the component, so any app that
consumes it should get pings for free.
This is the first time we're exercising the Rust appservices stack
from our unit tests.
New tests are a combination of old browser-storage-sync tests, and tests
from browser-storage-memory, adjusted for the slightly different
implementation.
At this point in the stack, we're not in control over what our
autocomplete providers are, what actions they'll do in order to
field our queries, etc. For example, some providers may hit the disk
and perform expensive DB queries internally. Some may even hit the
network, in theory!
In order to keep things perceptively speedy, let's run the actual work
off the main thread. This patch sets up a new pool thread to process
autocomplete requests. More than one thread is selected so that we maintain
liveliness during quick user input. Background tasks are cancelled as new
queries come in, and stale results are discarded.
- Remove a few unnecessary dependencies
- Add some comments around usage of 'api'
- Mark some of the dependencies as 'api' if they're necessary for use of the module
I think this is just vestigial: as things developed, we expected to
use JNA directly. Now all such access is an implementation detail of
the Application Services delivery vehicle.
The Proguard rules have been pushed to the application-services
repository, where they belong.
