With extended attribute attribution using exactly the same attribution strings as Windows (eg: no URLs), this turns out to be quite straightforward. Once we've pulled in the attribution string from the extended attribute, the existing parsing, validation, etc. just works.
The only wrinkle is that the extended attributes may have nul bytes or tabs that we need to strip away. (Tabs may be present because we use them to pad the attribution area when we prepare the DMG for attribution. Nul bytes may be present because we overwrite the entire attribution before updating the attribution data.)
Differential Revision: https://phabricator.services.mozilla.com/D189258
We can't add the provenance data to the `installation.first_seen` extra data because it is already at its maximum number of keys. So instead we will add the `installation.first_seen_prov_ext` event which will be sent at the same time as `installation.first_seen` and will contain provenance attribution data in its extras object.
Differential Revision: https://phabricator.services.mozilla.com/D172520
Add a function that sets scalars to convey what kind of data is present in the :Zone.Identifier Alternate Data Stream. Also adds testing that the scalars get set correctly.
Does not actually call the function that sends the telemetry. This will be done in the next patch since we need to call it in some slightly different places than we usually would since we want to be sure it is included in the `new-profile` ping and in the ping with the `installation.first_seen` event.
Differential Revision: https://phabricator.services.mozilla.com/D171818
Two notable things here:
- Small refactor of AttributionCode.jsm to make it more testable (allowing `getCampaignId` to be easily stubbed out)
- An ugly skip of one of the "invalid" attribution code tests -- for reasons noted in the comments. Ultimately, this points to maybe a refactor or larger change being needed, but I don't think it's worth blocking getting these tests on.
Differential Revision: https://phabricator.services.mozilla.com/D166688
Nearly all the complexity here is abstracted away in the previous patch, so this is quite straightforward other than the one oddity noted in the comments.
Differential Revision: https://phabricator.services.mozilla.com/D143112
Querying the macOS system quarantine database is relatively slow, so
we'd like to do it only once. This is relevant because the Telemetry
subsystem causes the attribution data to be fetched relatively early
during startup. By caching, we accept additional disk activity but
don't have to invoke an external process, query a database, etc.
We must make BROWSER_ATTRIBUTION_ERRORS histogram apply to macOS as
well as Windows. We add error codes to capture macOS-specific
detalis. And we push this out to a later cycle, since there's no
reason to revisit this immediately.
Differential Revision: https://phabricator.services.mozilla.com/D92695
On Windows, we parse the attribution data as if it was the
`URL.search` parameter. On macOS, we parse a URL slightly
differently: we parse it like `URL.searchParams` and we deduplicate
parameters.
For caching on macOS we need to serialize the parsed data. We could
use either approach but we elect to write the code serialized as it
would be on Windows, since that is the one that is getting actively
tested in the wild.
Differential Revision: https://phabricator.services.mozilla.com/D92694
The next step is to have the cached attribution file location vary
depending on the platform, and to have the tests run on additional
platforms.
Differential Revision: https://phabricator.services.mozilla.com/D84264
The issue this is addresses is that
[`CFURLCopyResourcePropertyForKey`](https://searchfox.org/mozilla-central/rev/ea7f70dac1c5fd18400f6d2a92679777d4b21492/xpcom/io/CocoaFileUtils.mm#212)
does not return quarantine data when launched as a GUI App. What
happens is that launching via the GUI requires the user to override
GateKeeper by going to Security & Privacy > Open Anyway. Doing that
updates the GateKeeper flags, and then the macOS API denies access:
once the GK flags reach some state, quarantine information is not
returned. This is not documented (as far as I can see) but moons ago,
[somebody else on the internet witnessed the same
thing](https://cocoa-dev.apple.narkive.com/kkYeAC8o/is-it-possible-to-read-your-own-quarantine-info-after-launch).
To work around, we run the system SQLite binary, to fish the relevant
information out of the per-user quarantine database. (SQLite is
installed by default on all relevant macOS versions.)
The most significant security concern I see is whether we can trust
this binary (in /usr/bin/sqlite3). Some discussion within the
Install/Update team suggested that an attacker who could corrupt or
modify that binary already had write access to the disk, which is an
attack vector equal to a totally compromised Firefox. If we determine
that we can't use the system SQLite binary, then we could use
Firefox's compiled copy of SQLite, but we might see versioning
issues. The system SQLite binary feels more robust.
This is implemented as a JS component for convenience, mostly: there
is no API for capturing output from `nsIProcess`. It would be
possible to maintain the existing XPCOM contract by renaming the
existing contract and adding a contract with a JS implementation that
passes through to the renamed implementation, but it doesn't seem
worth the effort.
In the next commits, we will generalize the existing caching mechanism
form Windows to also apply to macOS. This is mostly a performance
optimization, so that we sniff a single well-known location rather
than launching a process at each startup, although there is a
correctness argument here as well, since the quarantine database is
dynamic and the attribution URL could expire.
Differential Revision: https://phabricator.services.mozilla.com/D92693
The existing code handles Windows and macOS quite differently. On
macOS, the tests were not as comprehensive; this patch brings them
level and makes the form of the attribution data uniform on Windows
and macOS. In particular, attribution data fields will now be URI
coded, just as they are on Windows.
This will allow us to re-use the parsing machinery we have on Windows
when we cache attribution codes on macOS.
Differential Revision: https://phabricator.services.mozilla.com/D92692
Querying the macOS system quarantine database is relatively slow, so
we'd like to do it only once. This is relevant because the Telemetry
subsystem causes the attribution data to be fetched relatively early
during startup. By caching, we accept additional disk activity but
don't have to invoke an external process, query a database, etc.
We must make BROWSER_ATTRIBUTION_ERRORS histogram apply to macOS as
well as Windows. We add error codes to capture macOS-specific
detalis. And we push this out to a later cycle, since there's no
reason to revisit this immediately.
Differential Revision: https://phabricator.services.mozilla.com/D92695
On Windows, we parse the attribution data as if it was the
`URL.search` parameter. On macOS, we parse a URL slightly
differently: we parse it like `URL.searchParams` and we deduplicate
parameters.
For caching on macOS we need to serialize the parsed data. We could
use either approach but we elect to write the code serialized as it
would be on Windows, since that is the one that is getting actively
tested in the wild.
Differential Revision: https://phabricator.services.mozilla.com/D92694
The next step is to have the cached attribution file location vary
depending on the platform, and to have the tests run on additional
platforms.
Differential Revision: https://phabricator.services.mozilla.com/D84264
The issue this is addresses is that
[`CFURLCopyResourcePropertyForKey`](https://searchfox.org/mozilla-central/rev/ea7f70dac1c5fd18400f6d2a92679777d4b21492/xpcom/io/CocoaFileUtils.mm#212)
does not return quarantine data when launched as a GUI App. What
happens is that launching via the GUI requires the user to override
GateKeeper by going to Security & Privacy > Open Anyway. Doing that
updates the GateKeeper flags, and then the macOS API denies access:
once the GK flags reach some state, quarantine information is not
returned. This is not documented (as far as I can see) but moons ago,
[somebody else on the internet witnessed the same
thing](https://cocoa-dev.apple.narkive.com/kkYeAC8o/is-it-possible-to-read-your-own-quarantine-info-after-launch).
To work around, we run the system SQLite binary, to fish the relevant
information out of the per-user quarantine database. (SQLite is
installed by default on all relevant macOS versions.)
The most significant security concern I see is whether we can trust
this binary (in /usr/bin/sqlite3). Some discussion within the
Install/Update team suggested that an attacker who could corrupt or
modify that binary already had write access to the disk, which is an
attack vector equal to a totally compromised Firefox. If we determine
that we can't use the system SQLite binary, then we could use
Firefox's compiled copy of SQLite, but we might see versioning
issues. The system SQLite binary feels more robust.
This is implemented as a JS component for convenience, mostly: there
is no API for capturing output from `nsIProcess`. It would be
possible to maintain the existing XPCOM contract by renaming the
existing contract and adding a contract with a JS implementation that
passes through to the renamed implementation, but it doesn't seem
worth the effort.
In the next commits, we will generalize the existing caching mechanism
form Windows to also apply to macOS. This is mostly a performance
optimization, so that we sniff a single well-known location rather
than launching a process at each startup, although there is a
correctness argument here as well, since the quarantine database is
dynamic and the attribution URL could expire.
Differential Revision: https://phabricator.services.mozilla.com/D92693
The existing code handles Windows and macOS quite differently. On
macOS, the tests were not as comprehensive; this patch brings them
level and makes the form of the attribution data uniform on Windows
and macOS. In particular, attribution data fields will now be URI
coded, just as they are on Windows.
This will allow us to re-use the parsing machinery we have on Windows
when we cache attribution codes on macOS.
Differential Revision: https://phabricator.services.mozilla.com/D92692
Querying the macOS system quarantine database is relatively slow, so
we'd like to do it only once. This is relevant because the Telemetry
subsystem causes the attribution data to be fetched relatively early
during startup. By caching, we accept additional disk activity but
don't have to invoke an external process, query a database, etc.
We must make BROWSER_ATTRIBUTION_ERRORS histogram apply to macOS as
well as Windows. We add error codes to capture macOS-specific
detalis. And we push this out to a later cycle, since there's no
reason to revisit this immediately.
Differential Revision: https://phabricator.services.mozilla.com/D92695
On Windows, we parse the attribution data as if it was the
`URL.search` parameter. On macOS, we parse a URL slightly
differently: we parse it like `URL.searchParams` and we deduplicate
parameters.
For caching on macOS we need to serialize the parsed data. We could
use either approach but we elect to write the code serialized as it
would be on Windows, since that is the one that is getting actively
tested in the wild.
Differential Revision: https://phabricator.services.mozilla.com/D92694
The next step is to have the cached attribution file location vary
depending on the platform, and to have the tests run on additional
platforms.
Differential Revision: https://phabricator.services.mozilla.com/D84264
The issue this is addresses is that
[`CFURLCopyResourcePropertyForKey`](https://searchfox.org/mozilla-central/rev/ea7f70dac1c5fd18400f6d2a92679777d4b21492/xpcom/io/CocoaFileUtils.mm#212)
does not return quarantine data when launched as a GUI App. What
happens is that launching via the GUI requires the user to override
GateKeeper by going to Security & Privacy > Open Anyway. Doing that
updates the GateKeeper flags, and then the macOS API denies access:
once the GK flags reach some state, quarantine information is not
returned. This is not documented (as far as I can see) but moons ago,
[somebody else on the internet witnessed the same
thing](https://cocoa-dev.apple.narkive.com/kkYeAC8o/is-it-possible-to-read-your-own-quarantine-info-after-launch).
To work around, we run the system SQLite binary, to fish the relevant
information out of the per-user quarantine database. (SQLite is
installed by default on all relevant macOS versions.)
The most significant security concern I see is whether we can trust
this binary (in /usr/bin/sqlite3). Some discussion within the
Install/Update team suggested that an attacker who could corrupt or
modify that binary already had write access to the disk, which is an
attack vector equal to a totally compromised Firefox. If we determine
that we can't use the system SQLite binary, then we could use
Firefox's compiled copy of SQLite, but we might see versioning
issues. The system SQLite binary feels more robust.
This is implemented as a JS component for convenience, mostly: there
is no API for capturing output from `nsIProcess`. It would be
possible to maintain the existing XPCOM contract by renaming the
existing contract and adding a contract with a JS implementation that
passes through to the renamed implementation, but it doesn't seem
worth the effort.
In the next commits, we will generalize the existing caching mechanism
form Windows to also apply to macOS. This is mostly a performance
optimization, so that we sniff a single well-known location rather
than launching a process at each startup, although there is a
correctness argument here as well, since the quarantine database is
dynamic and the attribution URL could expire.
Differential Revision: https://phabricator.services.mozilla.com/D92693
The existing code handles Windows and macOS quite differently. On
macOS, the tests were not as comprehensive; this patch brings them
level and makes the form of the attribution data uniform on Windows
and macOS. In particular, attribution data fields will now be URI
coded, just as they are on Windows.
This will allow us to re-use the parsing machinery we have on Windows
when we cache attribution codes on macOS.
Differential Revision: https://phabricator.services.mozilla.com/D92692
Querying the macOS system quarantine database is relatively slow, so
we'd like to do it only once. This is relevant because the Telemetry
subsystem causes the attribution data to be fetched relatively early
during startup. By caching, we accept additional disk activity but
don't have to invoke an external process, query a database, etc.
We must make BROWSER_ATTRIBUTION_ERRORS histogram apply to macOS as
well as Windows. We add error codes to capture macOS-specific
detalis. And we push this out to a later cycle, since there's no
reason to revisit this immediately.
Differential Revision: https://phabricator.services.mozilla.com/D92695
