Querying the macOS system quarantine database is relatively slow, so
we'd like to do it only once. This is relevant because the Telemetry
subsystem causes the attribution data to be fetched relatively early
during startup. By caching, we accept additional disk activity but
don't have to invoke an external process, query a database, etc.
We must make BROWSER_ATTRIBUTION_ERRORS histogram apply to macOS as
well as Windows. We add error codes to capture macOS-specific
detalis. And we push this out to a later cycle, since there's no
reason to revisit this immediately.
Differential Revision: https://phabricator.services.mozilla.com/D92695
On Windows, we parse the attribution data as if it was the
`URL.search` parameter. On macOS, we parse a URL slightly
differently: we parse it like `URL.searchParams` and we deduplicate
parameters.
For caching on macOS we need to serialize the parsed data. We could
use either approach but we elect to write the code serialized as it
would be on Windows, since that is the one that is getting actively
tested in the wild.
Differential Revision: https://phabricator.services.mozilla.com/D92694
The next step is to have the cached attribution file location vary
depending on the platform, and to have the tests run on additional
platforms.
Differential Revision: https://phabricator.services.mozilla.com/D84264
The issue this is addresses is that
[`CFURLCopyResourcePropertyForKey`](https://searchfox.org/mozilla-central/rev/ea7f70dac1c5fd18400f6d2a92679777d4b21492/xpcom/io/CocoaFileUtils.mm#212)
does not return quarantine data when launched as a GUI App. What
happens is that launching via the GUI requires the user to override
GateKeeper by going to Security & Privacy > Open Anyway. Doing that
updates the GateKeeper flags, and then the macOS API denies access:
once the GK flags reach some state, quarantine information is not
returned. This is not documented (as far as I can see) but moons ago,
[somebody else on the internet witnessed the same
thing](https://cocoa-dev.apple.narkive.com/kkYeAC8o/is-it-possible-to-read-your-own-quarantine-info-after-launch).
To work around, we run the system SQLite binary, to fish the relevant
information out of the per-user quarantine database. (SQLite is
installed by default on all relevant macOS versions.)
The most significant security concern I see is whether we can trust
this binary (in /usr/bin/sqlite3). Some discussion within the
Install/Update team suggested that an attacker who could corrupt or
modify that binary already had write access to the disk, which is an
attack vector equal to a totally compromised Firefox. If we determine
that we can't use the system SQLite binary, then we could use
Firefox's compiled copy of SQLite, but we might see versioning
issues. The system SQLite binary feels more robust.
This is implemented as a JS component for convenience, mostly: there
is no API for capturing output from `nsIProcess`. It would be
possible to maintain the existing XPCOM contract by renaming the
existing contract and adding a contract with a JS implementation that
passes through to the renamed implementation, but it doesn't seem
worth the effort.
In the next commits, we will generalize the existing caching mechanism
form Windows to also apply to macOS. This is mostly a performance
optimization, so that we sniff a single well-known location rather
than launching a process at each startup, although there is a
correctness argument here as well, since the quarantine database is
dynamic and the attribution URL could expire.
Differential Revision: https://phabricator.services.mozilla.com/D92693
The existing code handles Windows and macOS quite differently. On
macOS, the tests were not as comprehensive; this patch brings them
level and makes the form of the attribution data uniform on Windows
and macOS. In particular, attribution data fields will now be URI
coded, just as they are on Windows.
This will allow us to re-use the parsing machinery we have on Windows
when we cache attribution codes on macOS.
Differential Revision: https://phabricator.services.mozilla.com/D92692
Querying the macOS system quarantine database is relatively slow, so
we'd like to do it only once. This is relevant because the Telemetry
subsystem causes the attribution data to be fetched relatively early
during startup. By caching, we accept additional disk activity but
don't have to invoke an external process, query a database, etc.
We must make BROWSER_ATTRIBUTION_ERRORS histogram apply to macOS as
well as Windows. We add error codes to capture macOS-specific
detalis. And we push this out to a later cycle, since there's no
reason to revisit this immediately.
Differential Revision: https://phabricator.services.mozilla.com/D92695
On Windows, we parse the attribution data as if it was the
`URL.search` parameter. On macOS, we parse a URL slightly
differently: we parse it like `URL.searchParams` and we deduplicate
parameters.
For caching on macOS we need to serialize the parsed data. We could
use either approach but we elect to write the code serialized as it
would be on Windows, since that is the one that is getting actively
tested in the wild.
Differential Revision: https://phabricator.services.mozilla.com/D92694
The next step is to have the cached attribution file location vary
depending on the platform, and to have the tests run on additional
platforms.
Differential Revision: https://phabricator.services.mozilla.com/D84264
The issue this is addresses is that
[`CFURLCopyResourcePropertyForKey`](https://searchfox.org/mozilla-central/rev/ea7f70dac1c5fd18400f6d2a92679777d4b21492/xpcom/io/CocoaFileUtils.mm#212)
does not return quarantine data when launched as a GUI App. What
happens is that launching via the GUI requires the user to override
GateKeeper by going to Security & Privacy > Open Anyway. Doing that
updates the GateKeeper flags, and then the macOS API denies access:
once the GK flags reach some state, quarantine information is not
returned. This is not documented (as far as I can see) but moons ago,
[somebody else on the internet witnessed the same
thing](https://cocoa-dev.apple.narkive.com/kkYeAC8o/is-it-possible-to-read-your-own-quarantine-info-after-launch).
To work around, we run the system SQLite binary, to fish the relevant
information out of the per-user quarantine database. (SQLite is
installed by default on all relevant macOS versions.)
The most significant security concern I see is whether we can trust
this binary (in /usr/bin/sqlite3). Some discussion within the
Install/Update team suggested that an attacker who could corrupt or
modify that binary already had write access to the disk, which is an
attack vector equal to a totally compromised Firefox. If we determine
that we can't use the system SQLite binary, then we could use
Firefox's compiled copy of SQLite, but we might see versioning
issues. The system SQLite binary feels more robust.
This is implemented as a JS component for convenience, mostly: there
is no API for capturing output from `nsIProcess`. It would be
possible to maintain the existing XPCOM contract by renaming the
existing contract and adding a contract with a JS implementation that
passes through to the renamed implementation, but it doesn't seem
worth the effort.
In the next commits, we will generalize the existing caching mechanism
form Windows to also apply to macOS. This is mostly a performance
optimization, so that we sniff a single well-known location rather
than launching a process at each startup, although there is a
correctness argument here as well, since the quarantine database is
dynamic and the attribution URL could expire.
Differential Revision: https://phabricator.services.mozilla.com/D92693
The existing code handles Windows and macOS quite differently. On
macOS, the tests were not as comprehensive; this patch brings them
level and makes the form of the attribution data uniform on Windows
and macOS. In particular, attribution data fields will now be URI
coded, just as they are on Windows.
This will allow us to re-use the parsing machinery we have on Windows
when we cache attribution codes on macOS.
Differential Revision: https://phabricator.services.mozilla.com/D92692
Querying the macOS system quarantine database is relatively slow, so
we'd like to do it only once. This is relevant because the Telemetry
subsystem causes the attribution data to be fetched relatively early
during startup. By caching, we accept additional disk activity but
don't have to invoke an external process, query a database, etc.
We must make BROWSER_ATTRIBUTION_ERRORS histogram apply to macOS as
well as Windows. We add error codes to capture macOS-specific
detalis. And we push this out to a later cycle, since there's no
reason to revisit this immediately.
Differential Revision: https://phabricator.services.mozilla.com/D92695
On Windows, we parse the attribution data as if it was the
`URL.search` parameter. On macOS, we parse a URL slightly
differently: we parse it like `URL.searchParams` and we deduplicate
parameters.
For caching on macOS we need to serialize the parsed data. We could
use either approach but we elect to write the code serialized as it
would be on Windows, since that is the one that is getting actively
tested in the wild.
Differential Revision: https://phabricator.services.mozilla.com/D92694
The next step is to have the cached attribution file location vary
depending on the platform, and to have the tests run on additional
platforms.
Differential Revision: https://phabricator.services.mozilla.com/D84264
The issue this is addresses is that
[`CFURLCopyResourcePropertyForKey`](https://searchfox.org/mozilla-central/rev/ea7f70dac1c5fd18400f6d2a92679777d4b21492/xpcom/io/CocoaFileUtils.mm#212)
does not return quarantine data when launched as a GUI App. What
happens is that launching via the GUI requires the user to override
GateKeeper by going to Security & Privacy > Open Anyway. Doing that
updates the GateKeeper flags, and then the macOS API denies access:
once the GK flags reach some state, quarantine information is not
returned. This is not documented (as far as I can see) but moons ago,
[somebody else on the internet witnessed the same
thing](https://cocoa-dev.apple.narkive.com/kkYeAC8o/is-it-possible-to-read-your-own-quarantine-info-after-launch).
To work around, we run the system SQLite binary, to fish the relevant
information out of the per-user quarantine database. (SQLite is
installed by default on all relevant macOS versions.)
The most significant security concern I see is whether we can trust
this binary (in /usr/bin/sqlite3). Some discussion within the
Install/Update team suggested that an attacker who could corrupt or
modify that binary already had write access to the disk, which is an
attack vector equal to a totally compromised Firefox. If we determine
that we can't use the system SQLite binary, then we could use
Firefox's compiled copy of SQLite, but we might see versioning
issues. The system SQLite binary feels more robust.
This is implemented as a JS component for convenience, mostly: there
is no API for capturing output from `nsIProcess`. It would be
possible to maintain the existing XPCOM contract by renaming the
existing contract and adding a contract with a JS implementation that
passes through to the renamed implementation, but it doesn't seem
worth the effort.
In the next commits, we will generalize the existing caching mechanism
form Windows to also apply to macOS. This is mostly a performance
optimization, so that we sniff a single well-known location rather
than launching a process at each startup, although there is a
correctness argument here as well, since the quarantine database is
dynamic and the attribution URL could expire.
Differential Revision: https://phabricator.services.mozilla.com/D92693
The existing code handles Windows and macOS quite differently. On
macOS, the tests were not as comprehensive; this patch brings them
level and makes the form of the attribution data uniform on Windows
and macOS. In particular, attribution data fields will now be URI
coded, just as they are on Windows.
This will allow us to re-use the parsing machinery we have on Windows
when we cache attribution codes on macOS.
Differential Revision: https://phabricator.services.mozilla.com/D92692
Querying the macOS system quarantine database is relatively slow, so
we'd like to do it only once. This is relevant because the Telemetry
subsystem causes the attribution data to be fetched relatively early
during startup. By caching, we accept additional disk activity but
don't have to invoke an external process, query a database, etc.
We must make BROWSER_ATTRIBUTION_ERRORS histogram apply to macOS as
well as Windows. We add error codes to capture macOS-specific
detalis. And we push this out to a later cycle, since there's no
reason to revisit this immediately.
Differential Revision: https://phabricator.services.mozilla.com/D92695
On Windows, we parse the attribution data as if it was the
`URL.search` parameter. On macOS, we parse a URL slightly
differently: we parse it like `URL.searchParams` and we deduplicate
parameters.
For caching on macOS we need to serialize the parsed data. We could
use either approach but we elect to write the code serialized as it
would be on Windows, since that is the one that is getting actively
tested in the wild.
Differential Revision: https://phabricator.services.mozilla.com/D92694
The next step is to have the cached attribution file location vary
depending on the platform, and to have the tests run on additional
platforms.
Differential Revision: https://phabricator.services.mozilla.com/D84264
The issue this is addresses is that
[`CFURLCopyResourcePropertyForKey`](https://searchfox.org/mozilla-central/rev/ea7f70dac1c5fd18400f6d2a92679777d4b21492/xpcom/io/CocoaFileUtils.mm#212)
does not return quarantine data when launched as a GUI App. What
happens is that launching via the GUI requires the user to override
GateKeeper by going to Security & Privacy > Open Anyway. Doing that
updates the GateKeeper flags, and then the macOS API denies access:
once the GK flags reach some state, quarantine information is not
returned. This is not documented (as far as I can see) but moons ago,
[somebody else on the internet witnessed the same
thing](https://cocoa-dev.apple.narkive.com/kkYeAC8o/is-it-possible-to-read-your-own-quarantine-info-after-launch).
To work around, we run the system SQLite binary, to fish the relevant
information out of the per-user quarantine database. (SQLite is
installed by default on all relevant macOS versions.)
The most significant security concern I see is whether we can trust
this binary (in /usr/bin/sqlite3). Some discussion within the
Install/Update team suggested that an attacker who could corrupt or
modify that binary already had write access to the disk, which is an
attack vector equal to a totally compromised Firefox. If we determine
that we can't use the system SQLite binary, then we could use
Firefox's compiled copy of SQLite, but we might see versioning
issues. The system SQLite binary feels more robust.
This is implemented as a JS component for convenience, mostly: there
is no API for capturing output from `nsIProcess`. It would be
possible to maintain the existing XPCOM contract by renaming the
existing contract and adding a contract with a JS implementation that
passes through to the renamed implementation, but it doesn't seem
worth the effort.
In the next commits, we will generalize the existing caching mechanism
form Windows to also apply to macOS. This is mostly a performance
optimization, so that we sniff a single well-known location rather
than launching a process at each startup, although there is a
correctness argument here as well, since the quarantine database is
dynamic and the attribution URL could expire.
Differential Revision: https://phabricator.services.mozilla.com/D92693
The existing code handles Windows and macOS quite differently. On
macOS, the tests were not as comprehensive; this patch brings them
level and makes the form of the attribution data uniform on Windows
and macOS. In particular, attribution data fields will now be URI
coded, just as they are on Windows.
This will allow us to re-use the parsing machinery we have on Windows
when we cache attribution codes on macOS.
Differential Revision: https://phabricator.services.mozilla.com/D92692
Add "ua" and increase allowed length for attribution codes. Allows asrouter devtools to set all keys.
Differential Revision: https://phabricator.services.mozilla.com/D57906
--HG--
extra : moz-landing-system : lando
***
Bug 1514594: Part 3a - Change ChromeUtils.import to return an exports object; not pollute global. r=mccr8
This changes the behavior of ChromeUtils.import() to return an exports object,
rather than a module global, in all cases except when `null` is passed as a
second argument, and changes the default behavior not to pollute the global
scope with the module's exports. Thus, the following code written for the old
model:
ChromeUtils.import("resource://gre/modules/Services.jsm");
is approximately the same as the following, in the new model:
var {Services} = ChromeUtils.import("resource://gre/modules/Services.jsm");
Since the two behaviors are mutually incompatible, this patch will land with a
scripted rewrite to update all existing callers to use the new model rather
than the old.
***
Bug 1514594: Part 3b - Mass rewrite all JS code to use the new ChromeUtils.import API. rs=Gijs
This was done using the followng script:
https://bitbucket.org/kmaglione/m-c-rewrites/src/tip/processors/cu-import-exports.jsm
***
Bug 1514594: Part 3c - Update ESLint plugin for ChromeUtils.import API changes. r=Standard8
Differential Revision: https://phabricator.services.mozilla.com/D16747
***
Bug 1514594: Part 3d - Remove/fix hundreds of duplicate imports from sync tests. r=Gijs
Differential Revision: https://phabricator.services.mozilla.com/D16748
***
Bug 1514594: Part 3e - Remove no-op ChromeUtils.import() calls. r=Gijs
Differential Revision: https://phabricator.services.mozilla.com/D16749
***
Bug 1514594: Part 3f.1 - Cleanup various test corner cases after mass rewrite. r=Gijs
***
Bug 1514594: Part 3f.2 - Cleanup various non-test corner cases after mass rewrite. r=Gijs
Differential Revision: https://phabricator.services.mozilla.com/D16750
--HG--
extra : rebase_source : 359574ee3064c90f33bf36c2ebe3159a24cc8895
extra : histedit_source : b93c8f42808b1599f9122d7842d2c0b3e656a594%2C64a3a4e3359dc889e2ab2b49461bab9e27fc10a7
Automatic changes by ESLint, except for manual corrections for .xml files.
Differential Revision: https://phabricator.services.mozilla.com/D4439
--HG--
extra : moz-landing-system : lando
