Since JSWindowActors don't have direct access to synchronous messaging,
ChromeScript callers are going to need to migrate to asynchronous messaging
and queries instead.
Since there's no comparable API to sendQuery for frame message managers, this
patch adds a stub that uses synchronous messaging, but makes the API appear
asynchronous, and migrates callers to use it instead of direct synchronous
messaging. This will be replaced with a true synchronous API in the actor
migration.
Fortunately, most of the time, this actually leads to simpler code. The
`sendQuery` API doesn't have the odd return value semantics of
`sendSyncMessage`, and can usually just be used as a drop-in replacement. Many
of the `sendSyncMessage` callers don't actually use the result, and can just
be changed to `sendAsyncMessage`. And many of the existing async messaging
users can be changed to just use `sendQuery` rather than sending messages and
adding response listeners.
However, the APZ code is an exception. It relies on intricate properties of
the event loop, and doesn't have an easy way to slot in promise handlers, so I
migrated it to using sync messaging via process message managers instead.
Differential Revision: https://phabricator.services.mozilla.com/D35055
--HG--
extra : rebase_source : d5707e87f293a831a5cf2e0b0a7e977090267f78
extra : source : 75ebd6fce136ab3bd0e591c2b8b2d06d3b5bf923
The SpecialPowers set*Pref/get*Pref APIs currently use synchronous messaging
to set and get preference values from the parent process. Aside from directly
affecting callers of those APIs, it also affects callers of `pushPrefEnv`,
which is meant to be asynchronous, but is in practice usually synchronous due
to the synchronous messaging it uses.
This patch updates the getPref APIs to use the in-process preference service
(which most callers are expecting anyway), and also updates the callers of
the setPref and pushPrefEnv APIs to await the result if they're relying on it
taking effect immediately.
Unfortunately, there are some corner cases in tests that appear to only work
because of the quirks of the current sync messaging approach. The synchronous
setPref APIs, for instance, trigger preference changes in the parent
instantly, but don't update the values in the child until we've returned to
the event loop and had a chance to process the notifications from the parent.
The differnce in timing leads some tests to fail in strange ways, which this
patch works around by just adding timeouts.
There should be follow-ups for test owners to fix the flakiness.
Differential Revision: https://phabricator.services.mozilla.com/D35054
--HG--
extra : rebase_source : 941298157e7c82f420cf50ce057154ce9b85301c
extra : source : 189dc8a359815e059a4a217f788d183260bb2bfe
When we migrate SpecialPowers to a JSWindowActor, it will no longer be able to
use synchronous IPC messaging, which means that its current synchronous APIs
will have to become asynchronous.
This patch doesn't change the behavior of those functions, but it does change
their callers to `await` their return values rather than using them directly.
This pattern will work the same whether the functions return a promise or a
plain value, which simplifies the migration.
Differential Revision: https://phabricator.services.mozilla.com/D35053
--HG--
extra : rebase_source : baffba2107b175250573baae3f54d48becbd2a16
extra : source : b4ed40bea2698802ef562a0931c0b560737fb89d
Firefox cannot open any URL when Safe Browsing database is corrupted.
This is because when Necko asks URL Classifier to classify an URL, it
first suspends the channel, and then wait for a callback to see if the
channel should be canceled or resumed.
In this bug, DoLocalLookup returns an error because Safe Browsing is not
able to read its database(database is corrupted), but we don't call the
callback to resume loading channel, so users cannot load any website.
This patch adds a scope exit handler to make sure the callback will
always be invoked.
Differential Revision: https://phabricator.services.mozilla.com/D36456
--HG--
extra : moz-landing-system : lando
Firefox cannot open any URL when Safe Browsing database is corrupted.
This is because when Necko asks URL Classifier to classify an URL, it
first suspends the channel, and then wait for a callback to see if the
channel should be canceled or resumed.
In this bug, DoLocalLookup returns an error because Safe Browsing is not
able to read its database(database is corrupted), but we don't call the
callback to resume loading channel, so users cannot load any website.
This patch adds a scope exit handler to make sure the callback will
always be invoked.
Differential Revision: https://phabricator.services.mozilla.com/D36456
--HG--
extra : moz-landing-system : lando
This patch does the following:
1. Remove testing files from disk because they are no longer required.
2. Load completions from previous version of HashStore until an update
is applied.
3. Older version of HashStore(.sbstore) & PrefixSet(.vlpset) will be
removed during an update
Differential Revision: https://phabricator.services.mozilla.com/D36002
--HG--
extra : moz-landing-system : lando
Create test entries via update introduces performance overhead.
We can store them directly in LookupCache and do not save test entries
to disk.
Differential Revision: https://phabricator.services.mozilla.com/D34576
--HG--
extra : moz-landing-system : lando
For Safe Browsing V2, Data for lookup(LookupCache) and data for update(HashStore)
are now separated. |RegenActiveTables| doesn't need to check the chunk
number in HashStore.
Differential Revision: https://phabricator.services.mozilla.com/D34575
--HG--
extra : moz-landing-system : lando
Completions are now stored in .vlpset, we can remove it from .sbstore
Functions related to optimize reading completions from .sbstore can also
be removed because it is no longer HashStore's responsibility
Differential Revision: https://phabricator.services.mozilla.com/D34574
--HG--
extra : moz-landing-system : lando
1. VariableLengthPrefixSet supports getting/setting prefixes with
AddPrefixArray and AddCompletesArray
2. VariableLengthPrefixSet supports passing prefix as an integer in
Match API. This is because how V2 and V4 see prefixes as an integer
works differently.
Differential Revision: https://phabricator.services.mozilla.com/D34547
--HG--
extra : moz-landing-system : lando
The goal of the series of patches is to improve Safe Browsing performance by
skipping uncessary file IO.
The first two patches is to remove the dependency between LookupCache and HashStore, so HashStore is only
responsible for udpates.
Before this patch, LookupCacheV2 treats prefixes and completions
differently. It uses two data structures to maintain
prefixes:
1. mPrefixSet to store prefixes from .pset
2. mUpdateCompletions to store completions from .sbstore
After this patch
1. LookupCacheV2 & LookupCacheV4 both use variable-length
prefix set. mUpdateCompletions and mPrefixSet are removed and
mVLPrefixSet is used to store all prefixes data.
2. Move common function to base class.
Note that in this patch, conversion between 4/32 bytes prefixes and
mVLPrefixSet is not yet included, it will be handled in next patch.
This patch tries not to deal with any logic changes, only focus on refining
LookupCacheV2 & LookupCacheV4 class structure to use variable-length
prefixset for both classes.
Differential Revision: https://phabricator.services.mozilla.com/D34546
--HG--
extra : moz-landing-system : lando
As now we won't load disabled text track, we have to mark track as default in order to trigger loading which would be done by automatically text track selection, or to set its track mode explicitly.
Differential Revision: https://phabricator.services.mozilla.com/D32359
--HG--
extra : moz-landing-system : lando
We use ".pset" to find active tables, but in Bug 1353956, v4 prefix files
are renamed to ".vlpset".
This patches include both 'pset' and 'vlpset' to ScanStoreDir.
Differential Revision: https://phabricator.services.mozilla.com/D32433
--HG--
extra : moz-landing-system : lando
This patch does the following:
1. Run the same prefixset tests when
* browser.safebrowsing.prefixset.max_array_size = 0
* browser.safebrowsing.prefixset.max_array_size = UINT32_MAX
This makes sure both of the methods to store prefixset are tested by existing testcases
2. Refine gtest with test fixture
3. Add TinySet and LargeSet testcases
Differential Revision: https://phabricator.services.mozilla.com/D30338
--HG--
extra : moz-landing-system : lando
The goal of this patch is to reduce the number of memory reallocation during
|MakePrefixSet|[1].
Here is the number of nsTArray memory reallocation occur during |MakePrefixSet|
(test in my local platform):
googpub-phish-proto: 58k times
goog-malware-proto: 9k times
goog-unwanted-proto: 25k times
goog-badbinurl-proto: 6k times
This patch improves the performance by:
1. For tables whose prefixes are less than 128*1024(malware, unwanted,
badinurl).
Store prefixes directly without dividing allocation into smaller chunks.
Because the maximum size to store all the prefixes in a single array for
these tables will be less than 512k, we can avoid Bug 1046038.
This simplifies the internal prefixset data structure generation and total
memory usage is also saved:
goog-malware-proto : 437K -> 163k
goog-unwanted-proto : 658k -> 446k
goog-badbinurl-proto: 320k -> 110k
The single largest allocated continuous memory size is:
goog-malware-proto : 86k -> 163k
goog-unwanted-proto : 86k -> 446k
goog-badbinurl-proto: 77k -> 110k
A further improvement can be done for this part is for tables with fewer
prefixes, we can use an one-dimension delta array to reduce the size of a
single continuous memory allocation.
2. For tables with more prefixes:
According to experiment, when prefixes are more than 400k
the delta arrays have very high chance that are full, in the case of
phishing table, we can estimate the capacity accurately before
applying delta algorithm.
The shortcoming of this part is when prefixes are between 130k~400k,
the capacity estimation is not accurate.
[1] https://searchfox.org/mozilla-central/rev/b2015fdd464f598d645342614593d4ebda922d95/toolkit/components/url-classifier/nsUrlClassifierPrefixSet.cpp#99
Differential Revision: https://phabricator.services.mozilla.com/D30046
--HG--
extra : moz-landing-system : lando
The checksum calculating code is used to find the root cause of a crash
bug during update(Bug 1362761). Since the algorithm will be update in
these series of patches, we don't need to keep it.
Differential Revision: https://phabricator.services.mozilla.com/D26667
--HG--
extra : moz-landing-system : lando
This patch does the following:
1. Run the same prefixset tests when
* browser.safebrowsing.prefixset.max_array_size = 0
* browser.safebrowsing.prefixset.max_array_size = UINT32_MAX
This makes sure both of the methods to store prefixset are tested by existing testcases
2. Refine gtest with test fixture
3. Add TinySet and LargeSet testcases
Differential Revision: https://phabricator.services.mozilla.com/D30338
--HG--
extra : moz-landing-system : lando
The goal of this patch is to reduce the number of memory reallocation during
|MakePrefixSet|[1].
Here is the number of nsTArray memory reallocation occur during |MakePrefixSet|
(test in my local platform):
googpub-phish-proto: 58k times
goog-malware-proto: 9k times
goog-unwanted-proto: 25k times
goog-badbinurl-proto: 6k times
This patch improves the performance by:
1. For tables whose prefixes are less than 128*1024(malware, unwanted,
badinurl).
Store prefixes directly without dividing allocation into smaller chunks.
Because the maximum size to store all the prefixes in a single array for
these tables will be less than 512k, we can avoid Bug 1046038.
This simplifies the internal prefixset data structure generation and total
memory usage is also saved:
goog-malware-proto : 437K -> 163k
goog-unwanted-proto : 658k -> 446k
goog-badbinurl-proto: 320k -> 110k
The single largest allocated continuous memory size is:
goog-malware-proto : 86k -> 163k
goog-unwanted-proto : 86k -> 446k
goog-badbinurl-proto: 77k -> 110k
A further improvement can be done for this part is for tables with fewer
prefixes, we can use an one-dimension delta array to reduce the size of a
single continuous memory allocation.
2. For tables with more prefixes:
According to experiment, when prefixes are more than 400k
the delta arrays have very high chance that are full, in the case of
phishing table, we can estimate the capacity accurately before
applying delta algorithm.
The shortcoming of this part is when prefixes are between 130k~400k,
the capacity estimation is not accurate.
[1] https://searchfox.org/mozilla-central/rev/b2015fdd464f598d645342614593d4ebda922d95/toolkit/components/url-classifier/nsUrlClassifierPrefixSet.cpp#99
Differential Revision: https://phabricator.services.mozilla.com/D30046
--HG--
extra : moz-landing-system : lando
The checksum calculating code is used to find the root cause of a crash
bug during update(Bug 1362761). Since the algorithm will be update in
these series of patches, we don't need to keep it.
Differential Revision: https://phabricator.services.mozilla.com/D26667
--HG--
extra : moz-landing-system : lando
Origin telemetry expects every tracking channel has tracker's hash.
Without hash value for test entries, it will trigger MOZ_ASSERT while running
testcases.
Differential Revision: https://phabricator.services.mozilla.com/D30061
--HG--
extra : moz-landing-system : lando
We can run /debug mochitests against geckoview for the cost of another dozen
or so test annotations. Both /opt and /debug mochitests are nearly worthy of
tier 1, but still waiting for bug 1534732.
Differential Revision: https://phabricator.services.mozilla.com/D30931
--HG--
extra : moz-landing-system : lando
The attributes for an interface should be on the line right before the
interface.
Interface attributes should be separated by spaces.
Clean up some trailing whitespace in widget/.
Differential Revision: https://phabricator.services.mozilla.com/D28234
--HG--
extra : moz-landing-system : lando
This excludes dom/, otherwise the file size is too large for phabricator to handle.
This is an autogenerated commit to handle scripts loading mochitest harness files, in
the simple case where the script src is on the same line as the tag.
This was generated with https://bug1544322.bmoattachments.org/attachment.cgi?id=9058170
using the `--part 2` argument.
Differential Revision: https://phabricator.services.mozilla.com/D27456
--HG--
extra : moz-landing-system : lando
