Also fixes existing code which fails the rule.
MozReview-Commit-ID: CkLFgsspGMU
--HG--
extra : rebase_source : 86a43837659aa2ad83a87eab53b7aa8d39ccf55b
Some of the tests don't handle update errors very well and rely on timeouts to
fire after several minutes. Since these tests are not actually testing update
failure modes, it's safe to fail quicly and terminate the test with an
exception.
MozReview-Commit-ID: EJgaWke6kl2
--HG--
extra : rebase_source : 486797e1a3c23c5a65ae7c5134fd4db7e663a3aa
This test is supposed to verify that Safe Browsing providers can
be initialized correctly even when a table is not configured
properly.
By removing a table from both google and google4, we ensure that
the test will be meaningful regardless of the stack in use.
Also filter out the console noise triggered by looking for the
update and gethash URLs of the "test" dummy provider.
MozReview-Commit-ID: KjWqSqA4FxJ
--HG--
extra : rebase_source : c68ab76201f200de5b5f4ab329bfe00a50603db2
This patch was autogenerated by my decomponents.py
It covers almost every file with the extension js, jsm, html, py,
xhtml, or xul.
It removes blank lines after removed lines, when the removed lines are
preceded by either blank lines or the start of a new block. The "start
of a new block" is defined fairly hackily: either the line starts with
//, ends with */, ends with {, <![CDATA[, """ or '''. The first two
cover comments, the third one covers JS, the fourth covers JS embedded
in XUL, and the final two cover JS embedded in Python. This also
applies if the removed line was the first line of the file.
It covers the pattern matching cases like "var {classes: Cc,
interfaces: Ci, utils: Cu, results: Cr} = Components;". It'll remove
the entire thing if they are all either Ci, Cr, Cc or Cu, or it will
remove the appropriate ones and leave the residue behind. If there's
only one behind, then it will turn it into a normal, non-pattern
matching variable definition. (For instance, "const { classes: Cc,
Constructor: CC, interfaces: Ci, utils: Cu } = Components" becomes
"const CC = Components.Constructor".)
MozReview-Commit-ID: DeSHcClQ7cG
--HG--
extra : rebase_source : d9c41878036c1ef7766ef5e91a7005025bc1d72b
This was done using the following script:
37e3803c7a/processors/chromeutils-import.jsm
MozReview-Commit-ID: 1Nc3XDu0wGl
--HG--
extra : source : 12fc4dee861c812fd2bd032c63ef17af61800c70
extra : intermediate-source : 34c999fa006bffe8705cf50c54708aa21a962e62
extra : histedit_source : b2be2c5e5d226e6c347312456a6ae339c1e634b0
This was done using the following script:
37e3803c7a/processors/chromeutils-import.jsm
MozReview-Commit-ID: 1Nc3XDu0wGl
--HG--
extra : source : 12fc4dee861c812fd2bd032c63ef17af61800c70
This was done using the following script:
37e3803c7a/processors/chromeutils-import.jsm
MozReview-Commit-ID: 1Nc3XDu0wGl
--HG--
extra : rebase_source : c004a023389f1f6bf3d2f3efe93c13d423b23ccd
After this patch landed, we will have 3 cases:
1. For providers are not "test", for example, google, google4, mozilla ... etc
backoff CANNOT be disabled.
2. For "test" provider, if preference "browser.safebrowsing.provider.test.disableBackoff" is ON
backoff is disabled.
3. For "test" provider, if preference "browser.safebrowsing.provider.test.disableBackoff" is Off
backoff is NOT disabled.
So if your testcase will use listmanager or hashcompleter, you should try to use "test" provider
if possible, otherwise testcase may encounter intermittent failure due to backoff.
MozReview-Commit-ID: 3BDxs0ARyQM
--HG--
extra : rebase_source : 1736abe307849089d68f0f2b1a0127b9b465abed
After this patch landed, we will have 3 cases:
1. For providers are not "test", for example, google, google4, mozilla ... etc
backoff CANNOT be disabled.
2. For "test" provider, if preference "browser.safebrowsing.provider.test.disableBackoff" is ON
backoff is disabled.
3. For "test" provider, if preference "browser.safebrowsing.provider.test.disableBackoff" is Off
backoff is NOT disabled.
So if your testcase will use listmanager or hashcompleter, you should try to use "test" provider
if possible, otherwise testcase may encounter intermittent failure due to backoff.
MozReview-Commit-ID: 3BDxs0ARyQM
--HG--
extra : rebase_source : 84250bc7aab28a896afb2d47f98435139139c1f3
With JSM global sharing, the object returned by Cu.import() is a
NonSyntacticVariablesObject, rather than a global. Various code tries
to use properties from a JSM global via an import.
Cu.importGlobalProperties can also be used in some places.
MozReview-Commit-ID: HudCXO2GKN0
--HG--
extra : rebase_source : 6b5fa6f5509397504cb461a761f6cc2399f18c40
The issue occurs when nsITimer is fired earlier than the backoff time. In that
case, the update doesn't proceed and we never make another attempt because the
backoff update timer was oneshot.
We fix the issue in two ways:
- Add a tolerance of 1 second in case the timer fires too early.
- Set another oneshot timer whenever we are prevented from updating due to
backoff.
MozReview-Commit-ID: E2ogNRsHJVK
--HG--
extra : rebase_source : c81fa77934f6c39e1c5d07b19785a01546e02542
The issue occurs when nsITimer is fired earlier than the backoff time. In that
case, the update doesn't proceed and we never make another attempt because the
backoff update timer was oneshot.
We fix the issue in two ways:
- Add a tolerance of 1 second in case the timer fires too early.
- Set another oneshot timer whenever we are prevented from updating due to
backoff.
MozReview-Commit-ID: E2ogNRsHJVK
--HG--
extra : rebase_source : 17aa70d8583cc84e28e57410de66eaac63bd18bb
Version 4 of the Google Safe Browsing server will return a 404 if any of the
application reputation lists are requested on Android. As a result, we should
avoid these threat types from being sent along with ANDROID_PLATFORM.
MozReview-Commit-ID: 6TUBVxe455y
--HG--
extra : rebase_source : dee095c008f4d328f359c66d20f0cc2dfcd109f3
extra : source : 5d6338807c6b21b7672236cac01b13bd75155225
The test is based on "test-harmful-simple" so we can add testing entries
in a human-readable way.
MozReview-Commit-ID: IJcYrp04uFs
--HG--
extra : rebase_source : afc4329af4cd5fc31a783a8f0f2e3a4bdcb2a432
extra : source : 55a2e275514d2154a23a656e48f685cc6032e5c3
After adopting the new thread model for safebrowsing, we will create a new
lookup cache for update so we can still check lookup cache at the same time.
Prefix set, completions will be generated when we open the new lookup cache
but it won't include cache, so we will loss cache after that.
This patch will copy cache data from old lookup cache to new lookup
cache while update.
MozReview-Commit-ID: L0WpiHOGIGm
--HG--
extra : rebase_source : ebaf249535561b87a983a3f24dacb8fbab7c096e
In V2 we shuffled the hash entries before sending the request to obscure the real
entry from noises. We should also hide the real entry in V4. Using sort() is
enough for both V2 and V4 because the array contains exactly 5 entries in almost
all cases.
MozReview-Commit-ID: 4uOXIF83KQL
--HG--
extra : rebase_source : 97c6439965e864863dc99f194356eb1efb0235df
In this patch, we will make Safebrowsing V2 caching use the same algorithm as V4.
So we remove "mMissCache" for negative caching and TableFresness check for
positive caching.
But Safebrowsing V2 doesn't contain negative/positive cache duration information in
gethash response. So we hard-code a fixed value, 15 minutes, as cache duration.
In this way, we can sync the mechanism we handle caching for V2 and V4.
An extra effort for V2 here is that we need to manually record prefixes misses
because we won't get any response for those prefixes(implemented in
nsUrlClassifierLookupCallback::CacheMisses).
--HG--
extra : rebase_source : 0fb7938464ad24a01a51493ecb1b5c0197c16123
This patch includes following changes:
1. nsUrlClassifierHashCompleter.js
nsUrlClassifierHashCompleter.idl
- Add completionV4 interface for hashCompleter to pass response data to
DB service.
- Process response data includes negative cache duration, matched full
hashes and cache duration for each match. Full matches are passed through
nsIFullHashMatch interface.
- Change _requests.responses from array contains matched fullhashes to
dictionary so that it can store additional information likes negative cache
duration.
2. nsUrlClassifierDBService.cpp
- Implement CompletionV4 interface, store response data to CacheResultV4
object. Expired duration to expired time is handled here.
- Add CacheResultToTableUpdate function to convert V2 & V4 cache result
to TableUpdate object.
3. LookupCache.h
- Extend CacheResult to CacheResultV2 and CacheResultV4 so we can store
response data in CompletionV2 and CompletionV4.
4. HashStore.h
- Add API and member variable in TableUpdateV4 to store response data.
TableUpdate object is used by DB service to pass update data or gethash
response to Classifier, so we need to extend TableUpdateV4 to be able
to store fullHashes.find response.
6. Entry.h
- Define the structure about how we cache fullHashes.find response.
MozReview-Commit-ID: FV4yAl2SAc6
--HG--
extra : rebase_source : 02676ded9bc0580c24b4c906199a4abaf004e296
This patch includes following changes:
1. nsUrlClassifierHashCompleter.js
nsUrlClassifierHashCompleter.idl
- Add completionV4 interface for hashCompleter to pass response data to
DB service.
- Process response data includes negative cache duration, matched full
hashes and cache duration for each match. Full matches are passed through
nsIFullHashMatch interface.
- Change _requests.responses from array contains matched fullhashes to
dictionary so that it can store additional information likes negative cache
duration.
2. nsUrlClassifierDBService.cpp
- Implement CompletionV4 interface, store response data to CacheResultV4
object. Expired duration to expired time is handled here.
- Add CacheResultToTableUpdate function to convert V2 & V4 cache result
to TableUpdate object.
3. LookupCache.h
- Extend CacheResult to CacheResultV2 and CacheResultV4 so we can store
response data in CompletionV2 and CompletionV4.
4. HashStore.h
- Add API and member variable in TableUpdateV4 to store response data.
TableUpdate object is used by DB service to pass update data or gethash
response to Classifier, so we need to extend TableUpdateV4 to be able
to store fullHashes.find response.
6. Entry.h
- Define the structure about how we cache fullHashes.find response.
MozReview-Commit-ID: KgR1NASl7GC
--HG--
extra : rebase_source : 424db14e4af2ffd691c384414d50f64083d5d20b
A new function Classifier::AsyncApplyUpdates() is implemented for async update.
Besides, all public Classifier interfaces become "worker thread only" and
we remove DBServiceWorker::ApplyUpdatesBackground/Foreground.
In DBServiceWorker::FinishUpdate, instead of calling Classifier::ApplyUpdates,
we call Classifier::AsyncApplyUpdates and install a callback for notifying
the update observer when update is finished. The callback will occur on the
caller thread (i.e. worker thread.)
As for the shutdown issue, when the main thread is notified to shut down,
we at first *synchronously* dispatch an event to the worker thread to
shut down the update thread. After getting synchronized with all other
threads, we send last two events "CancelUpdate" and "CloseDb" to notify
dangling update (i.e. BeginUpdate is called but FinishUpdate isn't)
and do cleanup work.
MozReview-Commit-ID: DXZvA2eFKlc
--HG--
extra : rebase_source : cd2e27a6b679d2c96e769854d1582ed2dcda12bb
Patch originally by Rahid Hasan. Updated by Akshendra Pratap Singh, jdm and Standard8.
MozReview-Commit-ID: F0edTCSsXzy
--HG--
extra : rebase_source : 7123a8162a4cdbcfb08f9a765d158e179c46b202