Commit graph

175 commits

Author SHA1 Message Date
Kris Maglione
01c298aca6 Bug 1460092: Add ESLint rule to enforce use of ChromeUtils.generateQI. r=Gijs
Also fixes existing code which fails the rule.

MozReview-Commit-ID: CkLFgsspGMU

--HG--
extra : rebase_source : 86a43837659aa2ad83a87eab53b7aa8d39ccf55b
2018-05-08 18:36:22 -07:00
Francois Marier
9b74d3b226 Bug 1438671 - Terminate tests early when updates fail. r=gcp
Some of the tests don't handle update errors very well and rely on timeouts to
fire after several minutes. Since these tests are not actually testing update
failure modes, it's safe to fail quicly and terminate the test with an
exception.

MozReview-Commit-ID: EJgaWke6kl2

--HG--
extra : rebase_source : 486797e1a3c23c5a65ae7c5134fd4db7e663a3aa
2018-03-27 16:29:02 -07:00
Francois Marier
8c3d3e2350 Bug 1436213 - Make test_bug1274685_unowned_list.js work on pver2 and pver4. r=gcp
This test is supposed to verify that Safe Browsing providers can
be initialized correctly even when a table is not configured
properly.

By removing a table from both google and google4, we ensure that
the test will be meaningful regardless of the stack in use.

Also filter out the console noise triggered by looking for the
update and gethash URLs of the "test" dummy provider.

MozReview-Commit-ID: KjWqSqA4FxJ

--HG--
extra : rebase_source : c68ab76201f200de5b5f4ab329bfe00a50603db2
2018-02-06 15:36:48 -08:00
Andrew McCreight
5dec0e0beb Bug 1432992, part 1 - Remove definitions of Ci, Cr, Cc, and Cu. r=florian
This patch was autogenerated by my decomponents.py

It covers almost every file with the extension js, jsm, html, py,
xhtml, or xul.

It removes blank lines after removed lines, when the removed lines are
preceded by either blank lines or the start of a new block. The "start
of a new block" is defined fairly hackily: either the line starts with
//, ends with */, ends with {, <![CDATA[, """ or '''. The first two
cover comments, the third one covers JS, the fourth covers JS embedded
in XUL, and the final two cover JS embedded in Python. This also
applies if the removed line was the first line of the file.

It covers the pattern matching cases like "var {classes: Cc,
interfaces: Ci, utils: Cu, results: Cr} = Components;". It'll remove
the entire thing if they are all either Ci, Cr, Cc or Cu, or it will
remove the appropriate ones and leave the residue behind. If there's
only one behind, then it will turn it into a normal, non-pattern
matching variable definition. (For instance, "const { classes: Cc,
Constructor: CC, interfaces: Ci, utils: Cu } = Components" becomes
"const CC = Components.Constructor".)

MozReview-Commit-ID: DeSHcClQ7cG

--HG--
extra : rebase_source : d9c41878036c1ef7766ef5e91a7005025bc1d72b
2018-02-06 09:36:57 -08:00
Florian Quèze
2b1c8dccb6 Bug 1339461 - script-generated patch to convert foo.indexOf(...) == -1 to foo.includes(), r=Mossop. 2018-02-01 20:45:22 +01:00
Kris Maglione
918ed6c474 Bug 1431533: Part 5a - Auto-rewrite code to use ChromeUtils import methods. r=florian
This was done using the following script:
37e3803c7a/processors/chromeutils-import.jsm

MozReview-Commit-ID: 1Nc3XDu0wGl

--HG--
extra : source : 12fc4dee861c812fd2bd032c63ef17af61800c70
extra : intermediate-source : 34c999fa006bffe8705cf50c54708aa21a962e62
extra : histedit_source : b2be2c5e5d226e6c347312456a6ae339c1e634b0
2018-01-29 15:20:18 -08:00
Cosmin Sabou
9a65a40178 Backed out 3 changesets (bug 1431533) for Android mochitest failures on testEventDispatcher on a CLOSED TREE
Backed out changeset a1eca62826a1 (bug 1431533)
Backed out changeset 34c999fa006b (bug 1431533)
Backed out changeset e2674287e57f (bug 1431533)
2018-01-30 07:17:48 +02:00
Kris Maglione
6476f95b13 Bug 1431533: Part 5a - Auto-rewrite code to use ChromeUtils import methods. r=florian
This was done using the following script:
37e3803c7a/processors/chromeutils-import.jsm

MozReview-Commit-ID: 1Nc3XDu0wGl

--HG--
extra : source : 12fc4dee861c812fd2bd032c63ef17af61800c70
2018-01-29 15:20:18 -08:00
Brindusan Cristian
af8879d1eb Backed out 2 changesets (bug 1431533) for ESlint failures on a CLOSED TREE
Backed out changeset 6e56f4c8843e (bug 1431533)
Backed out changeset 12fc4dee861c (bug 1431533)
2018-01-30 02:32:43 +02:00
Kris Maglione
c276bb9375 Bug 1431533: Part 5a - Auto-rewrite code to use ChromeUtils import methods. r=florian
This was done using the following script:
37e3803c7a/processors/chromeutils-import.jsm

MozReview-Commit-ID: 1Nc3XDu0wGl

--HG--
extra : rebase_source : c004a023389f1f6bf3d2f3efe93c13d423b23ccd
2018-01-29 15:20:18 -08:00
Florian Quèze
0f55cd45be Bug 1421992 - script-generated patch to replace do_execute_soon, do_print and do_register_cleanup with executeSoon, info and registerCleanupFunction, rs=Gijs. 2017-12-21 11:10:23 +01:00
Florian Quèze
032c961e0a Bug 1421992 - script-generated patch to replace do_check_* functions with their Assert.* equivalents, rs=Gijs. 2017-12-21 11:08:17 +01:00
DimiL
fc50fe21a9 Bug 1421500 - Disable back-off for SafeBrowsing testcases not testing back-off. r=francois
After this patch landed, we will have 3 cases:
1. For providers are not "test", for example, google, google4, mozilla ... etc
   backoff CANNOT be disabled.
2. For "test" provider, if preference "browser.safebrowsing.provider.test.disableBackoff" is ON
   backoff is disabled.
3. For "test" provider, if preference "browser.safebrowsing.provider.test.disableBackoff" is Off
   backoff is NOT disabled.

So if your testcase will use listmanager or hashcompleter, you should try to use "test" provider
if possible, otherwise testcase may encounter intermittent failure due to backoff.

MozReview-Commit-ID: 3BDxs0ARyQM

--HG--
extra : rebase_source : 1736abe307849089d68f0f2b1a0127b9b465abed
2017-12-08 12:18:34 +08:00
Csoregi Natalia
3257faa639 Backed out changeset 2467ce87767e (bug 1421500) for ESLint failure /toolkit/components/url-classifier/nsUrlClassifierLib.js:108. r=backout on a CLOSED TREE 2017-12-14 04:38:07 +02:00
DimiL
9e0fa6963a Bug 1421500 - Disable back-off for SafeBrowsing testcases not testing back-off. r=francois
After this patch landed, we will have 3 cases:
1. For providers are not "test", for example, google, google4, mozilla ... etc
   backoff CANNOT be disabled.
2. For "test" provider, if preference "browser.safebrowsing.provider.test.disableBackoff" is ON
   backoff is disabled.
3. For "test" provider, if preference "browser.safebrowsing.provider.test.disableBackoff" is Off
   backoff is NOT disabled.

So if your testcase will use listmanager or hashcompleter, you should try to use "test" provider
if possible, otherwise testcase may encounter intermittent failure due to backoff.

MozReview-Commit-ID: 3BDxs0ARyQM

--HG--
extra : rebase_source : 84250bc7aab28a896afb2d47f98435139139c1f3
2017-12-08 12:18:34 +08:00
Mark Banner
aef9c37979 Bug 1412893 - Change instances of using getService to Services.jsm where possible in toolkit/components/url-classifier r=francois
MozReview-Commit-ID: 7ejzcckcwQ1

--HG--
extra : rebase_source : 72eca0da3d8a0c125958c6272fb15edfbb122549
2017-10-31 20:22:58 +00:00
Mark Banner
4de6bf22b1 Bug 1411368 - Automatically fix no-multi-spaces issues raised when using ESLint 4. r=mossop
MozReview-Commit-ID: H5YVp3rnzGo

--HG--
extra : rebase_source : 5b45b6c0df834131812d094e975047eaad374e06
2017-10-26 11:47:01 +01:00
Dan Banner
7caa92d5d8 Bug 1408777 - Automatically fix instances of missing semicolons in the tree. r=Standard8
MozReview-Commit-ID: Jm8BRgt6mIv
2017-10-15 20:50:39 +01:00
Thomas Nguyen
d721de8d81 Bug 1389315 - Isolate Safe Browsing completions cached by each provider. r=francois
MozReview-Commit-ID: INiuM5ydv2f

--HG--
extra : rebase_source : 376fa8f04d850d23e8de908e8a3484350269a671
2017-09-14 17:05:52 +08:00
Thomas Nguyen
e70d76485c Bug 1394031 - Remove mCryptoHash members of nsUrlClassifierDBServiceWorker and ProtocolParser
The usage of cryptoHash consists of a complete set of Init, Update, and Finish, there's
no reason to keep it around

MozReview-Commit-ID: 7bT9IsWEM5m
2017-09-14 10:51:55 +08:00
Andrew McCreight
bfd580fb71 Bug 1186409 - Use Cu.getGlobalForObject when importing properties off a JSM global. r=kmag
With JSM global sharing, the object returned by Cu.import() is a
NonSyntacticVariablesObject, rather than a global. Various code tries
to use properties from a JSM global via an import.

Cu.importGlobalProperties can also be used in some places.

MozReview-Commit-ID: HudCXO2GKN0

--HG--
extra : rebase_source : 6b5fa6f5509397504cb461a761f6cc2399f18c40
2017-06-23 13:51:14 -07:00
Thomas Nguyen
dd42019824 Bug 1385609 - Fix backoff issue that makes SB lists no longer update r=francois
The issue occurs when nsITimer is fired earlier than the backoff time. In that
case, the update doesn't proceed and we never make another attempt because the
backoff update timer was oneshot.

We fix the issue in two ways:

- Add a tolerance of 1 second in case the timer fires too early.
- Set another oneshot timer whenever we are prevented from updating due to
backoff.

MozReview-Commit-ID: E2ogNRsHJVK

--HG--
extra : rebase_source : c81fa77934f6c39e1c5d07b19785a01546e02542
2017-08-30 18:04:10 +08:00
Sebastian Hengst
34e6bbd998 Backed out changeset 14b4af261515 (bug 1385609) for eslint failures in nsUrlClassifierLib.js. r=backout 2017-09-05 19:04:43 +02:00
Thomas Nguyen
a26ed706a3 Bug 1385609 - Fix backoff issue that makes SB lists no longer update r=francois
The issue occurs when nsITimer is fired earlier than the backoff time. In that
case, the update doesn't proceed and we never make another attempt because the
backoff update timer was oneshot.

We fix the issue in two ways:

- Add a tolerance of 1 second in case the timer fires too early.
- Set another oneshot timer whenever we are prevented from updating due to
backoff.

MozReview-Commit-ID: E2ogNRsHJVK

--HG--
extra : rebase_source : 17aa70d8583cc84e28e57410de66eaac63bd18bb
2017-08-30 18:04:10 +08:00
Francois Marier
530db90579 Bug 1395733 - Ensure the default value of urlclassifier.malwareTable is always sorted. r=gcp
MozReview-Commit-ID: 10M2ZpDyAI9

--HG--
extra : rebase_source : 4523a0a1fd3adbdc8f0c13676548409ee9536f06
2017-08-31 14:54:11 -07:00
Henry Chang
ed11934666 Bug 1392204 - Disable goog-badbinurl-proto and goog-downloadwhite-proto on Fennec. r=francois
Version 4 of the Google Safe Browsing server will return a 404 if any of the
application reputation lists are requested on Android. As a result, we should
avoid these threat types from being sent along with ANDROID_PLATFORM.

MozReview-Commit-ID: 6TUBVxe455y

--HG--
extra : rebase_source : dee095c008f4d328f359c66d20f0cc2dfcd109f3
extra : source : 5d6338807c6b21b7672236cac01b13bd75155225
2017-08-22 12:20:57 +08:00
Henry Chang
4950f839d2 Bug 1388582 - Skip unsupported threat types on current platform while making v4 request. r=francois
MozReview-Commit-ID: FTmOMQ339ZL

--HG--
extra : rebase_source : ea41ae42dc2faa9bab040018b4ee3a1ade49ed69
2017-08-16 15:30:20 +08:00
Dan Banner
42c8241ce0 Bug 1386684 - Enable ESLint for toolkit/components/url-classifier (manual fixes). r=hchang
MozReview-Commit-ID: JH1XDFMVd43

--HG--
extra : rebase_source : 41c83027c09e3071d3ddb929a01b5dd689109115
2017-08-02 20:09:14 +01:00
Dan Banner
436f06fda4 Bug 1386684 - Enable ESLint for toolkit/components/url-classifier (automatic fixes). r=hchang
MozReview-Commit-ID: F0Z8dRaYOku

--HG--
extra : rebase_source : 3ebc9ab6ea9f1741d8a986c9217575644411e2a1
2017-08-02 16:12:07 +01:00
Henry Chang
4a3106b91e Bug 1375277 - Test case for new error code NS_ERROR_HARMFUL_URI. r=francois
The test is based on "test-harmful-simple" so we can add testing entries
in a human-readable way.

MozReview-Commit-ID: IJcYrp04uFs

--HG--
extra : rebase_source : afc4329af4cd5fc31a783a8f0f2e3a4bdcb2a432
extra : source : 55a2e275514d2154a23a656e48f685cc6032e5c3
2017-07-28 17:38:01 +08:00
Florian Quèze
2924991bf6 Bug 1368456 - remove Promise.jsm imports in tests, r=mconley. 2017-06-23 11:25:52 +02:00
DimiL
e4d42d14a2 Bug 1359299 - V4 caches in LookupCache need to be copied around in copy constructor. r=hchang
MozReview-Commit-ID: AjzUUmQKiPW

--HG--
extra : rebase_source : 1c30ee5aa3548dce9b861a70660b1e60410f3654
2017-06-06 14:16:57 +08:00
Ryan VanderMeulen
35b3731614 Backed out changeset c0b940487708 (bug 1359299) for causing intermittent Windows safebrowsing crashes. 2017-05-24 09:11:04 -04:00
DimiL
81b83a1f76 Bug 1359299 - Copy fullhash cache when update. r=hchang
After adopting the new thread model for safebrowsing, we will create a new
lookup cache for update so we can still check lookup cache at the same time.

Prefix set, completions will be generated when we open the new lookup cache
but it won't include cache, so we will loss cache after that.

This patch will copy cache data from old lookup cache to new lookup
cache while update.

MozReview-Commit-ID: L0WpiHOGIGm

--HG--
extra : rebase_source : ebaf249535561b87a983a3f24dacb8fbab7c096e
2017-05-23 09:19:06 +08:00
Thomas Nguyen
c8d511e264 Bug 1363879 - Sort gethash prefixes to hide noise entries r=francois
In V2 we shuffled the hash entries before sending the request to obscure the real
entry from noises. We should also hide the real entry in V4. Using sort() is
enough for both V2 and V4 because the array contains exactly 5 entries in almost
all cases.

MozReview-Commit-ID: 4uOXIF83KQL

--HG--
extra : rebase_source : 97c6439965e864863dc99f194356eb1efb0235df
2017-05-17 16:11:36 +08:00
DimiL
5aef415cc6 Bug 1333328 - Refactor cache miss handling mechanism for V2. r=francois
In this patch, we will make Safebrowsing V2 caching use the same algorithm as V4.
So we remove "mMissCache" for negative caching and TableFresness check for
positive caching.

But Safebrowsing V2 doesn't contain negative/positive cache duration information in
gethash response. So we hard-code a fixed value, 15 minutes, as cache duration.
In this way, we can sync the mechanism we handle caching for V2 and V4.

An extra effort for V2 here is that we need to manually record prefixes misses
because we won't get any response for those prefixes(implemented in
nsUrlClassifierLookupCallback::CacheMisses).

--HG--
extra : rebase_source : 0fb7938464ad24a01a51493ecb1b5c0197c16123
2017-05-04 09:38:14 +08:00
Florian Quèze
97f401c644 Bug 1362882 - script-generated patch to switch from Promise.defer() to new Promise() where it's straightforward, rs=Mossop. 2017-05-12 14:55:27 +02:00
dimi
127351f6ef Bug 1311935 - P2. Process fullHashes.find response. r=francois
This patch includes following changes:

1. nsUrlClassifierHashCompleter.js
   nsUrlClassifierHashCompleter.idl
   - Add completionV4 interface for hashCompleter to pass response data to
     DB service.
   - Process response data includes negative cache duration, matched full
     hashes and cache duration for each match. Full matches are passed through
     nsIFullHashMatch interface.
   - Change _requests.responses from array contains matched fullhashes to
     dictionary so that it can store additional information likes negative cache
     duration.
2. nsUrlClassifierDBService.cpp
   - Implement CompletionV4 interface, store response data to CacheResultV4
     object. Expired duration to expired time is handled here.
   - Add CacheResultToTableUpdate function to convert V2 & V4 cache result
     to TableUpdate object.
3. LookupCache.h
   - Extend CacheResult to CacheResultV2 and CacheResultV4 so we can store
     response data in CompletionV2 and CompletionV4.
4. HashStore.h
   - Add API and member variable in TableUpdateV4 to store response data.
     TableUpdate object is used by DB service to pass update data or gethash
     response to Classifier, so we need to extend TableUpdateV4 to be able
     to store fullHashes.find response.
6. Entry.h
   - Define the structure about how we cache fullHashes.find response.

MozReview-Commit-ID: FV4yAl2SAc6

--HG--
extra : rebase_source : 02676ded9bc0580c24b4c906199a4abaf004e296
2017-04-11 11:50:48 +08:00
Iris Hsiao
3d85c0330d Backed out 4 changesets (bug 1311935) for causing assertion crash by developer's request
Backed out changeset 27e624cd9479 (bug 1311935)
Backed out changeset 4c0381ab0990 (bug 1311935)
Backed out changeset 73587838ef16 (bug 1311935)
Backed out changeset a5a6c0f79733 (bug 1311935)
2017-04-11 11:04:54 +08:00
dimi
5112db082c Bug 1311935 - P2. Process fullHashes.find response. r=francois
This patch includes following changes:

1. nsUrlClassifierHashCompleter.js
   nsUrlClassifierHashCompleter.idl
   - Add completionV4 interface for hashCompleter to pass response data to
     DB service.
   - Process response data includes negative cache duration, matched full
     hashes and cache duration for each match. Full matches are passed through
     nsIFullHashMatch interface.
   - Change _requests.responses from array contains matched fullhashes to
     dictionary so that it can store additional information likes negative cache
     duration.
2. nsUrlClassifierDBService.cpp
   - Implement CompletionV4 interface, store response data to CacheResultV4
     object. Expired duration to expired time is handled here.
   - Add CacheResultToTableUpdate function to convert V2 & V4 cache result
     to TableUpdate object.
3. LookupCache.h
   - Extend CacheResult to CacheResultV2 and CacheResultV4 so we can store
     response data in CompletionV2 and CompletionV4.
4. HashStore.h
   - Add API and member variable in TableUpdateV4 to store response data.
     TableUpdate object is used by DB service to pass update data or gethash
     response to Classifier, so we need to extend TableUpdateV4 to be able
     to store fullHashes.find response.
6. Entry.h
   - Define the structure about how we cache fullHashes.find response.

MozReview-Commit-ID: KgR1NASl7GC

--HG--
extra : rebase_source : 424db14e4af2ffd691c384414d50f64083d5d20b
2017-04-10 16:20:09 +08:00
Henry Chang
f3be63826d Bug 1339050 - Asynchronously apply safebrowsing DB update. r=francois,gcp
A new function Classifier::AsyncApplyUpdates() is implemented for async update.
Besides, all public Classifier interfaces become "worker thread only" and
we remove DBServiceWorker::ApplyUpdatesBackground/Foreground.

In DBServiceWorker::FinishUpdate, instead of calling Classifier::ApplyUpdates,
we call Classifier::AsyncApplyUpdates and install a callback for notifying
the update observer when update is finished. The callback will occur on the
caller thread (i.e. worker thread.)

As for the shutdown issue, when the main thread is notified to shut down,
we at first *synchronously* dispatch an event to the worker thread to
shut down the update thread. After getting synchronized with all other
threads, we send last two events "CancelUpdate" and "CloseDb" to notify
dangling update (i.e. BeginUpdate is called but FinishUpdate isn't)
and do cleanup work.

MozReview-Commit-ID: DXZvA2eFKlc

--HG--
extra : rebase_source : cd2e27a6b679d2c96e769854d1582ed2dcda12bb
2017-04-06 07:07:56 +08:00
Mark Banner
bd625e4e9f Bug 1347712 - Change the testing configurations into ESLint configurations within eslint-plugin-mozilla - automatically update .eslintrc.js test config files for new config locations. r=jaws
MozReview-Commit-ID: LH0CQOPfoe6

--HG--
extra : rebase_source : 83eaf40c5cbb5e25cfb3f3b5afadf4fbf0422c92
2017-03-20 12:36:37 +01:00
Henry Chang
c5e545347a Bug 1346757 - Change the downloadError callback timing. r=francois
MozReview-Commit-ID: JleLPltEBOw

--HG--
extra : rebase_source : 842aad60fb5b1f9345ceae4bac68825a6c2c718f
2017-03-13 21:16:07 +08:00
Thomas Nguyen
fdfc58a406 Bug 1329558 - Implement Minimum wait duration for V4 gethash r=francois
MozReview-Commit-ID: 7i9Wz7pq0yJ

--HG--
extra : rebase_source : a02c2e3cbd5f535192f5dbaadabce16abd6bb35f
2017-02-09 16:56:44 +08:00
Mark Banner
16e6d381ac Bug 503613 - Remove old 'tail =' lines from xpcshell.ini files; r=gps
MozReview-Commit-ID: 62Hp5ISxowJ

--HG--
extra : rebase_source : daa8efb3409031fea553f4fd0c9d0746e38dc308
extra : histedit_source : b4c23aacf678ba0d0ac9c09191a7c494ead11a08
2017-01-18 10:30:39 +00:00
Mark Banner
34bd93ded7 Bug 527444 - use do_register_cleanup calls instead of tail_*.js files in xpcshell, in Core and Toolkit. r=Paolo.
Patch originally by Rahid Hasan. Updated by Akshendra Pratap Singh, jdm and Standard8.

MozReview-Commit-ID: F0edTCSsXzy

--HG--
extra : rebase_source : 7123a8162a4cdbcfb08f9a765d158e179c46b202
2017-01-12 16:09:58 +00:00
Florian Quèze
34e970eb9b Bug 1329182 - remove trailing newURI null parameters in toolkit/, r=jaws. 2017-01-09 20:27:25 +01:00
Henry Chang
758d80f10b Bug 1276826 - Part 2. Send gethash request and handle gethash response for v4. r=francois
MozReview-Commit-ID: 6fIbN6a6gHI

--HG--
extra : rebase_source : d64a2b4f85b24aac0a847a46748b3b3fdf0db749
extra : intermediate-source : c486204319c1afc158b871bad7edd2f7a565022a
extra : source : adf76dccec9d51df0d3a659ffa3dd201d8115bc4
2016-11-28 14:55:36 +08:00
Henry Chang
a5327a4bbb Bug 1319286 - Cache nsIUrlClassifierDBService.getTables result until next update. r=francois.
MozReview-Commit-ID: ItjTQNzCVED
2016-11-22 10:39:58 +08:00
Thomas Nguyen
0ff9c49b98 Bug 1302917 - Call callback correctly to Fix intermittent test_streamupdater.js r=francois
MozReview-Commit-ID: 6Gz0KM2CprU

--HG--
extra : rebase_source : 19e4b06067081fb3dd9a111707df6e15cb3fa644
2016-11-22 17:34:56 +08:00