Commit graph

124 commits

Author SHA1 Message Date
Eric Rahm
9398bd6f43 Bug 1393235 - Fix improper usages of string functions. r=njn
This fixes usages of `Find`, `RFind` and the equality operator that kind of
work right now but will break with the proper type checking of a templatized
version of the string classes.

For `Find` and `RFind` it appears that `nsCString::(R)Find("foo", 0)` calls
were being coerced to the `Find(char*, bool, int, int)` versions. The intent was
probably to just start searching from position zero.

For the equality operator, the type of nullptr is nullptr_t rather than
char(16_t)* so we'd need to add an operator overload that takes nullptr_t. In
this case just using `IsVoid` is probably more appropriate.

--HG--
extra : rebase_source : 50f78519084012ca669da0a211c489520c11d6b6
2017-08-22 19:30:46 -07:00
Kartikaya Gupta
ba4b3b9101 Bug 1384233 - Remove SizePrintfMacros.h. r=froydnj
We have a minimum requirement of VS 2015 for Windows builds, which supports
the z length modifier for format specifiers. So we don't need SizePrintfMacros.h
any more, and can just use %zu and friends directly everywhere.

MozReview-Commit-ID: 6s78RvPFMzv

--HG--
extra : rebase_source : 009ea39eb4dac1c927aa03e4f97d8ab673de8a0e
2017-07-26 16:03:57 -04:00
Thomas Nguyen
7e59091f92 Bug 1378680 - Refactor usage fallible alloc in url-classifier r=francois
MozReview-Commit-ID: HHG1ByeIwZv

--HG--
extra : rebase_source : 1b025edaac49ee202b9a10cd2f2589acb587649b
2017-07-20 16:20:27 +08:00
Bill McCloskey
f115503a0b Bug 1372405 - Provide names for all runnables in the tree (r=froydnj)
MozReview-Commit-ID: DKR6ROiHRS7
2017-06-26 14:19:58 -07:00
Carsten "Tomcat" Book
8a1350b5a6 Backed out changeset 4f6302a98ae4 (bug 1372405)
--HG--
extra : rebase_source : 41632f3158e88e692809731394a683d065a73dfb
2017-06-21 13:59:26 +02:00
Bill McCloskey
6b3e84ed5f Bug 1372405 - Provide names for all runnables in the tree (r=froydnj)
MozReview-Commit-ID: DKR6ROiHRS7
2017-06-20 21:44:11 -07:00
Carsten "Tomcat" Book
ea1b86680c Backed out changeset 9846de3bd954 (bug 1372405)
--HG--
extra : rebase_source : 5d4a48e8ec394c329994689d938d2a6e9b2752b0
2017-06-20 08:27:02 +02:00
Bill McCloskey
4592152411 Bug 1372405 - Provide names for all runnables in the tree (r=froydnj)
MozReview-Commit-ID: DKR6ROiHRS7
2017-06-19 22:25:47 -07:00
DimiL
f8c93f3b70 Bug 1366965 - Remove telemetry that compare SafeBrowsing V2 & V4. r=francois
MozReview-Commit-ID: 7vudFBK3rdp

--HG--
extra : rebase_source : ef027d6ec3d44b16dd4115f599e839aab1eadb88
2017-06-12 11:27:19 +08:00
DimiL
e4d42d14a2 Bug 1359299 - V4 caches in LookupCache need to be copied around in copy constructor. r=hchang
MozReview-Commit-ID: AjzUUmQKiPW

--HG--
extra : rebase_source : 1c30ee5aa3548dce9b861a70660b1e60410f3654
2017-06-06 14:16:57 +08:00
Ryan VanderMeulen
35b3731614 Backed out changeset c0b940487708 (bug 1359299) for causing intermittent Windows safebrowsing crashes. 2017-05-24 09:11:04 -04:00
DimiL
81b83a1f76 Bug 1359299 - Copy fullhash cache when update. r=hchang
After adopting the new thread model for safebrowsing, we will create a new
lookup cache for update so we can still check lookup cache at the same time.

Prefix set, completions will be generated when we open the new lookup cache
but it won't include cache, so we will loss cache after that.

This patch will copy cache data from old lookup cache to new lookup
cache while update.

MozReview-Commit-ID: L0WpiHOGIGm

--HG--
extra : rebase_source : ebaf249535561b87a983a3f24dacb8fbab7c096e
2017-05-23 09:19:06 +08:00
DimiL
c632a44157 Bug 1360480 - about:url-classifier: Cache information. r=francois
MozReview-Commit-ID: 4YXtb2KPgwL

--HG--
extra : rebase_source : ad9f897aa5772d6001a757e189730d427713f915
2017-05-17 10:32:33 +08:00
Chung-Sheng Fu
949647be91 Bug 1342900 - Remove usage of nsTArray<nsAutoPtr<>> in Classifier. r=dimi
MozReview-Commit-ID: 2SGPEuEg0lz

--HG--
extra : rebase_source : 88073453e592e78d969c00c67092ffefe4fb6581
2017-05-05 14:33:18 +08:00
DimiL
0170f8b6dc Bug 1333328 - Remove mTableFreshness and gFreshnessGuarantee. r=francois
Functions and Preference related to mTableFreshness and gFreshnessGuarantee
could be removed since we will no longer require them.

MozReview-Commit-ID: IAa0UHLSQ56

--HG--
extra : rebase_source : daa959340c92cfcb751fe8d0548d30762db3783e
2017-05-04 11:13:21 +08:00
DimiL
5aef415cc6 Bug 1333328 - Refactor cache miss handling mechanism for V2. r=francois
In this patch, we will make Safebrowsing V2 caching use the same algorithm as V4.
So we remove "mMissCache" for negative caching and TableFresness check for
positive caching.

But Safebrowsing V2 doesn't contain negative/positive cache duration information in
gethash response. So we hard-code a fixed value, 15 minutes, as cache duration.
In this way, we can sync the mechanism we handle caching for V2 and V4.

An extra effort for V2 here is that we need to manually record prefixes misses
because we won't get any response for those prefixes(implemented in
nsUrlClassifierLookupCallback::CacheMisses).

--HG--
extra : rebase_source : 0fb7938464ad24a01a51493ecb1b5c0197c16123
2017-05-04 09:38:14 +08:00
Michael Layzell
8b7ba021a4 Bug 1339537 - Part 6: Pass std::function values tree by const reference instead of by value, r=ehsan
MozReview-Commit-ID: PVAqU2DPs2
2017-04-27 12:44:57 -04:00
Henry Chang
a1d114de50 Bug 1354968 - Avoid concurrent access of mTableRefreshness. r=francois
mTableRefreshness, a non-thread-safe object, might be accessed on worker thread
and update thread cocurrently. To solve this issue, on update thread we only
insert data to mNewTableRefreshness and merge to mTableRefreshness on
the worker thread later.

MozReview-Commit-ID: 9WgoeYfWVfK

--HG--
extra : rebase_source : b7a8b4cd9d0fb1471cb81ee239f8343ff9a7b38a
2017-04-11 01:02:42 +08:00
dimi
7842472dad Bug 1311935 - P3. Implement safebrowsing v4 caching logic. r=francois
LookupCacheV4::Has implements safebrowsing v4 caching logic.
1. Check if fullhash match any prefix in local database:
  - If not, the URL is safe.
2. Check if prefix is in the cache(prefix is always the first 4-byte of
   the fullhash, Bug 1323953):
  - If not, send fullhash request
3. Check if fullhash is in the positive cache:
  - If fullhash is found and it is not expired, the URL is not safe.
  - If fullhash is found and it is expired, send fullhash request.
4. If fullhash is not found, check negative cache expired time:
  - If negative cache time is not expired, the URL is safe.
  - If negative cache time is expired, send fullhash request.

MozReview-Commit-ID: GRX7CP8ig49

--HG--
extra : rebase_source : 6d3ae8929b11731584810c44d46a1526f7d83e12
2017-04-10 14:21:08 +08:00
DimiL
b4f6db923f Bug 1311935 - P1. Make ActiveTables() work for safebrowsing v4. r=francois
This patch fixes that Classifier::ActiveTables doesn't return v4 tables.

Classifier::mActiveTablesCache is generated by scanning safebrowsing directory.
We use Classifier::ScanStoreDir to do the work, but it will ignore subdirectory.
Since v4 tables are stored in subdirectory 'google4', mActiveTablesCache doesn't
include v4 tables.

Fix this issue by checking subdirectory recursively in ScanStoreDir.

MozReview-Commit-ID: I6pa6e4bFND

--HG--
extra : rebase_source : 49d19101bec780e21db668b2daff3c46bbdb3fd2
2017-04-07 14:30:11 +08:00
Iris Hsiao
3d85c0330d Backed out 4 changesets (bug 1311935) for causing assertion crash by developer's request
Backed out changeset 27e624cd9479 (bug 1311935)
Backed out changeset 4c0381ab0990 (bug 1311935)
Backed out changeset 73587838ef16 (bug 1311935)
Backed out changeset a5a6c0f79733 (bug 1311935)
2017-04-11 11:04:54 +08:00
dimi
a58847dca1 Bug 1311935 - P3. Implement safebrowsing v4 caching logic. r=francois
LookupCacheV4::Has implements safebrowsing v4 caching logic.
1. Check if fullhash match any prefix in local database:
  - If not, the URL is safe.
2. Check if prefix is in the cache(prefix is always the first 4-byte of
   the fullhash, Bug 1323953):
  - If not, send fullhash request
3. Check if fullhash is in the positive cache:
  - If fullhash is found and it is not expired, the URL is not safe.
  - If fullhash is found and it is expired, send fullhash request.
4. If fullhash is not found, check negative cache expired time:
  - If negative cache time is not expired, the URL is safe.
  - If negative cache time is expired, send fullhash request.

MozReview-Commit-ID: GRX7CP8ig49

--HG--
extra : rebase_source : bcb5fa7aa2b7b116d862e3382447611424603c2d
2017-04-10 14:21:08 +08:00
DimiL
516a287f2b Bug 1311935 - P1. Make ActiveTables() work for safebrowsing v4. r=francois
This patch fixes that Classifier::ActiveTables doesn't return v4 tables.

Classifier::mActiveTablesCache is generated by scanning safebrowsing directory.
We use Classifier::ScanStoreDir to do the work, but it will ignore subdirectory.
Since v4 tables are stored in subdirectory 'google4', mActiveTablesCache doesn't
include v4 tables.

Fix this issue by checking subdirectory recursively in ScanStoreDir.

MozReview-Commit-ID: I6pa6e4bFND

--HG--
extra : rebase_source : 49d19101bec780e21db668b2daff3c46bbdb3fd2
2017-04-07 14:30:11 +08:00
Henry Chang
f3be63826d Bug 1339050 - Asynchronously apply safebrowsing DB update. r=francois,gcp
A new function Classifier::AsyncApplyUpdates() is implemented for async update.
Besides, all public Classifier interfaces become "worker thread only" and
we remove DBServiceWorker::ApplyUpdatesBackground/Foreground.

In DBServiceWorker::FinishUpdate, instead of calling Classifier::ApplyUpdates,
we call Classifier::AsyncApplyUpdates and install a callback for notifying
the update observer when update is finished. The callback will occur on the
caller thread (i.e. worker thread.)

As for the shutdown issue, when the main thread is notified to shut down,
we at first *synchronously* dispatch an event to the worker thread to
shut down the update thread. After getting synchronized with all other
threads, we send last two events "CancelUpdate" and "CloseDb" to notify
dangling update (i.e. BeginUpdate is called but FinishUpdate isn't)
and do cleanup work.

MozReview-Commit-ID: DXZvA2eFKlc

--HG--
extra : rebase_source : cd2e27a6b679d2c96e769854d1582ed2dcda12bb
2017-04-06 07:07:56 +08:00
dimi
b512891a41 Bug 1338033 - Remove URLCLASSIFIER_PREFIX_MATCH. r=francois
URLCLASSIFIER_PREFIX_MATCH is replaced by URLCLASSIFIER_FULL_MATCH(Bug 1311931)
so it should be removed.

MozReview-Commit-ID: 4F1B07bGwRP

--HG--
extra : rebase_source : 2d3e5cda14df4e874bf31145ccb5330acec369f7
2017-03-22 17:27:11 +08:00
Thomas Nguyen
e98654c1e4 Bug 1343416 - Use LCG when generating noise entries for Safe Browsing completions r=gcp
MozReview-Commit-ID: FxQH2haAcrf

--HG--
extra : rebase_source : 4fbd5bd18c4278682a52783b41bf0cc09aa91a2f
2017-03-09 10:59:52 +08:00
Henry Chang
0b68cc7689 Bug 1339760 - Split update process to background/foreground and run background on update thread **synchronously**. r=francois,gcp
MozReview-Commit-ID: J0phPC1nWsf

--HG--
extra : rebase_source : c212bd9fba1ef5951d238967a9b32e4fde0d718b
extra : source : 3daedf653e989c2d91deea785ed9598f2254e609
2017-02-22 17:25:26 +08:00
Henry Chang
6b96dcfdab Bug 1342397 - Properly reset stuff when failing to create LookupCache for update. r=gcp
MozReview-Commit-ID: 1ecI2lFIj2U

--HG--
extra : rebase_source : 49c5e460a1178253ce2afc1cdfd71dd66e370284
2017-02-25 00:22:39 +08:00
Dimi Lee
244358a9ac Bug 1335974 - Add more error codes to URLCLASSIFIER_UPDATE_ERROR. r=francois
MozReview-Commit-ID: AAd6Yu0AuUO

--HG--
extra : rebase_source : 63fdc2309348355d74b9bb738d75953d0b435b0f
2017-02-22 16:25:04 +08:00
Thomas Nguyen
3f7a3e400f Bug 1297962 - Add noise data when sending v4 gethash request r=francois
MozReview-Commit-ID: GbyvX7wcg8c
* * *
[mq]: 1297962_review

MozReview-Commit-ID: 1U2T0wq778R

--HG--
extra : rebase_source : 55f852204aa29b135360e461b8c152e45fecc7b6
2017-02-24 10:22:12 +08:00
Dimi Lee
b740658c54 Bug 1311931 - Add telemetry to measure full match rate for v2 and v4. r=francois
MozReview-Commit-ID: H9jAR82rgDh

--HG--
extra : rebase_source : 45b40d6d097882134eff38fe9a434e78cd4a56b1
2017-02-23 23:07:13 +08:00
Henry Chang
2b62e1dabf Bug 1338970 - Introduce 2-step DB update scheme. (background update then swap in) r=francois,gcp
MozReview-Commit-ID: 4C9N0tuSso0

--HG--
extra : rebase_source : 80561fb0beec7fc9f6e852d81984b239ae62700e
2017-02-21 14:18:29 +08:00
Tom Tromey
5f8f360823 Bug 1060419 - make log_print use Printf.h, r=froydnj
MozReview-Commit-ID: BIZ1GQEZ1vs

--HG--
extra : rebase_source : 2f1f0aa12493c44f352d9a7e8683e7bb72d2d75b
2016-12-15 20:16:31 -07:00
dimi
1f53a466a3 Bug 1336909 - Restrict URLCLASSIFIER_PREFIX_MATCH to profiles that have working V4. r=francois
MozReview-Commit-ID: L3lKgiohalH

--HG--
extra : rebase_source : 1ac11242c4db4b129fdb316deee92dd514b65f49
2017-02-08 15:18:35 +08:00
Dimi Lee
c6aec3beb1 Bug 1333257 - Only cache V2 misses when doing Safe Browsing lookups. r=francois
MozReview-Commit-ID: 6kvM6z5OnPw

--HG--
extra : rebase_source : c180ac1b6c840a6112f72abe3b7984503fad7082
2017-01-26 11:36:52 +08:00
dimi
4f2ec370e6 Bug 1328821 - hash completion request for v4 should not depend on table freshness. r=francois,henry
MozReview-Commit-ID: EIjDrnj1I4S

--HG--
extra : rebase_source : 83be77637f14753a8baf008af65c44e83b93508c
2017-01-17 08:33:08 +08:00
Henry Chang
5401fe7ad1 Bug 1312339 - LookupResult to support variable length partial hash. r=francois
MozReview-Commit-ID: DKwNCNKJAW

--HG--
extra : rebase_source : 082c8a1d88a00f0d568c56c7ed7214c0b0ed20ef
2016-12-16 14:34:32 +08:00
Thomas Nguyen
493ff90f5c Bug 1315386 - Make Safe Browsing code more shutdown-aware. r=francois,gcp.
MozReview-Commit-ID: ATCVfh5YLZl

--HG--
extra : rebase_source : d87cfae3838f1a26dc67d2546264e47768011a63
2016-11-25 16:02:37 +08:00
Carsten "Tomcat" Book
4945f89da4 Merge mozilla-central to autoland
--HG--
rename : browser/base/content/test/general/browser_bug1064280_changeUrlInPinnedTab.js => browser/base/content/test/tabs/browser_navigatePinnedTab.js
rename : js/src/jit-test/tests/wasm/spec.js => js/src/jit-test/tests/wasm/wast.js
extra : rebase_source : 30ebe4e98435a1fd70b26908584d608587d969d9
2016-11-24 16:47:36 +01:00
Thomas Nguyen
bb51dce698 Bug 1315893 - Add telemetry to measure update time for V2 and V4. r=francois
MozReview-Commit-ID: KtrVJXHXMo

--HG--
extra : rebase_source : 6aa4d0fd01afd24e9e45f55a5d45da255962ca92
2016-11-24 11:13:54 +08:00
Henry Chang
a5327a4bbb Bug 1319286 - Cache nsIUrlClassifierDBService.getTables result until next update. r=francois.
MozReview-Commit-ID: ItjTQNzCVED
2016-11-22 10:39:58 +08:00
Iris Hsiao
f2a7e74795 Backed out changeset b3574d03261b (bug 1319286) for GTest Assertion failures 2016-11-24 13:26:39 +08:00
Henry Chang
4c00c22835 Bug 1319286 - Cache nsIUrlClassifierDBService.getTables result until next update. r=francois.
MozReview-Commit-ID: ItjTQNzCVED

--HG--
extra : rebase_source : b652adb6fe696d6c25c423c8c7456247983d8050
extra : source : 3cb85811b61f8d16e2d5604af838d9cd982a5a4c
2016-11-22 10:39:58 +08:00
Henry Chang
1a53231595 Bug 1315097 - Build the provider dictionary on the main thread to be used everywhere. r=francois,gcp
MozReview-Commit-ID: Ft1deSNKuVB

--HG--
extra : rebase_source : 0cf746b1d2def35606f012c994ddbda0857ccf2d
2016-11-04 17:54:05 +08:00
Henry Chang
6cde2e6e52 Bug 1310142 - Preserve backup databases and raw table updates for diagnostic. r=francois
MozReview-Commit-ID: GeJoBrhuTgA

--HG--
extra : rebase_source : 02585bca9853a00bc041874a999b39e6fd7246ea
2016-10-14 17:58:18 +08:00
Thomas Nguyen
107627047f Bug 1298257 - Implement url matching for variable-length prefix set. r=dimi,gcp
MozReview-Commit-ID: 8Goh7yyAotN

--HG--
extra : rebase_source : 349cad2d7b4b285e7c5f75c88d649c358956ce63
2016-11-04 12:00:33 +08:00
dimi
84c9845881 Bug 1305581 - Verify that V4 updates were applied correctly by computing a checksum on the final result. r=francois
MozReview-Commit-ID: LNtFOVMVw2U

--HG--
extra : rebase_source : 8d60d4e49389c4a27e8ea431a9e3a4bce66ff969
2016-10-27 08:36:26 +08:00
DimiL
48ddedb04e Bug 1305780 - P1. Implement the update fail scheme for v4. r=gcp
MozReview-Commit-ID: LeVpVIUdmjc

--HG--
extra : rebase_source : bfc189b5a3f1cfc17bd269bcb7e7edad826769a3
2016-10-19 12:52:05 +08:00
Henry Chang
dc3bbda5bd Bug 1305484 - Save/load state and checksum to/from disk rather than prefs. r=dimi,francois
MozReview-Commit-ID: 4gmmrI9wY4c

--HG--
extra : rebase_source : f7f4462b7325bce9a92f747c54b448e2693e40ac
2016-10-13 15:22:08 +08:00
Dimi Lee
9e95041e88 Bug 1305801 - Part 5: Support SafeBrowsing v4 partial update. r=gcp
MozReview-Commit-ID: 7OEWLaZbotS

--HG--
extra : rebase_source : ab7a5eda234aaff3166e7082f9dc1f8b450d715b
2016-10-04 09:14:39 +08:00