Commit graph

51 commits

Author SHA1 Message Date
DimiL
e4d42d14a2 Bug 1359299 - V4 caches in LookupCache need to be copied around in copy constructor. r=hchang
MozReview-Commit-ID: AjzUUmQKiPW

--HG--
extra : rebase_source : 1c30ee5aa3548dce9b861a70660b1e60410f3654
2017-06-06 14:16:57 +08:00
Ryan VanderMeulen
35b3731614 Backed out changeset c0b940487708 (bug 1359299) for causing intermittent Windows safebrowsing crashes. 2017-05-24 09:11:04 -04:00
DimiL
81b83a1f76 Bug 1359299 - Copy fullhash cache when update. r=hchang
After adopting the new thread model for safebrowsing, we will create a new
lookup cache for update so we can still check lookup cache at the same time.

Prefix set, completions will be generated when we open the new lookup cache
but it won't include cache, so we will loss cache after that.

This patch will copy cache data from old lookup cache to new lookup
cache while update.

MozReview-Commit-ID: L0WpiHOGIGm

--HG--
extra : rebase_source : ebaf249535561b87a983a3f24dacb8fbab7c096e
2017-05-23 09:19:06 +08:00
DimiL
c632a44157 Bug 1360480 - about:url-classifier: Cache information. r=francois
MozReview-Commit-ID: 4YXtb2KPgwL

--HG--
extra : rebase_source : ad9f897aa5772d6001a757e189730d427713f915
2017-05-17 10:32:33 +08:00
DimiL
0170f8b6dc Bug 1333328 - Remove mTableFreshness and gFreshnessGuarantee. r=francois
Functions and Preference related to mTableFreshness and gFreshnessGuarantee
could be removed since we will no longer require them.

MozReview-Commit-ID: IAa0UHLSQ56

--HG--
extra : rebase_source : daa959340c92cfcb751fe8d0548d30762db3783e
2017-05-04 11:13:21 +08:00
DimiL
5aef415cc6 Bug 1333328 - Refactor cache miss handling mechanism for V2. r=francois
In this patch, we will make Safebrowsing V2 caching use the same algorithm as V4.
So we remove "mMissCache" for negative caching and TableFresness check for
positive caching.

But Safebrowsing V2 doesn't contain negative/positive cache duration information in
gethash response. So we hard-code a fixed value, 15 minutes, as cache duration.
In this way, we can sync the mechanism we handle caching for V2 and V4.

An extra effort for V2 here is that we need to manually record prefixes misses
because we won't get any response for those prefixes(implemented in
nsUrlClassifierLookupCallback::CacheMisses).

--HG--
extra : rebase_source : 0fb7938464ad24a01a51493ecb1b5c0197c16123
2017-05-04 09:38:14 +08:00
Michael Layzell
8b7ba021a4 Bug 1339537 - Part 6: Pass std::function values tree by const reference instead of by value, r=ehsan
MozReview-Commit-ID: PVAqU2DPs2
2017-04-27 12:44:57 -04:00
Thomas Nguyen
ae9e3b7638 Bug 1353853 - Cache preferences when doing channel classify r=francois
We will cache all preferences which will be read during classifing channel
- Store them into static variables nsUrlClassifierDBService
- Use a singleton class to manage/update preferrences in nsChannelClassifier

MozReview-Commit-ID: GvyBI3rVpYh

--HG--
extra : rebase_source : 0cec0724bd47f55c7b1666e700d172698a708efb
2017-04-18 17:00:27 +08:00
Henry Chang
a1d114de50 Bug 1354968 - Avoid concurrent access of mTableRefreshness. r=francois
mTableRefreshness, a non-thread-safe object, might be accessed on worker thread
and update thread cocurrently. To solve this issue, on update thread we only
insert data to mNewTableRefreshness and merge to mTableRefreshness on
the worker thread later.

MozReview-Commit-ID: 9WgoeYfWVfK

--HG--
extra : rebase_source : b7a8b4cd9d0fb1471cb81ee239f8343ff9a7b38a
2017-04-11 01:02:42 +08:00
DimiL
b4f6db923f Bug 1311935 - P1. Make ActiveTables() work for safebrowsing v4. r=francois
This patch fixes that Classifier::ActiveTables doesn't return v4 tables.

Classifier::mActiveTablesCache is generated by scanning safebrowsing directory.
We use Classifier::ScanStoreDir to do the work, but it will ignore subdirectory.
Since v4 tables are stored in subdirectory 'google4', mActiveTablesCache doesn't
include v4 tables.

Fix this issue by checking subdirectory recursively in ScanStoreDir.

MozReview-Commit-ID: I6pa6e4bFND

--HG--
extra : rebase_source : 49d19101bec780e21db668b2daff3c46bbdb3fd2
2017-04-07 14:30:11 +08:00
Iris Hsiao
3d85c0330d Backed out 4 changesets (bug 1311935) for causing assertion crash by developer's request
Backed out changeset 27e624cd9479 (bug 1311935)
Backed out changeset 4c0381ab0990 (bug 1311935)
Backed out changeset 73587838ef16 (bug 1311935)
Backed out changeset a5a6c0f79733 (bug 1311935)
2017-04-11 11:04:54 +08:00
DimiL
516a287f2b Bug 1311935 - P1. Make ActiveTables() work for safebrowsing v4. r=francois
This patch fixes that Classifier::ActiveTables doesn't return v4 tables.

Classifier::mActiveTablesCache is generated by scanning safebrowsing directory.
We use Classifier::ScanStoreDir to do the work, but it will ignore subdirectory.
Since v4 tables are stored in subdirectory 'google4', mActiveTablesCache doesn't
include v4 tables.

Fix this issue by checking subdirectory recursively in ScanStoreDir.

MozReview-Commit-ID: I6pa6e4bFND

--HG--
extra : rebase_source : 49d19101bec780e21db668b2daff3c46bbdb3fd2
2017-04-07 14:30:11 +08:00
Henry Chang
f3be63826d Bug 1339050 - Asynchronously apply safebrowsing DB update. r=francois,gcp
A new function Classifier::AsyncApplyUpdates() is implemented for async update.
Besides, all public Classifier interfaces become "worker thread only" and
we remove DBServiceWorker::ApplyUpdatesBackground/Foreground.

In DBServiceWorker::FinishUpdate, instead of calling Classifier::ApplyUpdates,
we call Classifier::AsyncApplyUpdates and install a callback for notifying
the update observer when update is finished. The callback will occur on the
caller thread (i.e. worker thread.)

As for the shutdown issue, when the main thread is notified to shut down,
we at first *synchronously* dispatch an event to the worker thread to
shut down the update thread. After getting synchronized with all other
threads, we send last two events "CancelUpdate" and "CloseDb" to notify
dangling update (i.e. BeginUpdate is called but FinishUpdate isn't)
and do cleanup work.

MozReview-Commit-ID: DXZvA2eFKlc

--HG--
extra : rebase_source : cd2e27a6b679d2c96e769854d1582ed2dcda12bb
2017-04-06 07:07:56 +08:00
Henry Chang
0b68cc7689 Bug 1339760 - Split update process to background/foreground and run background on update thread **synchronously**. r=francois,gcp
MozReview-Commit-ID: J0phPC1nWsf

--HG--
extra : rebase_source : c212bd9fba1ef5951d238967a9b32e4fde0d718b
extra : source : 3daedf653e989c2d91deea785ed9598f2254e609
2017-02-22 17:25:26 +08:00
Henry Chang
6b96dcfdab Bug 1342397 - Properly reset stuff when failing to create LookupCache for update. r=gcp
MozReview-Commit-ID: 1ecI2lFIj2U

--HG--
extra : rebase_source : 49c5e460a1178253ce2afc1cdfd71dd66e370284
2017-02-25 00:22:39 +08:00
Thomas Nguyen
3f7a3e400f Bug 1297962 - Add noise data when sending v4 gethash request r=francois
MozReview-Commit-ID: GbyvX7wcg8c
* * *
[mq]: 1297962_review

MozReview-Commit-ID: 1U2T0wq778R

--HG--
extra : rebase_source : 55f852204aa29b135360e461b8c152e45fecc7b6
2017-02-24 10:22:12 +08:00
Henry Chang
2b62e1dabf Bug 1338970 - Introduce 2-step DB update scheme. (background update then swap in) r=francois,gcp
MozReview-Commit-ID: 4C9N0tuSso0

--HG--
extra : rebase_source : 80561fb0beec7fc9f6e852d81984b239ae62700e
2017-02-21 14:18:29 +08:00
dimi
4f2ec370e6 Bug 1328821 - hash completion request for v4 should not depend on table freshness. r=francois,henry
MozReview-Commit-ID: EIjDrnj1I4S

--HG--
extra : rebase_source : 83be77637f14753a8baf008af65c44e83b93508c
2017-01-17 08:33:08 +08:00
Henry Chang
a5327a4bbb Bug 1319286 - Cache nsIUrlClassifierDBService.getTables result until next update. r=francois.
MozReview-Commit-ID: ItjTQNzCVED
2016-11-22 10:39:58 +08:00
Henry Chang
1a53231595 Bug 1315097 - Build the provider dictionary on the main thread to be used everywhere. r=francois,gcp
MozReview-Commit-ID: Ft1deSNKuVB

--HG--
extra : rebase_source : 0cf746b1d2def35606f012c994ddbda0857ccf2d
2016-11-04 17:54:05 +08:00
Henry Chang
6cde2e6e52 Bug 1310142 - Preserve backup databases and raw table updates for diagnostic. r=francois
MozReview-Commit-ID: GeJoBrhuTgA

--HG--
extra : rebase_source : 02585bca9853a00bc041874a999b39e6fd7246ea
2016-10-14 17:58:18 +08:00
DimiL
48ddedb04e Bug 1305780 - P1. Implement the update fail scheme for v4. r=gcp
MozReview-Commit-ID: LeVpVIUdmjc

--HG--
extra : rebase_source : bfc189b5a3f1cfc17bd269bcb7e7edad826769a3
2016-10-19 12:52:05 +08:00
Henry Chang
dc3bbda5bd Bug 1305484 - Save/load state and checksum to/from disk rather than prefs. r=dimi,francois
MozReview-Commit-ID: 4gmmrI9wY4c

--HG--
extra : rebase_source : f7f4462b7325bce9a92f747c54b448e2693e40ac
2016-10-13 15:22:08 +08:00
Dimi Lee
3ef7d52f53 Bug 1305801 - Part 4: Store variable-length prefix to disk. r=francois, r=gcp
MozReview-Commit-ID: BMTGtgMuQdg

--HG--
extra : rebase_source : dc4975d9ac8d14ecfabf6fc1229fdb8e863cde46
2016-09-19 11:51:01 +08:00
Phil Ringnalda
35869db574 Backed out 5 changesets (bug 1305801) for ASan gtest bustage
Backed out changeset 0c95d5dec6d9 (bug 1305801)
Backed out changeset bca0e706dbc5 (bug 1305801)
Backed out changeset def8da367beb (bug 1305801)
Backed out changeset 56ceae52d847 (bug 1305801)
Backed out changeset 14457cc4c325 (bug 1305801)
2016-10-03 22:14:49 -07:00
Dimi Lee
f0aab4a2ce Bug 1305801 - Part 4: Store variable-length prefix to disk. r=francois, r=gcp
MozReview-Commit-ID: BMTGtgMuQdg

--HG--
extra : rebase_source : 6f613883071de209f53596060b637e928f458978
2016-09-19 11:51:01 +08:00
Henry Chang
6e64c9ee89 Bug 1254763 - Part 1: Use per-provider directory for V4 databases. r=dimi,gcp
MozReview-Commit-ID: KZGWIcKAQyW

--HG--
extra : rebase_source : f153e87ce132f0066d6c8e21398e7abaaaff6375
2016-09-02 12:00:00 +08:00
dimi
2fa583dc0b Bug 1037560 - P1. Safebrowsing pleasereset resets all tables. r=gcp
MozReview-Commit-ID: C5GkBBDAhn3

--HG--
extra : rebase_source : d51e94749549db975651e4bc2e4bde2f38b1a9d0
2016-08-23 08:24:07 +08:00
dimi
365c242c26 Bug 1254766 - Stop caching Safe Browsing completions to disk. r=gcp
MozReview-Commit-ID: 8Qnc8yymgDL
2016-08-11 08:17:39 +08:00
dimi
0faebe8b52 Bug 1287024 - Remove unused mHashStores variable in Classifier.cpp. r=francois
MozReview-Commit-ID: 2EIDaRHKW8Q

--HG--
extra : rebase_source : e1a4a8c578ec8e5c2f5869cbd775bfbba0c977bf
2016-07-15 12:16:26 +08:00
Gian-Carlo Pascutto
31cf95e1cd Bug 1175562 - Persist last update time for SafeBrowsing. r=francois 2015-10-06 13:53:07 +02:00
Birunthan Mohanathas
a8939590de Bug 1182996 - Fix and add missing namespace comments. rs=ehsan
The bulk of this commit was generated by running:

  run-clang-tidy.py \
    -checks='-*,llvm-namespace-comment' \
    -header-filter=^/.../mozilla-central/.* \
    -fix
2015-07-13 08:25:42 -07:00
Monica Chew
c283b146cd Bug 1120499: Proxy DoLocalLookup to the worker thread (r=gcp) 2015-01-13 17:09:13 -08:00
Monica Chew
55a8118ad1 Bug 1108009 - Make synchronous interface nsIURIClassifier.ClassifyLocal. r=gcp 2014-12-18 10:18:09 -08:00
Monica Chew
9b092b1051 Bug 985623: Force url classifier clients to specify which tables to lookup, add a pref to skip hash completion checks (r=gcp) 2014-03-20 14:25:35 -07:00
Gian-Carlo Pascutto
0da033e962 Bug 825891 - Remove the code for per-client randomization in the url-classifier. r=dcamp 2013-01-08 16:43:33 +01:00
Daniel Holbert
5f7e7fca0d Bug 825627, part 1: Make nsUrlClassifierDBService.cpp's "gethashNoise" & "aCount" variables as unsigned, since counts are unsigned by definition. r=gcp 2013-01-02 09:05:19 -08:00
Saurabh Anand
6de7befa37 Bug 818817 - Fix some compiler warnings, r=Ms2ger 2012-12-09 22:53:19 +05:30
Gian-Carlo Pascutto
abb9245bcc Bug 727370 - Recover correctly from a Reset during an update. r=dcamp 2012-09-28 18:31:19 +02:00
Gian-Carlo Pascutto
7530b9f562 Bug 727370 - Make SafeBrowsing updates atomic transactions. r=dcamp 2012-09-28 18:31:18 +02:00
Ehsan Akhgari
e368dc9c85 Bug 579517 - Part 1: Automated conversion of NSPR numeric types to stdint types in Gecko; r=bsmedberg
This patch was generated by a script.  Here's the source of the script for
future reference:

function convert() {
echo "Converting $1 to $2..."
find . ! -wholename "*nsprpub*" \
       ! -wholename "*security/nss*" \
       ! -wholename "*/.hg*" \
       ! -wholename "obj-ff-dbg*" \
       ! -name nsXPCOMCID.h \
       ! -name prtypes.h \
         -type f \
      \( -iname "*.cpp" \
         -o -iname "*.h" \
         -o -iname "*.c" \
         -o -iname "*.cc" \
         -o -iname "*.idl" \
         -o -iname "*.ipdl" \
         -o -iname "*.ipdlh" \
         -o -iname "*.mm" \) | \
    xargs -n 1 sed -i -e "s/\b$1\b/$2/g"
}

convert PRInt8 int8_t
convert PRUint8 uint8_t
convert PRInt16 int16_t
convert PRUint16 uint16_t
convert PRInt32 int32_t
convert PRUint32 uint32_t
convert PRInt64 int64_t
convert PRUint64 uint64_t

convert PRIntn int
convert PRUintn unsigned

convert PRSize size_t

convert PROffset32 int32_t
convert PROffset64 int64_t

convert PRPtrdiff ptrdiff_t

convert PRFloat64 double
2012-08-22 11:56:38 -04:00
Gian-Carlo Pascutto
328bf6ddf8 Bug 673470 - Update the new SafeBrowsing code to MPL 2.0. r=dcamp 2012-08-15 09:10:56 +02:00
Gian-Carlo Pascutto
52208070f2 Bug 768871 - Provide an option to disable per-client randomization. r=dcamp 2012-08-15 09:07:10 +02:00
Gian-Carlo Pascutto
b5c3234491 Bug 673470 - Optimize input buffer size. Cache active tables. r=dcamp 2012-08-15 09:04:31 +02:00
Gian-Carlo Pascutto
f6e66dd24e Bug 673470 - Replace the sqlite safeb store with a flat file. r=dcamp 2012-08-15 09:04:19 +02:00
Gian-Carlo Pascutto
08beb49296 Backout 173f90d397a8 (Bug 673470). rs=dcamp a=mfinkle 2012-04-20 07:46:47 +02:00
Gian-Carlo Pascutto
bdd93f6030 Backout db52b4916cde (Bug 673470). rs=dcamp a=mfinkle 2012-04-20 07:46:46 +02:00
Gian-Carlo Pascutto
dbd7ac64e0 Bug 673470 - Optimize input buffer size. Cache active tables. r=dcamp 2012-02-03 08:32:32 +01:00
Gian-Carlo Pascutto
a784af5a5b Bug 673470 - Replace the SQLite SafeBrowsing store with an optimized store. r=dcamp 2011-12-06 19:03:05 +01:00
Gian-Carlo Pascutto
99a251d9ae Backout 44a0dc4fb9ff (Bug 673470) for Talos regression. 2012-02-01 20:10:02 +01:00