Commit graph

75 commits

Author SHA1 Message Date
Francois Marier
e33ff43117 Bug 1467581 - Replace all nsAutoPtrs with UniquePtrs in Safe Browsing code. r=gcp
This should not change anything since we were using nsAutoPtrs properly,
but we may as well clean this up given the recent move to smart pointers
for everything.

MozReview-Commit-ID: FWS54SYNiBm

--HG--
extra : rebase_source : ade51b42049782e62f661c84b6d701f9ec33cc77
2018-06-18 13:54:59 -07:00
Francois Marier
79f1e9e5a8 Bug 1467581 - Remove the use of default captures in closures. r=gcp
Explicitly specify the arguments to copy to avoid making a copy of
a dangling `this` pointer.

Convert nsUrlClassifierDBService::mClassifier to a RefPtr since
the update closure might need to continue to access its members
after it's been released by the main thread.

MozReview-Commit-ID: CPio3n9MmsK

--HG--
extra : rebase_source : d7a8b207336209ee37441f3429bc608140e404ae
2018-06-07 17:22:32 -07:00
Francois Marier
e0fce2a920 Bug 1434206 - Keep LookupResult objects in smart pointers. r=gcp
Replace raw pointers to LookupResult with RefPtrs and eplace the
nsAutoPtr objects + raw pointers params with UniquePtrs.

Also remove unnecessarily paranoid OOM checks when creating single
LookupResult objects since those are pretty small.

MozReview-Commit-ID: G85RNnAat6H

--HG--
extra : rebase_source : a8f6a1ff1e24663d428c8d894cb624e1c67e1bd3
2018-06-05 13:15:03 -07:00
Francois Marier
8795e4c222 Bug 1434206 - Keep CacheResult objects in smart pointers. r=gcp
Some of the objects were kept in UniquePtr and nsAutoPtr but that
seemed unnecessary complexity given that we can simply use RefPtr
everywhere.

It's also possible to make all of the CacheResult arrays const
since we don't ever modify the elements once they are added.

MozReview-Commit-ID: 5OlcbkQLrGb

--HG--
extra : rebase_source : d58963443d97296236056c69881846f8e67ee3e9
2018-06-01 15:49:14 -07:00
Francois Marier
cde64ec41a Bug 1434206 - Add const to functions and members that can take it. r=gcp
MozReview-Commit-ID: D8IQoLZkFaA

--HG--
extra : rebase_source : 107a936dc4bfd000b84cecdabc017723cd920420
2018-05-16 15:39:33 -07:00
Francois Marier
214074e8d4 Bug 1434206 - Keep TableUpdate objects in smart pointers. r=gcp
Manually keeping tabs on the lifetime of these objects is a pain
and is the likely source of some of our crashes. I suspect we might
also be leaking memory.

This change creates an explicit copy of the main array into the
update thread to avoid using a non-thread-safe shared data
structure. This is a shallow copy. Only the pointers to the
TableUpdates are copied, which means one pointer per list (e.g. 5
in total for google4 in a new profile).

MozReview-Commit-ID: 221d6GkKt0M

--HG--
extra : rebase_source : e1b81f11bb9b41e465571a95845079f455b5868e
2018-06-01 15:48:48 -07:00
Francois Marier
30207d91aa Bug 1434206 - Add const to members and functions that can take it. r=gcp
MozReview-Commit-ID: B2aaQTttPAV

--HG--
extra : rebase_source : bbd8d9c734660a4380922ed9b76f1af1de427f3f
2018-05-16 15:26:14 -07:00
Francois Marier
deb19204a3 Bug 1434206 - Remove unused and undefined functions. r=gcp
mProtocolV2 is still used to skip the caching of misses on V4:

https://searchfox.org/mozilla-central/rev/d4b9e50875ad7e5d20f2fee6a53418315f6dfcc0/toolkit/components/url-classifier/nsUrlClassifierDBService.cpp#1353-1357

MozReview-Commit-ID: 2cHh9JiZuHh

--HG--
extra : rebase_source : 8c571faf246da2666c6ce1e14d4a701e17d88f89
2018-05-28 14:39:32 -07:00
dimi
48a63f5601 Bug 1408631 - Release SafeBrowsing lookupcache in worker thread while shutdown. r=francois
MozReview-Commit-ID: HuPUyIDFLPX

--HG--
extra : rebase_source : d6e4f5bbcf96c97541792e23447f0810150c5ac9
2017-10-20 10:18:59 +08:00
Thomas Nguyen
e70d76485c Bug 1394031 - Remove mCryptoHash members of nsUrlClassifierDBServiceWorker and ProtocolParser
The usage of cryptoHash consists of a complete set of Init, Update, and Finish, there's
no reason to keep it around

MozReview-Commit-ID: 7bT9IsWEM5m
2017-09-14 10:51:55 +08:00
Thomas Nguyen
9d1887c28c Bug 1363038 - Remove synchronous dispatch when worker is racing against shutdown r=hchang
MozReview-Commit-ID: JOkOukz1fov

--HG--
extra : rebase_source : 9e4204cebcd4615078019a51818250ff8125efe1
2017-08-22 14:51:44 +08:00
Francois Marier
adc9339128 Bug 1385484 - Add the login reputation whitelist behind a pref. r=hchang
MozReview-Commit-ID: Dbtl3WL5tMJ

--HG--
extra : rebase_source : d56e3db985865d0b513184d8ad41e2b750e7733e
2017-08-08 16:23:10 -07:00
Wes Kocher
207dd28c61 Backed out 4 changesets (bug 1385484) for build failures a=backout
Backed out changeset 401c2744f16b (bug 1385484)
Backed out changeset 2ff443a0c744 (bug 1385484)
Backed out changeset 1a1c476a9aaa (bug 1385484)
Backed out changeset 87ec064a3681 (bug 1385484)

MozReview-Commit-ID: D9WGXiXaNoZ

--HG--
extra : rebase_source : ebab6135905e972abf7eeab9af8db41454687373
2017-08-22 17:22:41 -07:00
Francois Marier
542d943dc4 Bug 1385484 - Add the login reputation whitelist behind a pref. r=hchang
MozReview-Commit-ID: Dbtl3WL5tMJ

--HG--
extra : rebase_source : 8f163e6871559a36e02b2c576970b4f1624da3b4
2017-08-08 16:23:10 -07:00
Andrew McCreight
1471488de6 Bug 1367521 - Fix misspellings of "delegate". r=dholbert
MozReview-Commit-ID: KqOnp8rpCZD

--HG--
extra : rebase_source : 90bf8965737a1e808a5d2305ad395bdc99102cd1
2017-05-24 10:43:11 -07:00
DimiL
c632a44157 Bug 1360480 - about:url-classifier: Cache information. r=francois
MozReview-Commit-ID: 4YXtb2KPgwL

--HG--
extra : rebase_source : ad9f897aa5772d6001a757e189730d427713f915
2017-05-17 10:32:33 +08:00
DimiL
5aef415cc6 Bug 1333328 - Refactor cache miss handling mechanism for V2. r=francois
In this patch, we will make Safebrowsing V2 caching use the same algorithm as V4.
So we remove "mMissCache" for negative caching and TableFresness check for
positive caching.

But Safebrowsing V2 doesn't contain negative/positive cache duration information in
gethash response. So we hard-code a fixed value, 15 minutes, as cache duration.
In this way, we can sync the mechanism we handle caching for V2 and V4.

An extra effort for V2 here is that we need to manually record prefixes misses
because we won't get any response for those prefixes(implemented in
nsUrlClassifierLookupCallback::CacheMisses).

--HG--
extra : rebase_source : 0fb7938464ad24a01a51493ecb1b5c0197c16123
2017-05-04 09:38:14 +08:00
Thomas Nguyen
52ed9fd77e Bug 1351472 - Skip AddNoise if the table is unknown or disallowed to getHash r=francois
MozReview-Commit-ID: GMWs2UpiyiP

--HG--
extra : rebase_source : 9b24bf9fa6ad5aa12077f6e4ffa2ac65b0862b40
2017-04-28 15:26:54 +08:00
Thomas Nguyen
ae9e3b7638 Bug 1353853 - Cache preferences when doing channel classify r=francois
We will cache all preferences which will be read during classifing channel
- Store them into static variables nsUrlClassifierDBService
- Use a singleton class to manage/update preferrences in nsChannelClassifier

MozReview-Commit-ID: GvyBI3rVpYh

--HG--
extra : rebase_source : 0cec0724bd47f55c7b1666e700d172698a708efb
2017-04-18 17:00:27 +08:00
dimi
127351f6ef Bug 1311935 - P2. Process fullHashes.find response. r=francois
This patch includes following changes:

1. nsUrlClassifierHashCompleter.js
   nsUrlClassifierHashCompleter.idl
   - Add completionV4 interface for hashCompleter to pass response data to
     DB service.
   - Process response data includes negative cache duration, matched full
     hashes and cache duration for each match. Full matches are passed through
     nsIFullHashMatch interface.
   - Change _requests.responses from array contains matched fullhashes to
     dictionary so that it can store additional information likes negative cache
     duration.
2. nsUrlClassifierDBService.cpp
   - Implement CompletionV4 interface, store response data to CacheResultV4
     object. Expired duration to expired time is handled here.
   - Add CacheResultToTableUpdate function to convert V2 & V4 cache result
     to TableUpdate object.
3. LookupCache.h
   - Extend CacheResult to CacheResultV2 and CacheResultV4 so we can store
     response data in CompletionV2 and CompletionV4.
4. HashStore.h
   - Add API and member variable in TableUpdateV4 to store response data.
     TableUpdate object is used by DB service to pass update data or gethash
     response to Classifier, so we need to extend TableUpdateV4 to be able
     to store fullHashes.find response.
6. Entry.h
   - Define the structure about how we cache fullHashes.find response.

MozReview-Commit-ID: FV4yAl2SAc6

--HG--
extra : rebase_source : 02676ded9bc0580c24b4c906199a4abaf004e296
2017-04-11 11:50:48 +08:00
Iris Hsiao
3d85c0330d Backed out 4 changesets (bug 1311935) for causing assertion crash by developer's request
Backed out changeset 27e624cd9479 (bug 1311935)
Backed out changeset 4c0381ab0990 (bug 1311935)
Backed out changeset 73587838ef16 (bug 1311935)
Backed out changeset a5a6c0f79733 (bug 1311935)
2017-04-11 11:04:54 +08:00
dimi
5112db082c Bug 1311935 - P2. Process fullHashes.find response. r=francois
This patch includes following changes:

1. nsUrlClassifierHashCompleter.js
   nsUrlClassifierHashCompleter.idl
   - Add completionV4 interface for hashCompleter to pass response data to
     DB service.
   - Process response data includes negative cache duration, matched full
     hashes and cache duration for each match. Full matches are passed through
     nsIFullHashMatch interface.
   - Change _requests.responses from array contains matched fullhashes to
     dictionary so that it can store additional information likes negative cache
     duration.
2. nsUrlClassifierDBService.cpp
   - Implement CompletionV4 interface, store response data to CacheResultV4
     object. Expired duration to expired time is handled here.
   - Add CacheResultToTableUpdate function to convert V2 & V4 cache result
     to TableUpdate object.
3. LookupCache.h
   - Extend CacheResult to CacheResultV2 and CacheResultV4 so we can store
     response data in CompletionV2 and CompletionV4.
4. HashStore.h
   - Add API and member variable in TableUpdateV4 to store response data.
     TableUpdate object is used by DB service to pass update data or gethash
     response to Classifier, so we need to extend TableUpdateV4 to be able
     to store fullHashes.find response.
6. Entry.h
   - Define the structure about how we cache fullHashes.find response.

MozReview-Commit-ID: KgR1NASl7GC

--HG--
extra : rebase_source : 424db14e4af2ffd691c384414d50f64083d5d20b
2017-04-10 16:20:09 +08:00
Henry Chang
f3be63826d Bug 1339050 - Asynchronously apply safebrowsing DB update. r=francois,gcp
A new function Classifier::AsyncApplyUpdates() is implemented for async update.
Besides, all public Classifier interfaces become "worker thread only" and
we remove DBServiceWorker::ApplyUpdatesBackground/Foreground.

In DBServiceWorker::FinishUpdate, instead of calling Classifier::ApplyUpdates,
we call Classifier::AsyncApplyUpdates and install a callback for notifying
the update observer when update is finished. The callback will occur on the
caller thread (i.e. worker thread.)

As for the shutdown issue, when the main thread is notified to shut down,
we at first *synchronously* dispatch an event to the worker thread to
shut down the update thread. After getting synchronized with all other
threads, we send last two events "CancelUpdate" and "CloseDb" to notify
dangling update (i.e. BeginUpdate is called but FinishUpdate isn't)
and do cleanup work.

MozReview-Commit-ID: DXZvA2eFKlc

--HG--
extra : rebase_source : cd2e27a6b679d2c96e769854d1582ed2dcda12bb
2017-04-06 07:07:56 +08:00
Henry Chang
0b68cc7689 Bug 1339760 - Split update process to background/foreground and run background on update thread **synchronously**. r=francois,gcp
MozReview-Commit-ID: J0phPC1nWsf

--HG--
extra : rebase_source : c212bd9fba1ef5951d238967a9b32e4fde0d718b
extra : source : 3daedf653e989c2d91deea785ed9598f2254e609
2017-02-22 17:25:26 +08:00
Dimi Lee
2dae4fbcea Bug 1329593 - Compiling with mingw-w64 is broken with conflict-with-previous-declaration errors. r=gcp
--HG--
extra : rebase_source : 07e5b745c371aeca4de55abd2a99b61c915113f0
2017-01-12 23:00:22 +08:00
Kershaw Chang
ccfc159353 Bug 1141814 - Part 1: Enable to update TP list if TP is disabled. r=francois 2016-11-30 23:12:00 -05:00
Thomas Nguyen
493ff90f5c Bug 1315386 - Make Safe Browsing code more shutdown-aware. r=francois,gcp.
MozReview-Commit-ID: ATCVfh5YLZl

--HG--
extra : rebase_source : d87cfae3838f1a26dc67d2546264e47768011a63
2016-11-25 16:02:37 +08:00
Henry Chang
6cde2e6e52 Bug 1310142 - Preserve backup databases and raw table updates for diagnostic. r=francois
MozReview-Commit-ID: GeJoBrhuTgA

--HG--
extra : rebase_source : 02585bca9853a00bc041874a999b39e6fd7246ea
2016-10-14 17:58:18 +08:00
Henry
36fa608182 Bug 1307541 - ProtocolParserProtobuf to init and return update time properly. r=francois.
MozReview-Commit-ID: CmVWVKUeunJ

--HG--
extra : rebase_source : e8cd28a805b892fd26ca9bede3523a04f1bb5686
2016-10-18 14:45:21 +08:00
Thomas Nguyen
73a98d5b79 Bug 999484 - Test FindInReadable calls in safebrowsing code. r=gcp
MozReview-Commit-ID: C27J4dh9s6T

--HG--
extra : rebase_source : d9a8ae7ea7073227b30ea2d74ea69a988b46b18e
2016-09-01 10:48:17 +08:00
Thomas Nguyen
b061760f96 Bug 1120728 - remove nsIUrlClassifierDBServiceWorker. r=francois,gcp
MozReview-Commit-ID: G3HeQ5pecrE

--HG--
extra : rebase_source : 39dbf5f60644f08606ab3cf3cc604fdfbcfb81cb
2016-08-11 11:15:24 +08:00
Yoshi Huang
e24cc195d9 Bug 1274893 - remove -forbid- list from SafeBrowsing. r=francois 2016-07-11 16:11:32 +08:00
Tobias Schneider
9dd3cff0b6 Bug 1237198 - Block SWFs on the content blocking list hosted on the Shavar service. r=francois 2016-03-29 00:37:44 -07:00
Francois Marier
5f9205dae9 Bug 1164518 - Avoid unnecessary DB updates when caching Safe Browsing results. r=gcp
MozReview-Commit-ID: HYNaTdCRohL

--HG--
extra : rebase_source : 681b33066b21719296c9d33f8d71a72335513188
2016-03-04 12:33:20 -08:00
Francois Marier
67506f65a6 Bug 1216723 - Add a new -forbid- Safe Browsing list type. r=gcp,r=smaug 2015-11-20 20:24:50 -08:00
Nathan Froyd
01583602a9 Bug 1207245 - part 6 - rename nsRefPtr<T> to RefPtr<T>; r=ehsan; a=Tomcat
The bulk of this commit was generated with a script, executed at the top
level of a typical source code checkout.  The only non-machine-generated
part was modifying MFBT's moz.build to reflect the new naming.

CLOSED TREE makes big refactorings like this a piece of cake.

 # The main substitution.
find . -name '*.cpp' -o -name '*.cc' -o -name '*.h' -o -name '*.mm' -o -name '*.idl'| \
    xargs perl -p -i -e '
 s/nsRefPtr\.h/RefPtr\.h/g; # handle includes
 s/nsRefPtr ?</RefPtr</g;   # handle declarations and variables
'

 # Handle a special friend declaration in gfx/layers/AtomicRefCountedWithFinalize.h.
perl -p -i -e 's/::nsRefPtr;/::RefPtr;/' gfx/layers/AtomicRefCountedWithFinalize.h

 # Handle nsRefPtr.h itself, a couple places that define constructors
 # from nsRefPtr, and code generators specially.  We do this here, rather
 # than indiscriminantly s/nsRefPtr/RefPtr/, because that would rename
 # things like nsRefPtrHashtable.
perl -p -i -e 's/nsRefPtr/RefPtr/g' \
     mfbt/nsRefPtr.h \
     xpcom/glue/nsCOMPtr.h \
     xpcom/base/OwningNonNull.h \
     ipc/ipdl/ipdl/lower.py \
     ipc/ipdl/ipdl/builtin.py \
     dom/bindings/Codegen.py \
     python/lldbutils/lldbutils/utils.py

 # In our indiscriminate substitution above, we renamed
 # nsRefPtrGetterAddRefs, the class behind getter_AddRefs.  Fix that up.
find . -name '*.cpp' -o -name '*.h' -o -name '*.idl' | \
    xargs perl -p -i -e 's/nsRefPtrGetterAddRefs/RefPtrGetterAddRefs/g'

if [ -d .git ]; then
    git mv mfbt/nsRefPtr.h mfbt/RefPtr.h
else
    hg mv mfbt/nsRefPtr.h mfbt/RefPtr.h
fi

--HG--
rename : mfbt/nsRefPtr.h => mfbt/RefPtr.h
2015-10-18 01:24:48 -04:00
Birunthan Mohanathas
a8939590de Bug 1182996 - Fix and add missing namespace comments. rs=ehsan
The bulk of this commit was generated by running:

  run-clang-tidy.py \
    -checks='-*,llvm-namespace-comment' \
    -header-filter=^/.../mozilla-central/.* \
    -fix
2015-07-13 08:25:42 -07:00
Andrea Marchesini
e6f385fb3d Bug 1148527 - Indentation fix after bug 1145631, r=ehsan 2015-03-27 18:52:19 +00:00
Ehsan Akhgari
883849ee32 Bug 1145631 - Part 1: Replace MOZ_OVERRIDE and MOZ_FINAL with override and final in the tree; r=froydnj
This patch was automatically generated using the following script:

function convert() {
echo "Converting $1 to $2..."
find . \
       ! -wholename "*/.git*" \
       ! -wholename "obj-ff-dbg*" \
         -type f \
      \( -iname "*.cpp" \
         -o -iname "*.h" \
         -o -iname "*.c" \
         -o -iname "*.cc" \
         -o -iname "*.idl" \
         -o -iname "*.ipdl" \
         -o -iname "*.ipdlh" \
         -o -iname "*.mm" \) | \
    xargs -n 1 sed -i -e "s/\b$1\b/$2/g"
}

convert MOZ_OVERRIDE override
convert MOZ_FINAL final
2015-03-21 12:28:04 -04:00
Monica Chew
c283b146cd Bug 1120499: Proxy DoLocalLookup to the worker thread (r=gcp) 2015-01-13 17:09:13 -08:00
Monica Chew
152c9afdc4 Bug 1100024: ClassifyLocal should not lookup in any tables if safebrowsing is disabled (r=gcp) 2015-01-09 13:25:02 -08:00
Monica Chew
55a8118ad1 Bug 1108009 - Make synchronous interface nsIURIClassifier.ClassifyLocal. r=gcp 2014-12-18 10:18:09 -08:00
Monica Chew
1ce06a3399 Bug 1024610: Register tracking protection list and hook it up in nsChannelClassifier (r=gcp) 2014-07-02 12:41:00 -07:00
Monica Chew
386ebd5ee3 Bug 1021419: Implement per-table update and gethash requests 2014-06-12 10:24:03 -07:00
Monica Chew
d1212d39f6 Bug 998396: Fix gethash completions, urlclassifier.malware_table and urlclassifier.phish_table may be comma-separated lists (r=gcp) 2014-04-22 09:46:46 -07:00
Ed Morley
74fa961dfe Backed out changeset 73bb0c592bf1 (bug 998396) for debug build failures; CLOSED TREE 2014-04-22 17:37:01 +01:00
Monica Chew
ade3e9a917 Bug 998396: Fix gethash completions, urlclassifier.malware_table and urlclassifier.phish_table may be comma-separated lists (r=gcp) 2014-04-22 09:14:51 -07:00
Ed Morley
92378320da Backed out changeset 96e274a243ff (bug 998396) for build failures; CLOSED TREE 2014-04-22 16:36:39 +01:00
Monica Chew
efa58c585d Bug 998396: Fix gethash completions, urlclassifier.malware_table and urlclassifier.phish_table may be comma-separated lists (r=gcp) 2014-04-22 08:13:59 -07:00
Monica Chew
9b092b1051 Bug 985623: Force url classifier clients to specify which tables to lookup, add a pref to skip hash completion checks (r=gcp) 2014-03-20 14:25:35 -07:00