After adopting the new thread model for safebrowsing, we will create a new
lookup cache for update so we can still check lookup cache at the same time.
Prefix set, completions will be generated when we open the new lookup cache
but it won't include cache, so we will loss cache after that.
This patch will copy cache data from old lookup cache to new lookup
cache while update.
MozReview-Commit-ID: L0WpiHOGIGm
--HG--
extra : rebase_source : ebaf249535561b87a983a3f24dacb8fbab7c096e
Functions and Preference related to mTableFreshness and gFreshnessGuarantee
could be removed since we will no longer require them.
MozReview-Commit-ID: IAa0UHLSQ56
--HG--
extra : rebase_source : daa959340c92cfcb751fe8d0548d30762db3783e
In this patch, we will make Safebrowsing V2 caching use the same algorithm as V4.
So we remove "mMissCache" for negative caching and TableFresness check for
positive caching.
But Safebrowsing V2 doesn't contain negative/positive cache duration information in
gethash response. So we hard-code a fixed value, 15 minutes, as cache duration.
In this way, we can sync the mechanism we handle caching for V2 and V4.
An extra effort for V2 here is that we need to manually record prefixes misses
because we won't get any response for those prefixes(implemented in
nsUrlClassifierLookupCallback::CacheMisses).
--HG--
extra : rebase_source : 0fb7938464ad24a01a51493ecb1b5c0197c16123
mTableRefreshness, a non-thread-safe object, might be accessed on worker thread
and update thread cocurrently. To solve this issue, on update thread we only
insert data to mNewTableRefreshness and merge to mTableRefreshness on
the worker thread later.
MozReview-Commit-ID: 9WgoeYfWVfK
--HG--
extra : rebase_source : b7a8b4cd9d0fb1471cb81ee239f8343ff9a7b38a
LookupCacheV4::Has implements safebrowsing v4 caching logic.
1. Check if fullhash match any prefix in local database:
- If not, the URL is safe.
2. Check if prefix is in the cache(prefix is always the first 4-byte of
the fullhash, Bug 1323953):
- If not, send fullhash request
3. Check if fullhash is in the positive cache:
- If fullhash is found and it is not expired, the URL is not safe.
- If fullhash is found and it is expired, send fullhash request.
4. If fullhash is not found, check negative cache expired time:
- If negative cache time is not expired, the URL is safe.
- If negative cache time is expired, send fullhash request.
MozReview-Commit-ID: GRX7CP8ig49
--HG--
extra : rebase_source : 6d3ae8929b11731584810c44d46a1526f7d83e12
This patch fixes that Classifier::ActiveTables doesn't return v4 tables.
Classifier::mActiveTablesCache is generated by scanning safebrowsing directory.
We use Classifier::ScanStoreDir to do the work, but it will ignore subdirectory.
Since v4 tables are stored in subdirectory 'google4', mActiveTablesCache doesn't
include v4 tables.
Fix this issue by checking subdirectory recursively in ScanStoreDir.
MozReview-Commit-ID: I6pa6e4bFND
--HG--
extra : rebase_source : 49d19101bec780e21db668b2daff3c46bbdb3fd2
LookupCacheV4::Has implements safebrowsing v4 caching logic.
1. Check if fullhash match any prefix in local database:
- If not, the URL is safe.
2. Check if prefix is in the cache(prefix is always the first 4-byte of
the fullhash, Bug 1323953):
- If not, send fullhash request
3. Check if fullhash is in the positive cache:
- If fullhash is found and it is not expired, the URL is not safe.
- If fullhash is found and it is expired, send fullhash request.
4. If fullhash is not found, check negative cache expired time:
- If negative cache time is not expired, the URL is safe.
- If negative cache time is expired, send fullhash request.
MozReview-Commit-ID: GRX7CP8ig49
--HG--
extra : rebase_source : bcb5fa7aa2b7b116d862e3382447611424603c2d
This patch fixes that Classifier::ActiveTables doesn't return v4 tables.
Classifier::mActiveTablesCache is generated by scanning safebrowsing directory.
We use Classifier::ScanStoreDir to do the work, but it will ignore subdirectory.
Since v4 tables are stored in subdirectory 'google4', mActiveTablesCache doesn't
include v4 tables.
Fix this issue by checking subdirectory recursively in ScanStoreDir.
MozReview-Commit-ID: I6pa6e4bFND
--HG--
extra : rebase_source : 49d19101bec780e21db668b2daff3c46bbdb3fd2
A new function Classifier::AsyncApplyUpdates() is implemented for async update.
Besides, all public Classifier interfaces become "worker thread only" and
we remove DBServiceWorker::ApplyUpdatesBackground/Foreground.
In DBServiceWorker::FinishUpdate, instead of calling Classifier::ApplyUpdates,
we call Classifier::AsyncApplyUpdates and install a callback for notifying
the update observer when update is finished. The callback will occur on the
caller thread (i.e. worker thread.)
As for the shutdown issue, when the main thread is notified to shut down,
we at first *synchronously* dispatch an event to the worker thread to
shut down the update thread. After getting synchronized with all other
threads, we send last two events "CancelUpdate" and "CloseDb" to notify
dangling update (i.e. BeginUpdate is called but FinishUpdate isn't)
and do cleanup work.
MozReview-Commit-ID: DXZvA2eFKlc
--HG--
extra : rebase_source : cd2e27a6b679d2c96e769854d1582ed2dcda12bb
URLCLASSIFIER_PREFIX_MATCH is replaced by URLCLASSIFIER_FULL_MATCH(Bug 1311931)
so it should be removed.
MozReview-Commit-ID: 4F1B07bGwRP
--HG--
extra : rebase_source : 2d3e5cda14df4e874bf31145ccb5330acec369f7
This is straightforward mapping of PR_LOG levels to their LogLevel
counterparts:
PR_LOG_ERROR -> LogLevel::Error
PR_LOG_WARNING -> LogLevel::Warning
PR_LOG_WARN -> LogLevel::Warning
PR_LOG_INFO -> LogLevel::Info
PR_LOG_DEBUG -> LogLevel::Debug
PR_LOG_NOTICE -> LogLevel::Debug
PR_LOG_VERBOSE -> LogLevel::Verbose
Instances of PRLogModuleLevel were mapped to a fully qualified
mozilla::LogLevel, instances of PR_LOG levels in #defines were mapped to a
fully qualified mozilla::LogLevel::* level, and all other instances were
mapped to us a shorter format of LogLevel::*.
Bustage for usage of the non-fully qualified LogLevel were fixed by adding
|using mozilla::LogLevel;| where appropriate.
This is straightforward mapping of PR_LOG levels to their LogLevel
counterparts:
PR_LOG_ERROR -> LogLevel::Error
PR_LOG_WARNING -> LogLevel::Warning
PR_LOG_WARN -> LogLevel::Warning
PR_LOG_INFO -> LogLevel::Info
PR_LOG_DEBUG -> LogLevel::Debug
PR_LOG_NOTICE -> LogLevel::Debug
PR_LOG_VERBOSE -> LogLevel::Verbose
Instances of PRLogModuleLevel were mapped to a fully qualified
mozilla::LogLevel, instances of PR_LOG levels in #defines were mapped to a
fully qualified mozilla::LogLevel::* level, and all other instances were
mapped to us a shorter format of LogLevel::*.
Bustage for usage of the non-fully qualified LogLevel were fixed by adding
|using mozilla::LogLevel;| where appropriate.
This is straightforward mapping of PR_LOG levels to their LogLevel
counterparts:
PR_LOG_ERROR -> LogLevel::Error
PR_LOG_WARNING -> LogLevel::Warning
PR_LOG_WARN -> LogLevel::Warning
PR_LOG_INFO -> LogLevel::Info
PR_LOG_DEBUG -> LogLevel::Debug
PR_LOG_NOTICE -> LogLevel::Debug
PR_LOG_VERBOSE -> LogLevel::Verbose
Instances of PRLogModuleLevel were mapped to a fully qualified
mozilla::LogLevel, instances of PR_LOG levels in #defines were mapped to a
fully qualified mozilla::LogLevel::* level, and all other instances were
mapped to us a shorter format of LogLevel::*.
Bustage for usage of the non-fully qualified LogLevel were fixed by adding
|using mozilla::LogLevel;| where appropriate.
