We need a test-only IPC message to socket process to trigger the Telemetry::Scalar set since no js engine in the socket process.
And hook the IPC call to AddPendingEvent |CallOrWaitSocketProcess| introduced by bug 1496257.
Differential Revision: https://phabricator.services.mozilla.com/D14822
--HG--
extra : moz-landing-system : lando
By refactoring this test code we can more easily adjust the number of fields
there are supposed to be in these objects.
--HG--
extra : source : 270b892f2c93aaed926c4ec70600f1c74b6325af
extra : histedit_source : 1e0d7ca3033dfb5aff8bd0e90b63d6ad23aea994
Nightly is reporting mozglue.dll as being untrusted in the untrusted modules
ping. Until now, xpcshell tests hard-code mozglue to appear as untrusted in
order to cover certain code paths related to startup modules.
This patch:
1. Checks explicitly for the xpcshell environment and only applies this logic
during xpcshell tests.
2. Uses a purpose-build DLL, "untrusted-startup-test-dll.dll", instead of
mozglue. This is more explicit and doesn't interfere with any "true"
processing of mozglue.
Differential Revision: https://phabricator.services.mozilla.com/D14720
--HG--
extra : moz-landing-system : lando
This has benefits both in terms of performance and memory usage. Aside from
the obvious savings of not loading additional JS scripts in every process,
this also allows us to move more of our expensive data collection work to a
background thread, where it doesn't risk janking both parent and content
processes.
MozReview-Commit-ID: 2A593R7bIKB
Differential Revision: https://phabricator.services.mozilla.com/D13872
--HG--
extra : rebase_source : ec634ee3a3b975809f542aa8077ad32236781452
The storage approach for dynamic scalars uses nsAtom to ensure we don't
repeatedly store the same string over and over.
Differential Revision: https://phabricator.services.mozilla.com/D12788
--HG--
extra : moz-landing-system : lando
BREAKING CHANGE:
This changes the API of the object returned from `Telemetry.getKeyedHistogramById`.
* `add` will continue to record into all registered stores for the histogram.
* `snapshot` now takes an object `{store: "storeName"}`, if not given it defaults to "main".
Support for getting a snapshot for just one key is dropped.
* `keys` now takes an object `{store: "storeName"}`, if not given it defaults to "main"
* `clear` now takes an object `{store: "storeName"}`, if not given it defaults to "main"
Depends on D12555
Differential Revision: https://phabricator.services.mozilla.com/D12556
--HG--
extra : moz-landing-system : lando
This changes the API of the object returned from `Telemetry.getHistogramById`
* `add` will continue to record into all registered stores for the histogram.
* `snapshot` now takes an object `{store: "storeName"}`, if not given it defaults to "main"
* `clear` now takes an object `{store: "storeName"}`, if not given it defaults to "main"
Differential Revision: https://phabricator.services.mozilla.com/D12555
--HG--
extra : moz-landing-system : lando
And test it.
Paritcular annoying bits are that we now have to iterate over Scalar objects
that are -present- even if they don't have data for the store being snapshot.
To maintain our previous semantics about not having process entries in the
snapshot object when the process is null (and, similarly for keyed scalars), we
need to retcon out some things during the snapshotting process.
Differential Revision: https://phabricator.services.mozilla.com/D12545
--HG--
extra : moz-landing-system : lando
For now we test that storing still works and the main store is still accessed for snapshots and clearing.
Depends on D11906
Differential Revision: https://phabricator.services.mozilla.com/D11907
--HG--
extra : moz-landing-system : lando
When registering dynamic builtin scalars and events (Build Faster support) we
didn't check whether they were expired. Let's do that.
Differential Revision: https://phabricator.services.mozilla.com/D11329
--HG--
extra : moz-landing-system : lando
When registering dynamic builtin scalars and events (Build Faster support) we
didn't check whether they were expired. Let's do that.
Differential Revision: https://phabricator.services.mozilla.com/D11329
--HG--
extra : moz-landing-system : lando
TELEMETRY_FAILED_SEND_PINGS_SIZE_KB and TELEMETRY_SUCCESSFUL_SEND_PINGS_SIZE_KB
are unmonitored and don't show evidence consistent with the original
hypothesis that size correlates with send failures. So let's remove them.
TELEMETRY_SEND_FAILURE_TYPE is a useful measure to track numbers and types of
send failures, in case they change. So let's make that permanent.
MozReview-Commit-ID: GWaKhrzIvph
Differential Revision: https://phabricator.services.mozilla.com/D10284
--HG--
extra : moz-landing-system : lando
This implementation is considerably more memory efficient than the existing JS
implementation. Migrating to it fully saves us about 28K per base content
process.
Differential Revision: https://phabricator.services.mozilla.com/D9887
--HG--
extra : source : a19c6b3e0402d16a77185f82d9fedab83a7ca52e
extra : intermediate-source : 28a19b7290ab288a1cb2cbf6d49f905cecc9682b
This implementation is considerably more memory efficient than the existing JS
implementation. Migrating to it fully saves us about 28K per base content
process.
Differential Revision: https://phabricator.services.mozilla.com/D9887
--HG--
extra : source : a19c6b3e0402d16a77185f82d9fedab83a7ca52e
This implementation is considerably more memory efficient than the existing JS
implementation. Migrating to it fully saves us about 28K per base content
process.
Differential Revision: https://phabricator.services.mozilla.com/D9887
--HG--
extra : rebase_source : f358f1f7097eaa8cd62ae916d048a58489e9f5ff
extra : histedit_source : c44c42de02750d71e23e70fdf22d3c3a40b91d3d
This is the first test for the untrusted modules ping. Tests for
- Basic payload structure validity.
- Expected results for a few specific DLLs
Differential Revision: https://phabricator.services.mozilla.com/D7177
--HG--
extra : moz-landing-system : lando
This allows tests to cut the time between accumulations being made in child
processes and them showing up in snapshots in the parent process.
For instance, setting it to 10ms instead of 2000ms takes test_ChildEvents.js
from 12s down to 10s on my local dev machine.
This patch also sets the default during telemetry xpcshell tests to 10ms.
Differential Revision: https://phabricator.services.mozilla.com/D9712
--HG--
extra : moz-landing-system : lando
We erroneously reset client IDs on Fennec to a canary client ID.
This is now detected and a new valid and random client ID is set.
This adds a new boolean attribute "wasCanary" to the `state.json` file
generated by ClientID.jsm.
Depends on D9544
Differential Revision: https://phabricator.services.mozilla.com/D9903
--HG--
extra : moz-landing-system : lando
This avoids loading the remaining parts of TelemetrySession in a content process.
This saves around 10 kb of memory.
Differential Revision: https://phabricator.services.mozilla.com/D8378
--HG--
extra : moz-landing-system : lando
ProfileAge now returns a promise that resolves to an instance that has already
loaded its times.json. This makes multiple attempts to update data in times.json
safer.
Differential Revision: https://phabricator.services.mozilla.com/D8463
--HG--
extra : moz-landing-system : lando
Previously, the test never waited for the notification to actually fire.
Differential Revision: https://phabricator.services.mozilla.com/D8249
--HG--
extra : moz-landing-system : lando
This patch enables Hybrid Content Telemetry's |initPromise| to reject
if it is disabled or if the host requesting the API doesn't have
enough privileges. This also updates the documentation to mention
the change in behaviour.
Differential Revision: https://phabricator.services.mozilla.com/D5849
--HG--
extra : moz-landing-system : lando
Removing footerURL code , few of the lines are left , i am not sure how to delete them and what to use instead of them there.
Differential Revision: https://phabricator.services.mozilla.com/D5795
--HG--
extra : moz-landing-system : lando
Use PrioEncoder to encode a few already-included histograms, so we can compare results on the Telemetry server side.
Differential Revision: https://phabricator.services.mozilla.com/D5088
--HG--
extra : moz-landing-system : lando
This patch moves the parsers' code to python_mozparsers, which is
the definitive name of the package on pypi. The data generation scripts
are adjusted accordingly. The README and setup files are added to
enable publishing on pypi.
Differential Revision: https://phabricator.services.mozilla.com/D5043
--HG--
rename : toolkit/components/telemetry/build_scripts/parsers/__init__.py => toolkit/components/telemetry/build_scripts/python_mozparsers/__init__.py
rename : toolkit/components/telemetry/build_scripts/parsers/parse_events.py => toolkit/components/telemetry/build_scripts/python_mozparsers/parse_events.py
rename : toolkit/components/telemetry/build_scripts/parsers/parse_histograms.py => toolkit/components/telemetry/build_scripts/python_mozparsers/parse_histograms.py
rename : toolkit/components/telemetry/build_scripts/parsers/parse_scalars.py => toolkit/components/telemetry/build_scripts/python_mozparsers/parse_scalars.py
rename : toolkit/components/telemetry/build_scripts/parsers/shared_telemetry_utils.py => toolkit/components/telemetry/build_scripts/python_mozparsers/shared_telemetry_utils.py
extra : moz-landing-system : lando
We settled on a new naming scheme for ping implementation: {name}Ping.jsm.
This changes the test filenames as well and fixup the path
used to reference the JSM files.
Depends on D4427
Differential Revision: https://phabricator.services.mozilla.com/D4428
--HG--
rename : toolkit/components/telemetry/tests/unit/test_TelemetryEventPing.js => toolkit/components/telemetry/tests/unit/test_EventPing.js
rename : toolkit/components/telemetry/tests/unit/test_TelemetryEventPing_disabled.js => toolkit/components/telemetry/tests/unit/test_EventPing_disabled.js
rename : toolkit/components/telemetry/tests/unit/test_TelemetryHealthPing.js => toolkit/components/telemetry/tests/unit/test_HealthPing.js
rename : toolkit/components/telemetry/tests/unit/test_TelemetryModules.js => toolkit/components/telemetry/tests/unit/test_ModulesPing.js
extra : moz-landing-system : lando
Automatic changes by ESLint, except for manual corrections for .xml files.
Differential Revision: https://phabricator.services.mozilla.com/D4439
--HG--
extra : moz-landing-system : lando
`cpp_guard` was only used for operating system checks.
We can check this value in our Python build scripts and exclude histograms that shouldn't be included for the target OS.
Differential Revision: https://phabricator.services.mozilla.com/D2927
--HG--
extra : moz-landing-system : lando
This introduces the machinery needed to generate crash annotations from a YAML
file. The relevant C++ functions are updated to take a typed enum. JavaScript
calls are unaffected but they will throw if the string argument does not
correspond to one of the known entries in the C++ enum. The existing whitelists
and blacklists of annotations are also generated from the YAML file and all
duplicate code related to them has been consolidated. Once written out to the
.extra file the annotations are converted in string form and are no different
than the existing ones.
All existing annotations have been included in the list (and some obsolete ones
have been removed) and all call sites have been updated including tests where
appropriate.
--HG--
extra : source : 4f6c43f2830701ec5552e08e3f1b06fe6d045860
Tested the functionality of 'CombinedStacks'. Added coverage for 'AddStack' and 'RemoveStack' methods.
MozReview-Commit-ID: CxBZHHZMN3z
--HG--
extra : rebase_source : 72ded4aebcbcd7b6a9e1b510326f64670b7b0f55
Created two new test coverages, `WrongscalarOperator' and 'WrongKeyedScalarOperator', that test scalar operations on wrong type of scalars.
MozReview-Commit-ID: 8iI8NaZlmfg
--HG--
extra : rebase_source : ec0fed2b18a729d92fe9cff32ae27c1304427a0f
Created two new test coverages, `WrongscalarOperator' and 'WrongKeyedScalarOperator', that test scalar operations on wrong type of scalars.
MozReview-Commit-ID: 6FIMqRi0dgk
--HG--
extra : rebase_source : dca08d8580b86eded03ca48e28f9b402ae3a1f7d
This follows the steps from the specification and also ensures sending
the optout ping is only tried once and discarded if that fails.
Depends on D1947
MozReview-Commit-ID: 99peURNq9jx
Differential Revision: https://phabricator.services.mozilla.com/D1948
--HG--
extra : moz-landing-system : lando
The HCT content library needs the chrome to broadcast the value of the
Telemetry state before it can report a reliable value. This patch adds
a way to wait for the broadcast to happen and singnal that it's ok
to read the value from canUpload.
Differential Revision: https://phabricator.services.mozilla.com/D2030
--HG--
extra : moz-landing-system : lando
We introduced these measures in bug 1033860 because of the variety of different
things besides pings we were sending using Telemetry. We don't do that any more
and haven't looked at these probes in at least two years.
Differential Revision: https://phabricator.services.mozilla.com/D2008
--HG--
extra : moz-landing-system : lando
We introduced these measures in bug 1033860 because of the variety of different
things besides pings we were sending using Telemetry. We don't do that any more
and haven't looked at these probes in at least two years.
Differential Revision: https://phabricator.services.mozilla.com/D2008
--HG--
extra : moz-landing-system : lando
This follows the steps from the specification and also ensures sending
the optout ping is only tried once and discarded if that fails.
Depends on D1947
MozReview-Commit-ID: 99peURNq9jx
Differential Revision: https://phabricator.services.mozilla.com/D1948
--HG--
extra : moz-landing-system : lando
This will make sure that when running |mach python-test --python 3| locally,
we only run the tests that also run in CI with python 3 (and therefore pass
presumably).
MozReview-Commit-ID: 3OBr9yLSlSq
--HG--
extra : rebase_source : 456340d0ecdddf1078f2b5b4ebb1eddf3813b26a
When it's false we also disable collecting events completely, in case the
reason we're disabling it is due to storage issues.
GeckoView doesn't presently support Events, so the 'event' ping is disabled by
default for that platform.
MozReview-Commit-ID: 9eKAtRiuER0
--HG--
extra : rebase_source : 71b3c9ff802420ff21941656c3d848c6d320578d
pingsOverdue is a telemetry-specific data field that is not used anymore.
Therefore it is being removed from both the docs and TelemetrySession.jsm
The logic that exports and computes the overdue pings count, and all related
code, is also removed.
Associated test failure (due to performing the above) is fixed by removing the
offending test code.
MozReview-Commit-ID: DZUapvZbC9U
--HG--
extra : rebase_source : b5207495f19d23d0a9f57ba62a8c6c6872958b49
This requires we take unsent event records out of about:telemetry since its
"Current Payload" view only looks at the "main" ping.
MozReview-Commit-ID: GLs2EYvFaAF
--HG--
extra : rebase_source : 63ffa636213bdcdc437e3768b4d449b7cb73ead4
I added checks that the new topic was appropriately notified and
that storage doesn't truncate upon hitting the limit, also covering the
case when there is a specified event limit for return.
MozReview-Commit-ID: FoSVvi7XSeM
--HG--
extra : rebase_source : 97ccd213985e0979f832c9f2956bf1e9d6bbfb74
yaml.load() can lead to arbitrary code execution because it isn't
secure by default and allows special YAML syntax.
While it shouldn't be a problem here, I'm trying to get rid of all
yaml.load() calls so we can add a lint to ban the practice.
Differential Revision: https://phabricator.services.mozilla.com/D1740
--HG--
extra : rebase_source : eed31255da88254cb248b51c5ab917bcae76f1db
extra : histedit_source : 4a681465ec8434e92dc9164a759eb521c10b9e79
Fairly straightforward, just a blanket removal. Haven't heard
anything on dev-platform or fx-data-dev regarding this removal,
so I think it's likely safe to remove on Nightly, and we can
revert if anyone makes a fuss.
As part of removing the HangMonitor, I renamed a few things and
reorganized the namespaces to not depend on a HangMonitor
namespace. Hopefully this doesn't produce too much noise in the
diff, it just seemed appropriate to move everything around
rather than keep dangling vestiges of the old system.
MozReview-Commit-ID: 8C8NFnOP5GU
--HG--
extra : rebase_source : dd000a05bfc2da40c586644d33ca4508fa5330f6
This follows the documentation example:
* Scalar deserialization is started
* “test” scalar is incremented by “10” by the application -> The operation [test, add, 10] is recorded into the list.
* The state of the “test” scalar is loaded off the persistence file, and the value “14” is set.
* Deserialization is finished and the pending operations are applied.
* The “test” scalar is incremented by “10”, the value is now “24”
It does that in both the parent and the content process.
MozReview-Commit-ID: CnzDcJ5o7jJ
--HG--
extra : rebase_source : cae5297f98b56b1a8fedad604d905e818c66e442
This patch changes the snapshotting code for keyed histograms
so that requesting a snapshot does not report expired histograms.
MozReview-Commit-ID: GDiw6yOcF8J
--HG--
extra : rebase_source : 30e10d7b1e63b6eef263066a49b8c4dea0c140e0
This follows the documentation example:
* Scalar deserialization is started
* “test” scalar is incremented by “10” by the application -> The operation [test, add, 10] is recorded into the list.
* The state of the “test” scalar is loaded off the persistence file, and the value “14” is set.
* Deserialization is finished and the pending operations are applied.
* The “test” scalar is incremented by “10”, the value is now “24”
It does that in both the parent and the content process.
MozReview-Commit-ID: FSIkuW2xYjD
--HG--
extra : rebase_source : cae0ecb98a0cea08dc56c38f8be1c6ff2341cc5e
This patch uses the GeckoView testing IDL interface and introduces a test
to verify that content histograms are correctly persisted and unpersisted.
MozReview-Commit-ID: 3Ja0hOTvEki
--HG--
extra : rebase_source : ad4288e988565391eec39fd82ab7f6ab6bb6ee06
This means that during a normal "mach build faster" build all the events
will be overwritten, as the generated EventArtifactsDefinitions.json
contains all events from the Events.yaml it was generated from.
Real dynamic events follow similar code paths internally.
We also ensure they trigger the right code path and don't overwrite.
MozReview-Commit-ID: JvFZfwCbjQN
--HG--
extra : rebase_source : cb6ed79afa678f9f0ef78a01c6cf5335b96c39d0
