Bug 1877193 part 1 - Make JS prefs in StaticPrefList.yaml available in SpiderMonkey. r=mgaudet,KrisWright

Adding a new pref for a new SpiderMonkey feature or optimization requires a lot of boilerplate code.

This patch stack lets us eliminate almost all of this boilerplate by parsing `StaticPrefList.yaml`
at SpiderMonkey build time and generating code to automatically expose the `javascript.options.*`
prefs that have the `set_spidermonkey_pref` attribute.

Both JS shell and browser builds will default to the pref value in `StaticPrefList.yaml`.
In the JS shell, it'll be possible to set a pref value with `--setpref` or a custom shell flag.
Code for this will be added in later patches.

Differential Revision: https://phabricator.services.mozilla.com/D199992

config/check_spidermonkey_style.py

@@ -68,6 +68,7 @@ included_inclnames_to_ignore = set(
         "jit/CacheIROpsGenerated.h",  # generated in $OBJDIR
         "jit/LIROpsGenerated.h",  # generated in $OBJDIR
         "jit/MIROpsGenerated.h",  # generated in $OBJDIR
+        "js/PrefsGenerated.h",  # generated in $OBJDIR
         "js/ProfilingCategoryList.h",  # comes from mozglue/baseprofiler
         "mozilla/glue/Debug.h",  # comes from mozglue/misc, shadowed by <mozilla/Debug.h>
         "jscustomallocator.h",  # provided by embedders;  allowed to be missing

js/public/Prefs.h

@@ -0,0 +1,100 @@
+/* -*- Mode: C++; tab-width: 8; indent-tabs-mode: nil; c-basic-offset: 2 -*- */
+/* This Source Code Form is subject to the terms of the Mozilla Public
+ * License, v. 2.0. If a copy of the MPL was not distributed with this
+ * file, You can obtain one at http://mozilla.org/MPL/2.0/. */
+
+#ifndef js_Prefs_h
+#define js_Prefs_h
+
+#include "js/PrefsGenerated.h"
+
+// [SMDOC] Prefs
+//
+// JS::Prefs is used to make JS preferences defined in StaticPrefList.yaml
+// available to SpiderMonkey code.
+//
+// Adding a Pref
+// =============
+// Adding a new pref is easy. For example, if you're adding a new JS feature,
+// you could add the following to StaticPrefList.yaml:
+//
+//   - name: javascript.options.experimental.my_new_feature
+//     type: bool
+//     value: false
+//     mirror: always
+//     set_spidermonkey_pref: startup
+//
+// The value of this pref can then be accessed in SpiderMonkey code with
+// |JS::Prefs::experimental_my_new_feature()|.
+//
+// The default pref value in the YAML file applies to all SpiderMonkey builds
+// (browser, JS shell, jsapi-tests, etc), so by default this feature will be
+// disabled everywhere.
+//
+// To enable your feature, use the |--setpref experimental.my_new_feature=true|
+// JS shell command line argument, or set the browser pref in about:config.
+// Because this is a 'startup' pref, a browser restart is required for this to
+// take effect.
+//
+// The rest of this comment describes more advanced use cases.
+//
+// Non-startup prefs
+// =================
+// Setting |set_spidermonkey_pref = startup| is recommended for most prefs.
+// In this case the pref is only set during startup so we don't have to worry
+// about the pref value changing at runtime.
+//
+// However, for some prefs this doesn't work. For instance, the WPT test harness
+// can set test-specific prefs after startup. To properly update the JS pref in
+// this case, |set_spidermonkey_pref = always| must be used. This means the
+// SpiderMonkey pref will be updated whenever it's changed in the browser.
+//
+// Setting Prefs
+// =============
+// Embedders can override pref values. For startup prefs, this should only be
+// done during startup to avoid races with worker threads and to avoid confusing
+// code with unexpected pref changes:
+//
+//   JS::Prefs::setAtStartup_experimental_my_new_feature(true);
+//
+// Non-startup prefs can also be changed after startup:
+//
+//   JS::Prefs::set_experimental_my_new_feature(true);
+//
+// JS Shell Prefs
+// ==============
+// The JS shell |--list-prefs| command line flag will print a list of all of the
+// available JS prefs and their current values.
+//
+// To change a pref, use |--setpref name=value|, for example
+// |--setpref experimental.my_new_feature=true|.
+//
+// It's also possible to add a custom shell flag. In this case you have to
+// override the pref value yourself based on this flag.
+//
+// Testing Functions
+// =================
+// The |getAllPrefNames()| function will return an array with all JS pref names.
+//
+// The |getPrefValue(name)| function can be used to look up the value of the
+// given pref. For example, use |getPrefValue("experimental.my_new_feature")|
+// for the pref defined above.
+
+namespace JS {
+
+class Prefs {
+  // For each pref, define a static |pref_| member.
+  JS_PREF_CLASS_FIELDS;
+
+ public:
+  // For each pref, define static getter/setter accessors.
+#define DEF_GETSET(NAME, CPP_NAME, TYPE, SETTER) \
+  static TYPE CPP_NAME() { return CPP_NAME##_; } \
+  static void SETTER(TYPE value) { CPP_NAME##_ = value; }
+  FOR_EACH_JS_PREF(DEF_GETSET)
+#undef DEF_GETSET
+};
+
+};  // namespace JS
+
+#endif /* js_Prefs_h */

js/src/GeneratePrefs.py

@@ -0,0 +1,175 @@
+# This Source Code Form is subject to the terms of the Mozilla Public
+# License, v. 2.0. If a copy of the MPL was not distributed with this
+# file, You can obtain one at http://mozilla.org/MPL/2.0/.
+
+# This script generates js/public/PrefsGenerated.h from StaticPrefList.yaml
+
+import buildconfig
+import six
+import yaml
+from mozbuild.preprocessor import Preprocessor
+
+HEADER_TEMPLATE = """\
+/* This Source Code Form is subject to the terms of the Mozilla Public
+ * License, v. 2.0. If a copy of the MPL was not distributed with this
+ * file, You can obtain one at http://mozilla.org/MPL/2.0/. */
+
+#ifndef js_PrefsGenerated_h
+#define js_PrefsGenerated_h
+
+/* This file is generated by js/src/GeneratePrefs.py. Do not edit! */
+
+#include "mozilla/Atomics.h"
+
+#include <stdint.h>
+
+%(contents)s
+
+#endif // js_PrefsGenerated_h
+"""
+
+
+def load_yaml(yaml_path):
+    # First invoke the preprocessor to handle #ifdefs in the YAML file.
+    pp = Preprocessor()
+    pp.context.update(buildconfig.defines["ALLDEFINES"])
+
+    # To make #ifdef DEBUG work, use a similar hack as in emit_code in
+    # generate_static_pref_list.py.
+    if buildconfig.substs.get("MOZ_DEBUG"):
+        pp.context["DEBUG"] = "1"
+
+    pp.out = six.StringIO()
+    pp.do_filter("substitution")
+    pp.do_include(yaml_path)
+    contents = pp.out.getvalue()
+    return yaml.safe_load(contents)
+
+
+# Returns the C++ type to use for the pref type from the YAML file. Always use
+# the non-atomic type for return values and arguments. The field type is
+# determined elsewhere.
+def get_cpp_type(type):
+    if type in ("bool", "RelaxedAtomicBool"):
+        return "bool"
+    if type in ("uint32_t", "RelaxedAtomicUint32"):
+        return "uint32_t"
+    if type in ("int32_t", "RelaxedAtomicInt32"):
+        return "int32_t"
+    raise Exception("Unexpected type: {}".format(type))
+
+
+# Returns a C++ expression for the default pref value. Booleans in the YAML file
+# are converted to Pythonic True or False, so those need special handling.
+def get_cpp_init_value(val):
+    if val is True:
+        return "true"
+    if val is False:
+        return "false"
+    return str(val)
+
+
+def generate_prefs_header(c_out, yaml_path):
+    prefs = load_yaml(yaml_path)
+
+    js_options_prefix = "javascript.options."
+
+    def is_js_pref(pref):
+        set_spidermonkey_pref = pref.get("set_spidermonkey_pref", False)
+        if set_spidermonkey_pref not in (False, "startup", "always"):
+            raise Exception("Invalid value for set_spidermonkey_pref")
+
+        # Ignore prefs that don't have the |set_spidermonkey_pref| attribute.
+        if set_spidermonkey_pref is False:
+            return False
+
+        # Only support prefs with javascript.options prefix.
+        if not pref["name"].startswith(js_options_prefix):
+            raise Exception("set_spidermonkey_pref only works for JS prefs")
+
+        return True
+
+    # Remove all non-JS prefs and sort prefs by name.
+    prefs = list(filter(is_js_pref, prefs))
+    prefs.sort(key=lambda pref: pref["name"])
+
+    class_fields = []
+    class_fields_inits = []
+
+    macro_entries = []
+    browser_set_statements = []
+    browser_set_non_startup_statements = []
+
+    for pref in prefs:
+        name = pref["name"]
+        name = name[len(js_options_prefix) :]
+
+        is_startup_pref = pref["set_spidermonkey_pref"] == "startup"
+
+        cpp_name = name.replace(".", "_").replace("-", "_")
+        type = get_cpp_type(pref["type"])
+        init_value = get_cpp_init_value(pref["value"])
+
+        setter_name = ("setAtStartup_" if is_startup_pref else "set_") + cpp_name
+
+        # Use a relaxed atomic for non-startup prefs because those might be changed
+        # after startup.
+        field_type = type
+        if not is_startup_pref:
+            field_type = "mozilla::Atomic<{}, mozilla::Relaxed>".format(field_type)
+        class_fields.append("static {} {}_;".format(field_type, cpp_name))
+        class_fields_inits.append(
+            "{} JS::Prefs::{}_{{{}}};".format(field_type, cpp_name, init_value)
+        )
+
+        # Generate a MACRO invocation like this:
+        #   MACRO("arraybuffer_transfer", arraybuffer_transfer, bool, setAtStartup_arraybuffer_transfer)
+        macro_entries.append(
+            'MACRO("{}", {}, {}, {})'.format(name, cpp_name, type, setter_name)
+        )
+
+        # Generate a C++ statement to set the JS pref based on Gecko's StaticPrefs:
+        #   JS::Prefs::setAtStartup_foo(StaticPrefs::javascript_options_foo());
+        browser_pref_cpp_name = pref["name"].replace(".", "_").replace("-", "_")
+        if pref.get("do_not_use_directly", False):
+            browser_pref_cpp_name += "_DoNotUseDirectly"
+
+        statement = "JS::Prefs::{}(StaticPrefs::{}());".format(
+            setter_name, browser_pref_cpp_name
+        )
+        browser_set_statements.append(statement)
+
+        # For non-startup prefs, also generate code to update the pref after startup.
+        if not is_startup_pref:
+            browser_set_non_startup_statements.append(statement)
+
+    contents = ""
+
+    contents += "#define JS_PREF_CLASS_FIELDS \\\n"
+    contents += "".join(map(lambda s: "  {}\\\n".format(s), class_fields))
+    contents += "\n\n"
+
+    contents += "#define JS_PREF_CLASS_FIELDS_INIT \\\n"
+    contents += "".join(map(lambda s: "  {}\\\n".format(s), class_fields_inits))
+    contents += "\n\n"
+
+    contents += "#define FOR_EACH_JS_PREF(MACRO) \\\n"
+    contents += "".join(map(lambda s: "  {}\\\n".format(s), macro_entries))
+    contents += "\n\n"
+
+    contents += "#define SET_JS_PREFS_FROM_BROWSER_PREFS \\\n"
+    contents += "".join(map(lambda s: "  {}\\\n".format(s), browser_set_statements))
+    contents += "\n\n"
+
+    contents += "#define SET_NON_STARTUP_JS_PREFS_FROM_BROWSER_PREFS \\\n"
+    contents += "".join(
+        map(lambda s: "  {}\\\n".format(s), browser_set_non_startup_statements)
+    )
+    contents += "\n\n"
+
+    c_out.write(
+        HEADER_TEMPLATE
+        % {
+            "contents": contents,
+        }
+    )

js/src/jsapi.cpp

@@ -58,6 +58,7 @@
 #include "js/LocaleSensitive.h"
 #include "js/MemoryCallbacks.h"
 #include "js/MemoryFunctions.h"
+#include "js/Prefs.h"
 #include "js/PropertySpec.h"
 #include "js/Proxy.h"
 #include "js/ScriptPrivate.h"

js/src/make-source-package.py

@@ -207,6 +207,9 @@ rsync_filter_list = """
 
 + /xpcom/geckoprocesstypes_generator/**
 
+# List of prefs.
++ /modules/libpref/init/StaticPrefList.yaml
+
 # SpiderMonkey itself
 
 + /js/src/**

js/src/moz.build

@@ -119,6 +119,7 @@ EXPORTS += [
 ]
 
 EXPORTS.js += [
+    "!../public/PrefsGenerated.h",
     "../public/AllocationLogging.h",
     "../public/AllocationRecording.h",
     "../public/AllocPolicy.h",
@@ -170,6 +171,7 @@ EXPORTS.js += [
     "../public/MemoryMetrics.h",
     "../public/Modules.h",
     "../public/Object.h",
+    "../public/Prefs.h",
     "../public/Principals.h",
     "../public/Printer.h",
     "../public/Printf.h",
@@ -396,6 +398,7 @@ UNIFIED_SOURCES += [
     "vm/OffThreadPromiseRuntimeState.cpp",
     "vm/PIC.cpp",
     "vm/PlainObject.cpp",
+    "vm/Prefs.cpp",
     "vm/Printer.cpp",
     "vm/Probes.cpp",
     "vm/PromiseLookup.cpp",
@@ -626,6 +629,13 @@ GeneratedFile(
     inputs=selfhosted_inputs,
 )
 
+GeneratedFile(
+    "../public/PrefsGenerated.h",
+    script="GeneratePrefs.py",
+    entry_point="generate_prefs_header",
+    inputs=["../../modules/libpref/init/StaticPrefList.yaml"],
+)
+
 if CONFIG["JS_HAS_CTYPES"]:
     if CONFIG["MOZ_SYSTEM_FFI"]:
         CXXFLAGS += CONFIG["MOZ_FFI_CFLAGS"]

js/src/vm/Prefs.cpp

@@ -0,0 +1,10 @@
+/* -*- Mode: C++; tab-width: 8; indent-tabs-mode: nil; c-basic-offset: 2 -*-
+ * vim: set ts=8 sts=2 et sw=2 tw=80:
+ * This Source Code Form is subject to the terms of the Mozilla Public
+ * License, v. 2.0. If a copy of the MPL was not distributed with this
+ * file, You can obtain one at http://mozilla.org/MPL/2.0/. */
+
+#include "js/Prefs.h"
+
+// Set all static JS::Prefs fields to the default pref values.
+JS_PREF_CLASS_FIELDS_INIT;

js/xpconnect/src/XPCJSContext.cpp

@@ -45,6 +45,7 @@
 #include "js/HelperThreadAPI.h"
 #include "js/Initialization.h"
 #include "js/MemoryMetrics.h"
+#include "js/Prefs.h"
 #include "js/WasmFeatures.h"
 #include "mozilla/dom/BindingUtils.h"
 #include "mozilla/dom/ContentChild.h"
@@ -877,6 +878,9 @@ static void LoadStartupJSPrefs(XPCJSContext* xpccx) {
   //
   // 'Live' prefs are handled by ReloadPrefsCallback below.
 
+  // Set all JS::Prefs.
+  SET_JS_PREFS_FROM_BROWSER_PREFS;
+
   JSContext* cx = xpccx->Context();
 
   // Some prefs are unlisted in all.js / StaticPrefs (and thus are invisible in
@@ -1008,6 +1012,9 @@ static void LoadStartupJSPrefs(XPCJSContext* xpccx) {
 static void ReloadPrefsCallback(const char* pref, void* aXpccx) {
   // Note: Prefs that require a restart are handled in LoadStartupJSPrefs above.
 
+  // Update all non-startup JS::Prefs.
+  SET_NON_STARTUP_JS_PREFS_FROM_BROWSER_PREFS;
+
   auto xpccx = static_cast<XPCJSContext*>(aXpccx);
   JSContext* cx = xpccx->Context();
 

modules/libpref/init/StaticPrefList.yaml

@@ -29,13 +29,14 @@
 # -----------
 # A pref definition looks like this:
 #
-#   - name: <pref-name>                    # mandatory
-#     type: <cpp-type>                     # mandatory
-#     value: <default-value>               # mandatory
-#     mirror: <never | once | always>      # mandatory
-#     do_not_use_directly: <true | false>  # optional
-#     include: <header-file>               # optional
-#     rust: <true | false>                 # optional
+#   - name: <pref-name>                                 # mandatory
+#     type: <cpp-type>                                  # mandatory
+#     value: <default-value>                            # mandatory
+#     mirror: <never | once | always>                   # mandatory
+#     do_not_use_directly: <true | false>               # optional
+#     include: <header-file>                            # optional
+#     rust: <true | false>                              # optional
+#     set_spidermonkey_pref: <false | startup | always> # optional
 #
 # - `name` is the name of the pref, without double-quotes, as it appears
 #   in about:config. It is used in most libpref API functions (from both C++
@@ -98,6 +99,13 @@
 #   will be usable via the `static_prefs::pref!` macro, e.g.
 #   `static_prefs::pref!("layout.css.cross-fade.enabled")`.
 #
+# - `set_spidermonkey_pref` indicates whether SpiderMonkey boilerplate code
+#   should be generated for this pref. If this is set to 'startup', the
+#   pref on the SpiderMonkey side is only set during process startup. If set to
+#   'always', the SpiderMonkey pref value is also updated when this pref is
+#   changed at runtime.
+#   This option is only valid for javascript.options.* prefs.
+#
 # The getter function's base name is the same as the pref's name, but with
 # '.' or '-' chars converted to '_', to make a valid identifier. For example,
 # the getter for `foo.bar_baz` is `foo_bar_baz()`. This is ugly but clear,

modules/libpref/init/generate_static_pref_list.py

@@ -20,6 +20,7 @@ VALID_KEYS = {
     "do_not_use_directly",
     "include",
     "rust",
+    "set_spidermonkey_pref",
 }
 
 # Each key is a C++ type; its value is the equivalent non-atomic C++ type.