gecko-dev/docs/contributing/debugging/debugging_firefox_with_gdb.rst

Debugging Firefox with GDB
==========================

This page details how you can more easily debug Firefox with gdb. :ref:`rr
<Debugging Firefox with rr>` is most often a better choice to debug a problem,
but sometimes it isn't possible to use it, such as attempting to reproduce a
race condition or when performance is important to reproduce an issue. ``rr``
chaos mode allows reproducing a lot of issues though, and should be tried.

Where can I find general gdb documentation?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Using GDB is beyond the scope of this document. Documentation is likely
available on your system if you have GDB installed, in the form of
**info,** **man** pages, or the gnome help browser. Additionally, you
can use a graphical front-end to GDB like
`ddd <https://www.gnu.org/software/ddd/>`__ or
`insight <https://sourceware.org/insight/>`__. For more information see
https://sourceware.org/gdb/current/onlinedocs/gdb/

How to debug Firefox with gdb?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Firefox is a multiprocess application, with a parent process, and several child
processes, each specialized and sandboxed differently.

.. code:: bash

   $ ./mach run --debugger=gdb

allows running Firefox in gdb, and debug the parent process. You can substitute
``run`` with another action, such as ``test``, ``mochitest``, ``xpcshell-test``,
etc.

Debugging child processes can be done by attaching the debugger.

.. code:: bash

   $ gdb --pid <pid>

There's a number of ways to find the PID of a process: hovering over a tab for a
content process, opening ``about:processes``, using ``ps ... | grep firefox`` on
the command line, etc.

Sometimes, it is desirable to attach to a child process at startup, to diagnose
something very close to the start of the process. Setting the environment
variable ``MOZ_DEBUG_CHILD_PROCESS=10`` will make each new process print an few
informative lines, including the process type and its PID. The process with then
sleep for a number of seconds equal to the value of the environment variable,
allowing to attach a debugger.

.. code:: bash

   $ MOZ_DEBUG_CHILD_PROCESS=10 ./mach run
   ...
   ...
   ...

   CHILDCHILDCHILDCHILD (process type tab)
   debug me @ 65230

   ...
   ...
   ...

Attaching gdb to Firefox might fail on Linux distributions that enable common
kernel hardening features such as the Yama security module. If you encounter the
following error when attaching:

.. code:: bash

   $ gdb --pid <pid>
   ...
   ...

   Attaching to process <pid>
   ptrace: Operation not permitted.

Check the contents of `/proc/sys/kernel/yama/ptrace_scope`. If  it set to `1`
you won't be able to attach to processes, set it to `0` from a root shell:

.. code:: bash

  \# echo 0 > /proc/sys/kernel/yama/ptrace_scope

If you still can't attach check your setup carefully. Do not, under any
circumstances, run gdb as root. Since gdb can execute arbitrary code and spawn
shells it can be extremely dangerous to use it with root permissions.


Advanced gdb configuration
~~~~~~~~~~~~~~~~~~~~~~~~~~

The preferred method, is using the
:ref:`mach` command-line tool to run the
debugger, which can bypass several optional defaults. Use "mach help
run" to get more details. If inside the source directory, you would use
"./mach". Please note that
:ref:`mach is aware of mozconfigs <Configuring Build Options>`.

.. code:: bash

   $ ./mach run --debug [arguments to pass to firefox]

If you need to direct arguments to gdb, you can use '--debugger-args'
options via the command line parser, taking care to adhere to shell
splitting rules. For example, if you wanted to run the command 'show
args' when gdb starts, you would use:

.. code:: bash

   $ ./mach run --debug --debugger-args "-ex 'show args'"

Alternatively, you can run gdb directly against Firefox. However, you
won't get some of the more useful capabilities this way. For example,
mach sets an environment variable (see below) to stop the JS engine from
generating synthetic segfaults to support the slower script dialoging
mechanism.

.. code::

   (gdb) $OBJDIR/dist/bin/firefox

How to debug a Firefox in the field (not compiled on the host)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

If you need to attach to a Firefox process live on a machine, and this Firefox
was built by Mozilla, or by certain Linux distros, it's possible to get symbols
and sources using the Mozilla symbol server, see :ref:`this section <Downloading
symbols on Linux / Mac OS X>` for setup instructions, it's just a matter of
sourcing a python script in ``.gdbinit``.

Debugging then works as usual, except the build probably has a very high
optimization level.

How do I pass arguments in prun?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Set the arguments in GDB before calling prun. Here's an example on how
to do that:

.. code::

   (gdb) set args https://www.mozilla.org
   (gdb) prun

Why breakpoints seem to not be hit?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The most likely cause is that `gdb` hasn't been attached to the process in which
the code to diagnose is ran. Enabling the relevant `MOZ_LOG` modules can help,
since by default it prints the process type and pid of all logging statements.

`break list` will display a list of breakpoints, and whether or not they're
enabled. C++ namespaces need to be specified entirely, and it's sometimes hard
to break in lambda. Breaking by line number is an alternative strategy that
often works in this case.

How do I display an nsString?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. code::

   (gdb) p ToNewCString(string);

This leaks a bit of memory but it doesn't really matter.

How do I determine the concrete type of an object pointed to by an interface pointer?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

You can determine the concrete type of any object pointed to, by an
XPCOM interface pointer, by looking at the mangled name of the symbol
for the object's vtable:

.. code::

   (gdb) p aKidFrame
   $1 = (nsIFrame *) 0x85058d4
   (gdb) x/wa *(void**)aKidFrame
   0x4210d380 <__vt_14nsRootBoxFrame>: 0x0
   (gdb) p *(nsRootBoxFrame*)aKidFrame
    [ all the member variables of aKidFrame ]

Or use the gdb command ``set print object on``.

How can I debug JavaScript from gdb?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

If you have JavaScript Engine code on the stack, you'll probably want a
JS stack in addition to the C++ stack.

.. code::

   (gdb) call DumpJSStack()

Please note that if `gdb` has been attached to a process, the stack might be
printed in the terminal window in which Firefox was started.

See
`this MDN page
<https://developer.mozilla.org/en-US/docs/Mozilla/Debugging/Debugging_JavaScript>`__
for more JS debugging tricks.

How can I debug race conditions
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Try :ref:`rr <Debugging Firefox with rr>` first. If this doesn't work, good
luck, maybe try :ref:`logging <Gecko Logging>` or sprinkling assertions.

I keep getting a SIGSYS, or SIGSEGV in JS/JIT code under gdb even though there is no crash when gdb is not attached.  How do I fix it?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Allow gdb to read mozilla-central's .gdbinit, located at `build/.gdbinit`. In
your own `.gdbinit`, add the line:

  .. code::

     add-auto-load-safe-path /path/to/mozilla-central

How do I get useful stack traces inside system libraries?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Many Linux distributions provide separate packages with debugging
information for system libraries, such as gdb, Valgrind, profiling
tools, etc., to give useful stack traces via system libraries.

The modern way to do this is to enable ``debuginfod``. This can be done by adding:

  .. code::

    set debuginfod enabled on

in your ``.gdbinit``, but there might be distro-specific instructions.
Alternatively, you can install the packages that contain the debug symbols for
the libraries you want to debug.

When using ``debuginfod``, the correct information will be downloaded
automatically when needed (and subsequently cached).

If you're not sure what to use, there's a federated debuginfod server that
provides debug information for most mainstream distributions. You can use it
by adding the following line to your ``.gdbinit`` file:

  .. code::

    set debuginfod urls "https://debuginfod.elfutils.org/"

Keep in mind that it might take a while to download debug information the
very first time. This queries all the servers of multiple distributions
sequentially and debug information tends to be large. It will be cached for the
next run though.

Fedora
^^^^^^

On Fedora, you need to enable the debuginfo repositories, as the
packages are in separate repositories. Enable them permanently, so when
you get updates you also get security updates for these packages. A way
to do this is edit ``/etc/yum.repos.d/fedora.repo`` and
``fedora-updates.repo`` to change the ``enabled=0`` line in the
debuginfo section to ``enabled=1``. This may then flag a conflict when
upgrading to a new distribution version. You would the need to perform
this edit again.

You can finally install debuginfo packages with yum or other package
management tools. The best way is install the ``yum-utils`` package, and
then use the ``debuginfo-install`` command to install all the debuginfo:

.. code:: bash

   $ yum install yum-utils
   $ debuginfo-install firefox

This can be done manually using:

.. code:: bash

    $ yum install GConf2-debuginfo ORBit2-debuginfo atk-debuginfo \
    cairo-debuginfo dbus-debuginfo expat-debuginfo \
    fontconfig-debuginfo freetype-debuginfo gcc-debuginfo glib2-debuginfo \
    glibc-debuginfo gnome-vfs2-debuginfo gtk2-debuginfo gtk2-engines-debuginfo \
    hal-debuginfo libX11-debuginfo libXcursor-debuginfo libXext-debuginfo \
    libXfixes-debuginfo libXft-debuginfo libXi-debuginfo libXinerama-debuginfo \
    libXrender-debuginfo libbonobo-debuginfo libgnome-debuginfo \
    libselinux-debuginfo pango-debuginfo popt-debuginfo scim-bridge-debuginfo

Disabling multiprocess
~~~~~~~~~~~~~~~~~~~~~~

``mach run`` and ``mach test`` both accept a ``--disable-e10s`` argument. Some
debuggers can't catch child-process crashes without it. This is sometimes a
viable alternative to attaching, but these days it changes enough thing that
it's not always a usable option.

See also
~~~~~~~~~


-  `Mike Conley's blog post <https://mikeconley.ca/blog/2014/04/25/electrolysis-debugging-child-processes-of-content-for-make-benefit-glorious-browser-of-firefox>`__
-  `Performance tools <https://wiki.mozilla.org/Performance:Tools>`__
-  `Fun with
   gdb <https://blog.mozilla.com/sfink/2011/02/22/fun-with-gdb/>`__ by
   Steve Fink