Episode 13: March, 2020
by Christian Biesinger in Madison, WI
Do you find yourself running the same test over and over in the debugger,
trying to figure out how the code got in a bad state? We have a tool for you!
Easy to install and setup, it will record an execution trace, and that gives
magical new powers to
gdb. Step backwards, run backwards, see where
variables changed their value or when a function was last called on an object
(using conditional breakpoints).
On Linux, you can use rr. Install using
sudo apt-get install rr or
This is not officially supported, but very useful. The way
rr works is that
you first record a trace, then replay it.
rr record .../content_shell --no-sandbox --disable-hang-monitor --single-process
# record the trace. --single-process is optional, see below. The other flags are required.
rr replay # This will replay the last trace
(gdb) # rr uses GDB to let you replay traces
Conveniently, timing and pointer addresses stay the same every time you replay
the same trace. Traces can be made portable using
rr pack so that you
can copy them to another machine and replay there, or replay even after
recompiling. Run your program using
continue. You can use all regular
watch, etc. However, you can also use
rn), reverse-cont (
rc), reverse-step (
These still respect any breakpoints you’ve set. For example:
(gdb) c # Execute to the end (gdb) break blink::LayoutFlexibleBox::UpdateLayout (gdb) rc # Run back to the last layout call Thread 5 hit Breakpoint 1, blink::LayoutBlock::UpdateLayout ( this=0x121672224010) (gdb) # Inspect anything you want here. To find the previous Layout call on this object: (gdb) cond 1 this == 0x121672224010 (gdb) rc Thread 5 hit Breakpoint 1, blink::LayoutBlock::UpdateLayout ( this=0x121672224010) (gdb) watch -l style_.ptr_ # Or find the last time the style_ was changed (gdb) rc Thread 5 hit Hardware watchpoint 2: -location style_.ptr_ Old value = (const blink::ComputedStyle *) 0x1631ad3dbb0 New value = (const blink::ComputedStyle *) 0x0 0x00007f68cabcf78e in std::__Cr::swap<blink::ComputedStyle const*> (
In this example, I have used
--single-process for simplicity, but that’s
not necessary. RR can trace multiple processes; after recording, you can
see a list using
rr ps and pick one to replay with
rr replay -f PID.
There are lots of ways RR can be useful. There are other commands you can use,
such as when to find out at which event number you are at, or
rr replay -M
stdout with a process ID and event number for each line. See
the RR website and documentation for more details.