Google is committed to advancing racial equity for Black communities. See how.

The Chromium Chronicle: Time-Travel Debugging with RR

Episode 13: March, 2020

by Christian Biesinger in Madison, WI

Do you find yourself running the same test over and over in the debugger, trying to figure out how the code got in a bad state? We have a tool for you! Easy to install and setup, it will record an execution trace, and that gives magical new powers to gdb. Step backwards, run backwards, see where variables changed their value or when a function was last called on an object (using conditional breakpoints).

On Linux, you can use rr. Install using sudo apt-get install rr or from https://rr-project.org/.

This is not officially supported, but very useful. The way rr works is that you first record a trace, then replay it.

rr record .../content_shell --no-sandbox  --disable-hang-monitor --single-process
# record the trace. --single-process is optional, see below. The other flags are required.
rr replay # This will replay the last trace
(gdb)       # rr uses GDB to let you replay traces

Conveniently, timing and pointer addresses stay the same every time you replay the same trace. Traces can be made portable using rr pack so that you can copy them to another machine and replay there, or replay even after recompiling. Run your program using continue. You can use all regular GDB commands -b, next, watch, etc. However, you can also use reverse-next (rn), reverse-cont (rc), reverse-step (rs), reverse-fin.

These still respect any breakpoints you’ve set. For example:

(gdb) c  # Execute to the end
(gdb) break blink::LayoutFlexibleBox::UpdateLayout
(gdb) rc # Run back to the last layout call
Thread 5 hit Breakpoint 1, blink::LayoutBlock::UpdateLayout (
    this=0x121672224010)
(gdb) # Inspect anything you want here. To find the previous Layout call on this object:
(gdb) cond 1 this == 0x121672224010
(gdb) rc
Thread 5 hit Breakpoint 1, blink::LayoutBlock::UpdateLayout (
    this=0x121672224010)
(gdb) watch -l style_.ptr_ # Or find the last time the style_ was changed
(gdb) rc
Thread 5 hit Hardware watchpoint 2: -location style_.ptr_

Old value = (const blink::ComputedStyle *) 0x1631ad3dbb0
New value = (const blink::ComputedStyle *) 0x0
0x00007f68cabcf78e in std::__Cr::swap<blink::ComputedStyle const*> (

In this example, I have used --single-process for simplicity, but that’s not necessary. RR can trace multiple processes; after recording, you can see a list using rr ps and pick one to replay with rr replay -f PID.

There are lots of ways RR can be useful. There are other commands you can use, such as when to find out at which event number you are at, or rr replay -M to annotate stdout with a process ID and event number for each line. See the RR website and documentation for more details.

Was this page helpful?
Yes
What was the best thing about this page?
It helped me complete my goal(s)
Thank you for the feedback. If you have specific ideas on how to improve this page, please create an issue.
It had the information I needed
Thank you for the feedback. If you have specific ideas on how to improve this page, please create an issue.
It had accurate information
Thank you for the feedback. If you have specific ideas on how to improve this page, please create an issue.
It was easy to read
Thank you for the feedback. If you have specific ideas on how to improve this page, please create an issue.
Something else
Thank you for the feedback. If you have specific ideas on how to improve this page, please create an issue.
No
What was the worst thing about this page?
It didn't help me complete my goal(s)
Thank you for the feedback. If you have specific ideas on how to improve this page, please create an issue.
It was missing information I needed
Thank you for the feedback. If you have specific ideas on how to improve this page, please create an issue.
It had inaccurate information
Thank you for the feedback. If you have specific ideas on how to improve this page, please create an issue.
It was hard to read
Thank you for the feedback. If you have specific ideas on how to improve this page, please create an issue.
Something else
Thank you for the feedback. If you have specific ideas on how to improve this page, please create an issue.

Subscribe to our RSS or Atom feed and get the latest updates in your favorite feed reader!