The document covers advanced debugging techniques, focusing on tools like gdb, strace, and valgrind for analyzing and resolving test failures in software development. It discusses the challenges of debugging intermittent failures and emphasizes the importance of scripted debugging using Python in gdb. Additionally, it provides examples of controlling logging levels and capturing system calls to enhance troubleshooting during testing.

1. Next Generation Debugging

2. Next Generation Debugging
Julian Smith, co-founder and CTO, Undo.
jsmith@undo.io
http://undo.io/

3. Overview
Testing.
Debugging:
Debugging with gdb.
Strace.
Valgrind.
Recording execution with Undo.
Undo research:
Execution analysis.
Dataflow analysis.
(Linux-specific.)

4. Testing has changed:
Resulting in:
Testing.
Continuous integration.
Test-driven development.
Cloud testing.
1,000s of tests per hour.
Many intermittent test failures.
Very difficult to fix them all.

5. Testing.
Security breaches.
Production outages.
Unhappy users.

6. Fixing test failures is hard.
Testing.
Recreate complex setups:
Multi-application.
Networking.
Multi-machine.
Re-run flakey tests many times to reproduce failure
Recompile/link with changes when investigating.
Changes behaviour.
Slow.
Requires a developer machine.

7. Fixing test failures is slow.
Testing
Reproducing slow failures is… slow.
Reproducing intermittent failures is also slow.
Requires repeatedly running a test many times in order to catch the failure.
Critical bugs:
Can occur one in a thousand runs.
Each run can take hours.

8. Tools to fix test failures
Testing.
Debuggers.
Logging.
System logging.
Memory checkers.
Recording execution + reversible debugging.

9. GDB
Debugging.
Better than you may remember:
TUI mode shows source code within terminal window:
gdb --tui
Or toggle with Ctrl-X Ctrl-A.
GDB-7 uses python as extension language.
Scripted debugging, e.g. to reproduce intermittent failures.
Compilers can generate debug symbols even for optimised code:
gcc -O2 -g ...
Optimisation may give non-linear results when stepping through code.
Miscellaneous:
Show backtrace of all threads: thread apply all bt.
Show stack pointer of first four threads: thread apply 1-4 print $sp.
Convenience variables: print $ax, print $pc, print $_exitcode.
Show source for an address: info line *0x12345678.
Show address bounds of current line: info line *$pc.
Show address of source code line: info line foo.c:42.

10. Python scripting in undodb-gdb and gdb.
GDB supports python as an extension language.
Used by undodb-gdb.
Capturing an intermittent failure.
Create python script and run with:
(gdb) source foo.py
(undodb-gdb) source foo.py
Examples:
repeat_until_non_zero_exit.py
repeat_until_breakpoint.py
repeat_until_signal.py

11. Python scripting in undodb-gdb and gdb.
repeat_until_non_zero_exit.py
'''
Repeatedly run debuggee until it fails.
'''
import gdb
while 1:
print '-' * 40
gdb.execute( 'run')
e = gdb.parse_and_eval( '$_exitcode')
print( '$_exitcode is: %s' % e)
if e != 0:
break

12. Python scripting in undodb-gdb and gdb.
repeat_until_breakpoint.py
'''
Repeatedly run debuggee until it hits a breakpoint.
'''
import gdb
events = []
def event_handler( event):
events.append( event)
gdb.events.exited.connect( event_handler)
gdb.events.stop.connect( event_handler)
while 1:
print '-' * 40
events = []
gdb.execute( 'run')
breakpoint = None
for event in events:
if isinstance( event, gdb.BreakpointEvent):
breakpoint = event
break
if breakpoint:
print( 'have hit breakpoint: %s' % breakpoint)
break

13. Python scripting in undodb-gdb and gdb.
repeat_until_signal.py
'''
Repeatedly run debuggee until it receives SIGSEGV.
'''
import gdb
import signal
events = []
def event_handler( event):
events.append( event)
gdb.events.exited.connect( event_handler)
gdb.events.stop.connect( event_handler)
while 1:
print '-' * 40
events = []
gdb.execute( 'run')
breakpoint = None
for event in events:
if ( isinstance( event, gdb.SignalEvent)
and event.stop_signal == 'SIGSEGV'
):
breakpoint = event
break
if breakpoint:
print( 'have hit breakpoint: %s' % breakpoint.stop_signal)
break

14. Examples.
Python scripting in undodb-gdb and gdb.

15. GUIs for gdb are getting better:
Debugging.
CLion.
Eclipse.
Qt Creator.
KDbg.
Emacs.

16. Logging.
Can sometimes work well.
Need to control what to log.
Define areas of functionality and assign different debug levels.
E.g. parser, lexer, network.
More detailed: memory allocator, socket, serialiser.
We can define debug levels for different categories to match the bug we are investigating.
This can get complicated.
logcategory_t* io_category = ...;
logcategory_t* serialisation_category = ...;
...
logf( io_category, "have read %zi bytes from socket fd=%i", n, fd);
...
logf( serialisation_category, "serialised %p to %zi bytes", foo, actualsize);
...

17. Problems with logging categories.
Logging.
How many categories - how detailed should we go?
Depends on the bug we are investigating.
May need to recompile with new categories.
What category do we use for code that writes serialised data to a file - io_category or serialisation_category ?

18. Use programme structure for categories.
Logging.
We already have areas of functionality:
Source code directories.
Source files.
Functions.
We can use these as implicit categories:
No need to define our own categories.
We get different levels of categories for free.
We get nested categories for free.

19. Controlling verbosity programmatically:
Logging.
debug_add( "network/socket", NULL, 1);
// Extra verbose for all diagnosics in network/socket*.*.
debug_add( "network/", NULL, 1);
debug_add( "network/socket", NULL, 1);
// Extra verbose for all diagnostics in network/*.*.
// Even more verbose in network/socket*.*.
debug_add( "heap/alloc.c", "", 1);
debug_add( "network/socket.c", Send, 2);
debug_add( "parser/", "", -1);
// Verbose for heap operations.
// Very verbose for all diagnostics in network/socket.c:Send().
// Less verbose in parser/.

20. Control verbosity with environmental variables:
Example:
Logging.
QA-friendly.
No need to recompile/link/build.
Activate logging in different parts of the programme depending on the bug which is being investigated.
DEBUG_LEVELS="heap/alloc.c=1 parser/=-1 network/socket.c:Send=2" myprog ...

21. Strace.
Linux/unix-specific.
Get a detailed log of all syscalls.
> strace date
execve("/bin/date", ["date"], [/* 34 vars */]) = 0
brk(0) = 0xd50000
access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory)
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f7602059000
access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory)
open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3</etc/ld.so.cache>
fstat(3</etc/ld.so.cache>, {st_mode=S_IFREG|0644, st_size=144491, ...}) = 0
mmap(NULL, 144491, PROT_READ, MAP_PRIVATE, 3</etc/ld.so.cache>, 0) = 0x7f7602035000
close(3</etc/ld.so.cache>) = 0
access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libc.so.6", O_RDONLY|O_CLOEXEC) = 3</lib/x86_64-linux-gnu/libc-2.19.so>
read(3</lib/x86_64-linux-gnu/libc-2.19.so>, "177ELF21130000000030>01000P34200000"..., 832) = 832
fstat(3</lib/x86_64-linux-gnu/libc-2.19.so>, {st_mode=S_IFREG|0755, st_size=1738176, ...}) = 0
mmap(NULL, 3844640, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3</lib/x86_64-linux-gnu/libc-2.19.so>, 0) = 0x7f7601a90000
mprotect(0x7f7601c32000, 2093056, PROT_NONE) = 0
mmap(0x7f7601e31000, 24576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3</lib/x86_64-linux-gnu/libc-2.19.so>, 0x1a1000) =
0x7f7601e31000
mmap(0x7f7601e37000, 14880, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f7601e37000
close(3</lib/x86_64-linux-gnu/libc-2.19.so>) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f7602034000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f7602033000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f7602032000
arch_prctl(ARCH_SET_FS, 0x7f7602033700) = 0
mprotect(0x7f7601e31000, 16384, PROT_READ) = 0
mprotect(0x60e000, 4096, PROT_READ) = 0
mprotect(0x7f760205b000, 4096, PROT_READ) = 0
munmap(0x7f7602035000, 144491) = 0
brk(0) = 0xd50000
brk(0xd71000) = 0xd71000
open("/usr/lib/locale/locale-archive", O_RDONLY|O_CLOEXEC) = 3</usr/lib/locale/locale-archive>
fstat(3</usr/lib/locale/locale-archive>, {st_mode=S_IFREG|0644, st_size=1607760, ...}) = 0
mmap(NULL, 1607760, PROT_READ, MAP_PRIVATE, 3</usr/lib/locale/locale-archive>, 0) = 0x7f7601ea9000
close(3</usr/lib/locale/locale-archive>) = 0
open("/etc/localtime", O_RDONLY|O_CLOEXEC) = 3</etc/localtime>
fstat(3</etc/localtime>, {st_mode=S_IFREG|0644, st_size=3661, ...}) = 0
fstat(3</etc/localtime>, {st_mode=S_IFREG|0644, st_size=3661, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f7602058000
read(3</etc/localtime>, "TZif2000000000000000000700070000"..., 4096) = 3661
lseek(3</etc/localtime>, -2338, SEEK_CUR) = 1323
read(3</etc/localtime>, "TZif200000000000000000010000100000"..., 4096) = 2338
close(3</etc/localtime>) = 0
munmap(0x7f7602058000, 4096) = 0
fstat(1</dev/pts/50>, {st_mode=S_IFCHR|0620, st_rdev=makedev(136, 50), ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f7602058000
write(1</dev/pts/50>, "Mon 26 Sep 12:27:50 BST 2016n", 29Mon 26 Sep 12:27:50 BST 2016
) = 29
close(1</dev/pts/50>) = 0
munmap(0x7f7602058000, 4096) = 0
close(2</dev/pts/50>) = 0
exit_group(0) = ?
+++ exited with 0 +++
Subset of syscalls - file operations:
> strace -y -e trace=file date
execve("/bin/date", ["date"], [/* 34 vars */]) = 0
access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory)
open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3</etc/ld.so.cache>
access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libc.so.6", O_RDONLY|O_CLOEXEC) = 3</lib/x86_64-linux-gnu/libc-2.19.so>
open("/usr/lib/locale/locale-archive", O_RDONLY|O_CLOEXEC) = 3</usr/lib/locale/locale-archive>
open("/etc/localtime", O_RDONLY|O_CLOEXEC) = 3</etc/localtime>
Mon 26 Sep 12:29:01 BST 2016
+++ exited with 0 +++
Subset of syscalls - memory operations:
> strace -y -e trace=memory date
brk(0) = 0x25b8000
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f14cc871000
mmap(NULL, 144491, PROT_READ, MAP_PRIVATE, 3</etc/ld.so.cache>, 0) = 0x7f14cc84d000
mmap(NULL, 3844640, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3</lib/x86_64-linux-gnu/libc-2.19.so>, 0) = 0x7f14cc2a8000
mprotect(0x7f14cc44a000, 2093056, PROT_NONE) = 0
mmap(0x7f14cc649000, 24576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3</lib/x86_64-linux-gnu/libc-2.19.so>, 0x1a1000) =
0x7f14cc649000
mmap(0x7f14cc64f000, 14880, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f14cc64f000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f14cc84c000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f14cc84b000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f14cc84a000
mprotect(0x7f14cc649000, 16384, PROT_READ) = 0
mprotect(0x60e000, 4096, PROT_READ) = 0
mprotect(0x7f14cc873000, 4096, PROT_READ) = 0
munmap(0x7f14cc84d000, 144491) = 0
brk(0) = 0x25b8000
brk(0x25d9000) = 0x25d9000
mmap(NULL, 1607760, PROT_READ, MAP_PRIVATE, 3</usr/lib/locale/locale-archive>, 0) = 0x7f14cc6c1000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f14cc870000
munmap(0x7f14cc870000, 4096) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f14cc870000
Mon 26 Sep 12:29:40 BST 2016
munmap(0x7f14cc870000, 4096) = 0
+++ exited with 0 +++

22. Summary:
Strace.
Not perfect - only works on syscall level.
But still useful for low-level investigations.
No recompilation required.

23. Overview:
Valgrind.
Linux, OS X, Solaris, Android.
Very detailed checking of execution.
Free.
Similar to Purify etc.

24. Memory checking:
Thread checking.
Other:
Valgrind.
Illegal memory accesses:
Overrun/under run heap blocks.
Overrun stack.
Use-after-free.
Double free.
Memory leaks.
Inconsistent lock orderings.
Data races (e.g. missing mutex).
CPU cache behaviour.
Heap profiler.

25. Highly recommended!
Valgrind.

26. New debugging technology in recent years.
Recording execution.
Linux:
Undo Live Recorder.
RR.
Windows:
Intellitrace (partial recording only).
TimeMachineFor.Net (partial recording only).
Java:
Chronon.
Undo (soon).

27. Live Recorder.
A library, for linking into an application.
Allows the application to control the recording of its own execution.
Provides a simple C API to start/save/stop recording.
API is defined in undolr.h header file and implemented in libundolr library.

28. Live Recorder.
Live Recorder recordings:
Are standard Undo Recording files.
Contain everything need to replay execution:
Non-deterministic events (inputs to program).
Initial state (initial memory and registers).
Also contain information needed for symbolic debugging:
Complete executable and .so files.
Debuginfo files.
Allows debugging even when libraries and/or debug information is not available locally (e.g. load and replay on a
different Linux distribution).
Loaded into UndoDB:
undodb-gdb --undodb-load <filename>
(undodb-gdb) undodb-load <filename>
Full reversible debugging.

29. Live Recorder.
Library API (undolr.h):
int undolr_recording_start( undolr_error_t* o_error);
int undolr_recording_stop( void);
int undolr_recording_save( const char* filename);
int undolr_recording_stop_and_save( const char* filename);
int undolr_save_on_termination( const char* filename);
int undolr_save_on_termination_cancel( void);
int undolr_event_log_size_get( long* o_bytes);
int undolr_event_log_size_set( long bytes);
int undolr_include_symbol_files( int include);

30. Live Recorder.
Use Live Recorder:
In internal testing.
At customer site.
Advantages:
Investigate bugs easily using reversible debugging.
You or your customer control when/if recording is enabled.
Avoid problems with differing environments.
No need to reproduce complex multi-machine setups.
Have multiple developers work on the same test failure.

31. Live Recorder.
Save a recording only if something went wrong.
...
undolr_save_on_termination_cancel();
exit(0);
...
exit(1);
...
int main( int argv, char** argv)
{
undolr_recording_start( NULL);
undolr_save_on_termination( "failure.undo");
...
if (...)
{
return 1;
}
...
undolr_save_on_termination_cancel();
return 0;
}

32. Live Recorder: Demo.

33. Questions?
Live Recorder.

34. Execution Analysis
Traditional debugging only gives partial information:
Core file:
Gives programme state only at the end of execution.
Traditional debugger:
Cannot go back to an earlier state.
Valgrind:
Have to know in advance what type of error is going to happen, and choose appropriate engine:
Cachegrind
Memcheck
Callgrind
Helgrind
What if you run with Helgrind but encounter an intermittent memory error?
Or you run with Memcheck and get a rare race condition?
Runs slowly because Valgrind engines are doing very complicated analysis.
Logging:
Shows information only from the logging that is enabled.
Usually not what you need to fix the bug.
So re-run with different logging.
Hope that bug isn’t intermittent.

35. An Undo recording gives you 100% knowledge of what your
programme did:
Undo is faster than Valgrind:
Execution Analysis
Proof: You can find to the state of the entire programme (memory plus registers) at any point in its execution.
stepi
stepi
…
and/or:
reverse-stepi
reverse-stepi
…
Because it is not performing analysis when recording.

36. But this isn’t always enough to track down really difficult bugs.
Execution Analysis
More useful navigation of programme execution:
Normal and reverse debugging:
undodb-goto-time ...
next, finish, reverse-next, reverse-finish, …
Breakpoints
Watchpoints.
Also:
Move to next/previous event matching a criteria:
ugo event prev name=='read' or name=='mmap'
ugo event prev name=='read' and result < 0

37. Execution Analysis
How could we make better use of this 100% complete knowledge?
Replay exact execution, but gather / output extra information:
Annotate replay with extra diagnostics from debugger (without changing the execution). E.g. gdb’s breakpoint
commands.
Replay with Memcheck-style memory checking.
Replay with Helgrind-style memory checking.
Log all calls to a particular set of functions.
Navigate forwards/backwards to next/previous call of a particular set of functions.
Log locks of a particular mutex.
Navigate forwards/backwards to next/previous lock of a particular mutex.
Go back to last call to a function with particular parameter values.
…
Suggestions ?

38. With a recording, it must be possible to figure out the bug.
Execution Analysis
How to find and fix a bug:
Replay recording, use forward/reverse commands, watchpoints, breakpoints to investigate.
If not able to figure it out:
Replay with appropriate annotations or valgrind-style checking to find more information.
Repeat until bug is found.
Because recording contains 100% information about the programme’s execution.

39. Execution Analysis
Summary of Undo advantages:
Constant modest speed overhead:
Undo recording is faster than Valgrind doing memory/thread/cache analysis.
Incur large analysis overhead only when required.
Retrospective analysis:
Run programme today.
Save recording.
Analyse against new regulations in 6 months time.

40. In development: Dataflow Analysis.
This is internal research and development.
A particular type of execution analysis.
Not available yet.
Subject to change before release.

41. In development: Dataflow Analysis.
When debugging, we usually focus on function calls:
Look at functions in the stack trace.
Step forwards with stepi, step, nexti, next, finish.
Step backwards with reverse-stepi, reverse-step, reverse-nexti, reverse-next, reverse-finish.
Move forwards and backwards with continue and reverse-continue.
Use breakpoints.
Use watchpoints.
Then look at state of data wherever we end up:
Registers.
Local variables.
Global variables.

42. In development: Dataflow Analysis.
Can we instead look more generally at data?
What inputs does current CPU instruction use, and where did these inputs come from?
New undodb-gdb command: undodb-current-instruction-info
Show inputs/outputs of current instruction.
Example, instruction sub -0x8(%rbp),%eax:
(undodb-gdb) undodb-current-instruction-info
undodb-gdb: current instruction is at 0x4005df:
undodb-gdb: reads:
undodb-gdb: memory 0x7ffc8d6ed6d8..+4
undodb-gdb: register 0 (ax)
undodb-gdb: writes:
undodb-gdb: register 0 (ax)
(undodb-gdb)
To follow the data:
We can set a 4-byte watchpoint on address 0x7ffc8d6ed6d8.
We need to be able to set a "watchpoint" on register %ax.
Not supported by CPU hardware.

43. In development: Dataflow Analysis.
But Undo knows everything about execution.
So we can make it specially instrument code to do register watchpoints.
New undodb-gdb command: undodb-search-registers
Search backwards for reads/writes to registers.
Example:
(undodb-gdb) undodb-search-registers ax,bp ds
Goes back to most recently executed instruction that did any of:
Read register %ax.
Read register %bp.
Wrote to register %ds.
Notes:
This is only possible because we have a record/reply engine.
Without record/replay engine, combinatorics make things impossible.
Only possible if we modify replay to also track changes to registers.

44. In development: Dataflow Analysis.
Putting it together:
We can look at what register/memory is used by current instruction.
We can go back to most recent time register/memory was changed.
Reverse dataflow analysis:
If current instruction reads one memory location:
Set watchpoint.
Reverse continue.
If current instruction reads one register:
Use undodb-search-registers.

45. Dataflow is a tree.
In development: Dataflow Analysis.
But what if current instruction …
… reads more than one register?
… reads register and memory?
… reads more than one location in memory?

46. In development: Dataflow Analysis.
Semi-automatic dataflow analysis:
Explore dataflow tree one step at a time.
At each step:
Use memory-watchpoint+reverse-continue, or undob-search-registers.
Remember results of each step.
Breadth-first search.
(undodb-gdb) undodb-dataflow-backwards-explore-start
(undodb-gdb) undodb-dataflow-backwards-explore-next
(undodb-gdb) undodb-dataflow-backwards-explore-next
...
undodb-gdb: Have finished dataflow exploration
(undodb-gdb) undodb-dataflow-backwards-explore-show

47. In development: Dataflow Analysis.
Example output:
(undodb-gdb) undodb-dataflow-backwards-explore-show
undodb-gdb: Dataflow is (13):
undodb-gdb: 316038:0x400728 => 316039:0x400735: memory 0x7ffd23ec1214..+4
undodb-gdb: 316039:0x400732 => 316039:0x400735: register 0 (ax)
undodb-gdb: 316029:0x4006f6 => 316040:0x40071e: memory 0x7ffd23ec11f0..+4
undodb-gdb: 316029:0x4006f6 => 316040:0x40071e: memory 0x7ffd23ec11f0..+4
undodb-gdb: 316040:0x40071e => 316040:0x400722: register 0 (ax)
undodb-gdb: 316040:0x40071e => 316040:0x400722: register 0 (ax)
undodb-gdb: 316040:0x400722 => 316040:0x400725: memory 0x7ffd23ec11fc..+4
undodb-gdb: 316040:0x400725 => 316040:0x400728: register 0 (ax)
undodb-gdb: 316039:0x400735 => 316040:0x400728: memory 0x7ffd23ec1214..+4
undodb-gdb: 316040:0x400722 => 316041:0x400732: memory 0x7ffd23ec11fc..+4
undodb-gdb: 316040:0x400728 => 316041:0x400735: memory 0x7ffd23ec1214..+4
undodb-gdb: 316041:0x400732 => 316041:0x400735: register 0 (ax)
undodb-gdb: 316041:0x400735 => 316042:0x400747: memory 0x7ffd23ec1214..+4
(undodb-gdb)

48. In development: Dataflow Analysis.

49. Questions?
In development: Dataflow Analysis.
What could we do with reverse dataflow?
Easier tracing of the ultimate cause of incorrect data.
Test-case synthesis - find what inputs would be required to make programme take a particular path.
Issues:
Currently only detect reads/writes to general purpose registers.
Consider: add %eax,-0xc(%rbp)
Reads memory at %bp-0xc.
Does this also read register %bp ?
(Currently we say: No.)
Currently slow.

50. EOF.
http://undo.io/