SlideShare a Scribd company logo
1 of 50
Download to read offline
Next	Generation	Debugging
Next	Generation	Debugging
Julian	Smith,	co-founder	and	CTO,	Undo.
jsmith@undo.io
http://undo.io/
Overview
Testing.
Debugging:
Debugging	with	gdb.
Strace.
Valgrind.
Recording	execution	with	Undo.
Undo	research:
Execution	analysis.
Dataflow	analysis.
(Linux-specific.)
Testing	has	changed:
Resulting	in:
Testing.
Continuous	integration.
Test-driven	development.
Cloud	testing.
1,000s	of	tests	per	hour.
Many	intermittent	test	failures.
Very	difficult	to	fix	them	all.
Testing.
Security	breaches.
Production	outages.
Unhappy	users.
Fixing	test	failures	is	hard.
Testing.
Recreate	complex	setups:
Multi-application.
Networking.
Multi-machine.
Re-run	flakey	tests	many	times	to	reproduce	failure
Recompile/link	with	changes	when	investigating.
Changes	behaviour.
Slow.
Requires	a	developer	machine.
Fixing	test	failures	is	slow.
Testing
Reproducing	slow	failures	is…	slow.
Reproducing	intermittent	failures	is	also	slow.
Requires	repeatedly	running	a	test	many	times	in	order	to	catch	the	failure.
Critical	bugs:
Can	occur	one	in	a	thousand	runs.
Each	run	can	take	hours.
Tools	to	fix	test	failures
Testing.
Debuggers.
Logging.
System	logging.
Memory	checkers.
Recording	execution	+	reversible	debugging.
GDB
Debugging.
Better	than	you	may	remember:
TUI	mode	shows	source	code	within	terminal	window:
gdb	--tui
Or	toggle	with	Ctrl-X	Ctrl-A.
GDB-7	uses	python	as	extension	language.
Scripted	debugging,	e.g.	to	reproduce	intermittent	failures.
Compilers	can	generate	debug	symbols	even	for	optimised	code:
gcc	-O2	-g	...
Optimisation	may	give	non-linear	results	when	stepping	through	code.
Miscellaneous:
Show	backtrace	of	all	threads:	thread	apply	all	bt.
Show	stack	pointer	of	first	four	threads:	thread	apply	1-4	print	$sp.
Convenience	variables:	print	$ax,	print	$pc,	print	$_exitcode.
Show	source	for	an	address:	info	line	*0x12345678.
Show	address	bounds	of	current	line:	info	line	*$pc.
Show	address	of	source	code	line:	info	line	foo.c:42.
Python	scripting	in	undodb-gdb	and	gdb.
GDB	supports	python	as	an	extension	language.
Used	by	undodb-gdb.
Capturing	an	intermittent	failure.
Create	python	script	and	run	with:
(gdb)	source	foo.py
(undodb-gdb)	source	foo.py
Examples:
repeat_until_non_zero_exit.py
repeat_until_breakpoint.py
repeat_until_signal.py
Python	scripting	in	undodb-gdb	and	gdb.
repeat_until_non_zero_exit.py
'''
Repeatedly	run	debuggee	until	it	fails.
'''
import	gdb
while	1:
				print	'-'	*	40
				gdb.execute(	'run')
				e	=	gdb.parse_and_eval(	'$_exitcode')
				print(	'$_exitcode	is:	%s'	%	e)
				if	e	!=	0:
								break
Python	scripting	in	undodb-gdb	and	gdb.
repeat_until_breakpoint.py
'''
Repeatedly	run	debuggee	until	it	hits	a	breakpoint.
'''
import	gdb
events	=	[]
def	event_handler(	event):
				events.append(	event)
gdb.events.exited.connect(	event_handler)
gdb.events.stop.connect(	event_handler)
while	1:
				print	'-'	*	40
				events	=	[]
				gdb.execute(	'run')
				breakpoint	=	None
				for	event	in	events:
								if	isinstance(	event,	gdb.BreakpointEvent):
												breakpoint	=	event
												break
				if	breakpoint:
								print(	'have	hit	breakpoint:	%s'	%	breakpoint)
								break
Python	scripting	in	undodb-gdb	and	gdb.
repeat_until_signal.py
'''
Repeatedly	run	debuggee	until	it	receives	SIGSEGV.
'''
import	gdb
import	signal
events	=	[]
def	event_handler(	event):
				events.append(	event)
gdb.events.exited.connect(	event_handler)
gdb.events.stop.connect(	event_handler)
while	1:
				print	'-'	*	40
				events	=	[]
				gdb.execute(	'run')
				breakpoint	=	None
				for	event	in	events:
								if	(	isinstance(	event,	gdb.SignalEvent)
																and	event.stop_signal	==	'SIGSEGV'
												):
												breakpoint	=	event
												break
				if	breakpoint:
								print(	'have	hit	breakpoint:	%s'	%	breakpoint.stop_signal)
								break
Examples.
Python	scripting	in	undodb-gdb	and	gdb.
GUIs	for	gdb	are	getting	better:
Debugging.
CLion.
Eclipse.
Qt	Creator.
KDbg.
Emacs.
Logging.
Can	sometimes	work	well.
Need	to	control	what	to	log.
Define	areas	of	functionality	and	assign	different	debug	levels.
E.g.	parser,	lexer,	network.
More	detailed:	memory	allocator,	socket,	serialiser.
We	can	define	debug	levels	for	different	categories	to	match	the	bug	we	are	investigating.
This	can	get	complicated.
				logcategory_t*		io_category	=	...;
				logcategory_t*		serialisation_category	=	...;
				...
				logf(	io_category,	"have	read	%zi	bytes	from	socket	fd=%i",	n,	fd);
				...
				logf(	serialisation_category,	"serialised	%p	to	%zi	bytes",	foo,	actualsize);
				...
Problems	with	logging	categories.
Logging.
How	many	categories	-	how	detailed	should	we	go?
Depends	on	the	bug	we	are	investigating.
May	need	to	recompile	with	new	categories.
What	category	do	we	use	for	code	that	writes	serialised	data	to	a	file	-	io_category	or	serialisation_category	?
Use	programme	structure	for	categories.
Logging.
We	already	have	areas	of	functionality:
Source	code	directories.
Source	files.
Functions.
We	can	use	these	as	implicit	categories:
No	need	to	define	our	own	categories.
We	get	different	levels	of	categories	for	free.
We	get	nested	categories	for	free.
Controlling	verbosity	programmatically:
Logging.
debug_add(	"network/socket",	NULL,	1);
//	Extra	verbose	for	all	diagnosics	in	network/socket*.*.
debug_add(	"network/",	NULL,	1);
debug_add(	"network/socket",	NULL,	1);
//	Extra	verbose	for	all	diagnostics	in	network/*.*.
//	Even	more	verbose	in	network/socket*.*.
debug_add(	"heap/alloc.c",	"",	1);
debug_add(	"network/socket.c",	Send,	2);
debug_add(	"parser/",	"",	-1);
//	Verbose	for	heap	operations.
//	Very	verbose	for	all	diagnostics	in	network/socket.c:Send().
//	Less	verbose	in	parser/.
Control	verbosity	with	environmental	variables:
Example:
Logging.
QA-friendly.
No	need	to	recompile/link/build.
Activate	logging	in	different	parts	of	the	programme	depending	on	the	bug	which	is	being	investigated.
DEBUG_LEVELS="heap/alloc.c=1	parser/=-1	network/socket.c:Send=2"	myprog	...
Strace.
Linux/unix-specific.
Get	a	detailed	log	of	all	syscalls.
>	strace	date
execve("/bin/date",	["date"],	[/*	34	vars	*/])	=	0
brk(0)																																		=	0xd50000
access("/etc/ld.so.nohwcap",	F_OK)						=	-1	ENOENT	(No	such	file	or	directory)
mmap(NULL,	8192,	PROT_READ|PROT_WRITE,	MAP_PRIVATE|MAP_ANONYMOUS,	-1,	0)	=	0x7f7602059000
access("/etc/ld.so.preload",	R_OK)						=	-1	ENOENT	(No	such	file	or	directory)
open("/etc/ld.so.cache",	O_RDONLY|O_CLOEXEC)	=	3</etc/ld.so.cache>
fstat(3</etc/ld.so.cache>,	{st_mode=S_IFREG|0644,	st_size=144491,	...})	=	0
mmap(NULL,	144491,	PROT_READ,	MAP_PRIVATE,	3</etc/ld.so.cache>,	0)	=	0x7f7602035000
close(3</etc/ld.so.cache>)														=	0
access("/etc/ld.so.nohwcap",	F_OK)						=	-1	ENOENT	(No	such	file	or	directory)
open("/lib/x86_64-linux-gnu/libc.so.6",	O_RDONLY|O_CLOEXEC)	=	3</lib/x86_64-linux-gnu/libc-2.19.so>
read(3</lib/x86_64-linux-gnu/libc-2.19.so>,	"177ELF21130000000030>01000P34200000"...,	832)	=	832
fstat(3</lib/x86_64-linux-gnu/libc-2.19.so>,	{st_mode=S_IFREG|0755,	st_size=1738176,	...})	=	0
mmap(NULL,	3844640,	PROT_READ|PROT_EXEC,	MAP_PRIVATE|MAP_DENYWRITE,	3</lib/x86_64-linux-gnu/libc-2.19.so>,	0)	=	0x7f7601a90000
mprotect(0x7f7601c32000,	2093056,	PROT_NONE)	=	0
mmap(0x7f7601e31000,	24576,	PROT_READ|PROT_WRITE,	MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE,	3</lib/x86_64-linux-gnu/libc-2.19.so>,	0x1a1000)	=	
0x7f7601e31000
mmap(0x7f7601e37000,	14880,	PROT_READ|PROT_WRITE,	MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS,	-1,	0)	=	0x7f7601e37000
close(3</lib/x86_64-linux-gnu/libc-2.19.so>)	=	0
mmap(NULL,	4096,	PROT_READ|PROT_WRITE,	MAP_PRIVATE|MAP_ANONYMOUS,	-1,	0)	=	0x7f7602034000
mmap(NULL,	4096,	PROT_READ|PROT_WRITE,	MAP_PRIVATE|MAP_ANONYMOUS,	-1,	0)	=	0x7f7602033000
mmap(NULL,	4096,	PROT_READ|PROT_WRITE,	MAP_PRIVATE|MAP_ANONYMOUS,	-1,	0)	=	0x7f7602032000
arch_prctl(ARCH_SET_FS,	0x7f7602033700)	=	0
mprotect(0x7f7601e31000,	16384,	PROT_READ)	=	0
mprotect(0x60e000,	4096,	PROT_READ)					=	0
mprotect(0x7f760205b000,	4096,	PROT_READ)	=	0
munmap(0x7f7602035000,	144491)										=	0
brk(0)																																		=	0xd50000
brk(0xd71000)																											=	0xd71000
open("/usr/lib/locale/locale-archive",	O_RDONLY|O_CLOEXEC)	=	3</usr/lib/locale/locale-archive>
fstat(3</usr/lib/locale/locale-archive>,	{st_mode=S_IFREG|0644,	st_size=1607760,	...})	=	0
mmap(NULL,	1607760,	PROT_READ,	MAP_PRIVATE,	3</usr/lib/locale/locale-archive>,	0)	=	0x7f7601ea9000
close(3</usr/lib/locale/locale-archive>)	=	0
open("/etc/localtime",	O_RDONLY|O_CLOEXEC)	=	3</etc/localtime>
fstat(3</etc/localtime>,	{st_mode=S_IFREG|0644,	st_size=3661,	...})	=	0
fstat(3</etc/localtime>,	{st_mode=S_IFREG|0644,	st_size=3661,	...})	=	0
mmap(NULL,	4096,	PROT_READ|PROT_WRITE,	MAP_PRIVATE|MAP_ANONYMOUS,	-1,	0)	=	0x7f7602058000
read(3</etc/localtime>,	"TZif2000000000000000000700070000"...,	4096)	=	3661
lseek(3</etc/localtime>,	-2338,	SEEK_CUR)	=	1323
read(3</etc/localtime>,	"TZif200000000000000000010000100000"...,	4096)	=	2338
close(3</etc/localtime>)																=	0
munmap(0x7f7602058000,	4096)												=	0
fstat(1</dev/pts/50>,	{st_mode=S_IFCHR|0620,	st_rdev=makedev(136,	50),	...})	=	0
mmap(NULL,	4096,	PROT_READ|PROT_WRITE,	MAP_PRIVATE|MAP_ANONYMOUS,	-1,	0)	=	0x7f7602058000
write(1</dev/pts/50>,	"Mon	26	Sep	12:27:50	BST	2016n",	29Mon	26	Sep	12:27:50	BST	2016
)	=	29
close(1</dev/pts/50>)																			=	0
munmap(0x7f7602058000,	4096)												=	0
close(2</dev/pts/50>)																			=	0
exit_group(0)																											=	?
+++	exited	with	0	+++
Subset	of	syscalls	-	file	operations:
>	strace	-y	-e	trace=file	date
execve("/bin/date",	["date"],	[/*	34	vars	*/])	=	0
access("/etc/ld.so.nohwcap",	F_OK)						=	-1	ENOENT	(No	such	file	or	directory)
access("/etc/ld.so.preload",	R_OK)						=	-1	ENOENT	(No	such	file	or	directory)
open("/etc/ld.so.cache",	O_RDONLY|O_CLOEXEC)	=	3</etc/ld.so.cache>
access("/etc/ld.so.nohwcap",	F_OK)						=	-1	ENOENT	(No	such	file	or	directory)
open("/lib/x86_64-linux-gnu/libc.so.6",	O_RDONLY|O_CLOEXEC)	=	3</lib/x86_64-linux-gnu/libc-2.19.so>
open("/usr/lib/locale/locale-archive",	O_RDONLY|O_CLOEXEC)	=	3</usr/lib/locale/locale-archive>
open("/etc/localtime",	O_RDONLY|O_CLOEXEC)	=	3</etc/localtime>
Mon	26	Sep	12:29:01	BST	2016
+++	exited	with	0	+++
Subset	of	syscalls	-	memory	operations:
>	strace	-y	-e	trace=memory	date
brk(0)																																		=	0x25b8000
mmap(NULL,	8192,	PROT_READ|PROT_WRITE,	MAP_PRIVATE|MAP_ANONYMOUS,	-1,	0)	=	0x7f14cc871000
mmap(NULL,	144491,	PROT_READ,	MAP_PRIVATE,	3</etc/ld.so.cache>,	0)	=	0x7f14cc84d000
mmap(NULL,	3844640,	PROT_READ|PROT_EXEC,	MAP_PRIVATE|MAP_DENYWRITE,	3</lib/x86_64-linux-gnu/libc-2.19.so>,	0)	=	0x7f14cc2a8000
mprotect(0x7f14cc44a000,	2093056,	PROT_NONE)	=	0
mmap(0x7f14cc649000,	24576,	PROT_READ|PROT_WRITE,	MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE,	3</lib/x86_64-linux-gnu/libc-2.19.so>,	0x1a1000)	=	
0x7f14cc649000
mmap(0x7f14cc64f000,	14880,	PROT_READ|PROT_WRITE,	MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS,	-1,	0)	=	0x7f14cc64f000
mmap(NULL,	4096,	PROT_READ|PROT_WRITE,	MAP_PRIVATE|MAP_ANONYMOUS,	-1,	0)	=	0x7f14cc84c000
mmap(NULL,	4096,	PROT_READ|PROT_WRITE,	MAP_PRIVATE|MAP_ANONYMOUS,	-1,	0)	=	0x7f14cc84b000
mmap(NULL,	4096,	PROT_READ|PROT_WRITE,	MAP_PRIVATE|MAP_ANONYMOUS,	-1,	0)	=	0x7f14cc84a000
mprotect(0x7f14cc649000,	16384,	PROT_READ)	=	0
mprotect(0x60e000,	4096,	PROT_READ)					=	0
mprotect(0x7f14cc873000,	4096,	PROT_READ)	=	0
munmap(0x7f14cc84d000,	144491)										=	0
brk(0)																																		=	0x25b8000
brk(0x25d9000)																										=	0x25d9000
mmap(NULL,	1607760,	PROT_READ,	MAP_PRIVATE,	3</usr/lib/locale/locale-archive>,	0)	=	0x7f14cc6c1000
mmap(NULL,	4096,	PROT_READ|PROT_WRITE,	MAP_PRIVATE|MAP_ANONYMOUS,	-1,	0)	=	0x7f14cc870000
munmap(0x7f14cc870000,	4096)												=	0
mmap(NULL,	4096,	PROT_READ|PROT_WRITE,	MAP_PRIVATE|MAP_ANONYMOUS,	-1,	0)	=	0x7f14cc870000
Mon	26	Sep	12:29:40	BST	2016
munmap(0x7f14cc870000,	4096)												=	0
+++	exited	with	0	+++
Summary:
Strace.
Not	perfect	-	only	works	on	syscall	level.
But	still	useful	for	low-level	investigations.
No	recompilation	required.
Overview:
Valgrind.
Linux,	OS	X,	Solaris,	Android.
Very	detailed	checking	of	execution.
Free.
Similar	to	Purify	etc.
Memory	checking:
Thread	checking.
Other:
Valgrind.
Illegal	memory	accesses:
Overrun/under	run	heap	blocks.
Overrun	stack.
Use-after-free.
Double	free.
Memory	leaks.
Inconsistent	lock	orderings.
Data	races	(e.g.	missing	mutex).
CPU	cache	behaviour.
Heap	profiler.
Highly	recommended!
Valgrind.
New	debugging	technology	in	recent	years.
Recording	execution.
Linux:
Undo	Live	Recorder.
RR.
Windows:
Intellitrace	(partial	recording	only).
TimeMachineFor.Net	(partial	recording	only).
Java:
Chronon.
Undo	(soon).
Live	Recorder.
A	library,	for	linking	into	an	application.
Allows	the	application	to	control	the	recording	of	its	own	execution.
Provides	a	simple	C	API	to	start/save/stop	recording.
API	is	defined	in	undolr.h	header	file	and	implemented	in	libundolr	library.
Live	Recorder.
Live	Recorder	recordings:
Are	standard	Undo	Recording	files.
Contain	everything	need	to	replay	execution:
Non-deterministic	events	(inputs	to	program).
Initial	state	(initial	memory	and	registers).
Also	contain	information	needed	for	symbolic	debugging:
Complete	executable	and	.so	files.
Debuginfo	files.
Allows	debugging	even	when	libraries	and/or	debug	information	is	not	available	locally	(e.g.	load	and	replay	on	a
different	Linux	distribution).
Loaded	into	UndoDB:
undodb-gdb	--undodb-load	<filename>
(undodb-gdb)	undodb-load	<filename>
Full	reversible	debugging.
Live	Recorder.
Library	API	(undolr.h):
int	undolr_recording_start(	undolr_error_t*	o_error);
int	undolr_recording_stop(	void);
int	undolr_recording_save(	const	char*	filename);
int	undolr_recording_stop_and_save(	const	char*	filename);
int	undolr_save_on_termination(	const	char*	filename);
int	undolr_save_on_termination_cancel(	void);
int	undolr_event_log_size_get(	long*	o_bytes);
int	undolr_event_log_size_set(	long	bytes);
int	undolr_include_symbol_files(	int	include);
Live	Recorder.
Use	Live	Recorder:
In	internal	testing.
At	customer	site.
Advantages:
Investigate	bugs	easily	using	reversible	debugging.
You	or	your	customer	control	when/if	recording	is	enabled.
Avoid	problems	with	differing	environments.
No	need	to	reproduce	complex	multi-machine	setups.
Have	multiple	developers	work	on	the	same	test	failure.
Live	Recorder.
Save	a	recording	only	if	something	went	wrong.
...
				undolr_save_on_termination_cancel();
				exit(0);
...
				exit(1);
...
int	main(	int	argv,	char**	argv)
{
				undolr_recording_start(	NULL);
				undolr_save_on_termination(	"failure.undo");
				...
				if	(...)
				{
								return	1;
				}
				...
				undolr_save_on_termination_cancel();
				return	0;
}
Live	Recorder:	Demo.
Questions?
Live	Recorder.
Execution	Analysis
Traditional	debugging	only	gives	partial	information:
Core	file:
Gives	programme	state	only	at	the	end	of	execution.
Traditional	debugger:
Cannot	go	back	to	an	earlier	state.
Valgrind:
Have	to	know	in	advance	what	type	of	error	is	going	to	happen,	and	choose	appropriate	engine:
Cachegrind
Memcheck
Callgrind
Helgrind
What	if	you	run	with	Helgrind	but	encounter	an	intermittent	memory	error?
Or	you	run	with	Memcheck	and	get	a	rare	race	condition?
Runs	slowly	because	Valgrind	engines	are	doing	very	complicated	analysis.
Logging:
Shows	information	only	from	the	logging	that	is	enabled.
Usually	not	what	you	need	to	fix	the	bug.
So	re-run	with	different	logging.
Hope	that	bug	isn’t	intermittent.
An	Undo	recording	gives	you	100%	knowledge	of	what	your
programme	did:
Undo	is	faster	than	Valgrind:
Execution	Analysis
Proof:	You	can	find	to	the	state	of	the	entire	programme	(memory	plus	registers)	at	any	point	in	its	execution.
stepi
stepi
…
and/or:
reverse-stepi
reverse-stepi
…
Because	it	is	not	performing	analysis	when	recording.
But	this	isn’t	always	enough	to	track	down	really	difficult	bugs.
Execution	Analysis
More	useful	navigation	of	programme	execution:
Normal	and	reverse	debugging:
undodb-goto-time	...
next,	finish,	reverse-next,	reverse-finish,	…
Breakpoints
Watchpoints.
Also:
Move	to	next/previous	event	matching	a	criteria:
ugo	event	prev	name=='read'	or	name=='mmap'
ugo	event	prev	name=='read'	and	result	<	0
Execution	Analysis
How	could	we	make	better	use	of	this	100%	complete	knowledge?
Replay	exact	execution,	but	gather	/	output	extra	information:
Annotate	replay	with	extra	diagnostics	from	debugger	(without	changing	the	execution).	E.g.	gdb’s	breakpoint
commands.
Replay	with	Memcheck-style	memory	checking.
Replay	with	Helgrind-style	memory	checking.
Log	all	calls	to	a	particular	set	of	functions.
Navigate	forwards/backwards	to	next/previous	call	of	a	particular	set	of	functions.
Log	locks	of	a	particular	mutex.
Navigate	forwards/backwards	to	next/previous	lock	of	a	particular	mutex.
Go	back	to	last	call	to	a	function	with	particular	parameter	values.
…
Suggestions	?
With	a	recording,	it	must	be	possible	to	figure	out	the	bug.
Execution	Analysis
How	to	find	and	fix	a	bug:
Replay	recording,	use	forward/reverse	commands,	watchpoints,	breakpoints	to	investigate.
If	not	able	to	figure	it	out:
Replay	with	appropriate	annotations	or	valgrind-style	checking	to	find	more	information.
Repeat	until	bug	is	found.
Because	recording	contains	100%	information	about	the	programme’s	execution.
Execution	Analysis
Summary	of	Undo	advantages:
Constant	modest	speed	overhead:
Undo	recording	is	faster	than	Valgrind	doing	memory/thread/cache	analysis.
Incur	large	analysis	overhead	only	when	required.
Retrospective	analysis:
Run	programme	today.
Save	recording.
Analyse	against	new	regulations	in	6	months	time.
In	development:	Dataflow	Analysis.
This	is	internal	research	and	development.
A	particular	type	of	execution	analysis.
Not	available	yet.
Subject	to	change	before	release.
In	development:	Dataflow	Analysis.
When	debugging,	we	usually	focus	on	function	calls:
Look	at	functions	in	the	stack	trace.
Step	forwards	with	stepi,	step,	nexti,	next,	finish.
Step	backwards	with	reverse-stepi,	reverse-step,	reverse-nexti,	reverse-next,	reverse-finish.
Move	forwards	and	backwards	with	continue	and	reverse-continue.
Use	breakpoints.
Use	watchpoints.
Then	look	at	state	of	data	wherever	we	end	up:
Registers.
Local	variables.
Global	variables.
In	development:	Dataflow	Analysis.
Can	we	instead	look	more	generally	at	data?
What	inputs	does	current	CPU	instruction	use,	and	where	did	these	inputs	come	from?
New	undodb-gdb	command:	undodb-current-instruction-info
Show	inputs/outputs	of	current	instruction.
Example,	instruction	sub	-0x8(%rbp),%eax:
(undodb-gdb)	undodb-current-instruction-info
undodb-gdb:	current	instruction	is	at	0x4005df:
undodb-gdb:					reads:
undodb-gdb:									memory			0x7ffc8d6ed6d8..+4
undodb-gdb:									register	0	(ax)
undodb-gdb:					writes:
undodb-gdb:									register	0	(ax)
(undodb-gdb)
To	follow	the	data:
We	can	set	a	4-byte	watchpoint	on	address	0x7ffc8d6ed6d8.
We	need	to	be	able	to	set	a	"watchpoint"	on	register	%ax.
Not	supported	by	CPU	hardware.
In	development:	Dataflow	Analysis.
But	Undo	knows	everything	about	execution.
So	we	can	make	it	specially	instrument	code	to	do	register	watchpoints.
New	undodb-gdb	command:	undodb-search-registers
Search	backwards	for	reads/writes	to	registers.
Example:
(undodb-gdb)	undodb-search-registers	ax,bp	ds
Goes	back	to	most	recently	executed	instruction	that	did	any	of:
Read	register	%ax.
Read	register	%bp.
Wrote	to	register	%ds.
Notes:
This	is	only	possible	because	we	have	a	record/reply	engine.
Without	record/replay	engine,	combinatorics	make	things	impossible.
Only	possible	if	we	modify	replay	to	also	track	changes	to	registers.
In	development:	Dataflow	Analysis.
Putting	it	together:
We	can	look	at	what	register/memory	is	used	by	current	instruction.
We	can	go	back	to	most	recent	time	register/memory	was	changed.
Reverse	dataflow	analysis:
If	current	instruction	reads	one	memory	location:
Set	watchpoint.
Reverse	continue.
If	current	instruction	reads	one	register:
Use	undodb-search-registers.
Dataflow	is	a	tree.
In	development:	Dataflow	Analysis.
But	what	if	current	instruction	…
…	reads	more	than	one	register?
…	reads	register	and	memory?
…	reads	more	than	one	location	in	memory?
In	development:	Dataflow	Analysis.
Semi-automatic	dataflow	analysis:
Explore	dataflow	tree	one	step	at	a	time.
At	each	step:
Use	memory-watchpoint+reverse-continue,	or	undob-search-registers.
Remember	results	of	each	step.
Breadth-first	search.
(undodb-gdb)	undodb-dataflow-backwards-explore-start
(undodb-gdb)	undodb-dataflow-backwards-explore-next
(undodb-gdb)	undodb-dataflow-backwards-explore-next
...
undodb-gdb:	Have	finished	dataflow	exploration
(undodb-gdb)	undodb-dataflow-backwards-explore-show
In	development:	Dataflow	Analysis.
Example	output:
(undodb-gdb)	undodb-dataflow-backwards-explore-show
undodb-gdb:	Dataflow	is	(13):
undodb-gdb:					316038:0x400728	=>	316039:0x400735:	memory	0x7ffd23ec1214..+4
undodb-gdb:					316039:0x400732	=>	316039:0x400735:	register	0	(ax)
undodb-gdb:					316029:0x4006f6	=>	316040:0x40071e:	memory	0x7ffd23ec11f0..+4
undodb-gdb:					316029:0x4006f6	=>	316040:0x40071e:	memory	0x7ffd23ec11f0..+4
undodb-gdb:					316040:0x40071e	=>	316040:0x400722:	register	0	(ax)
undodb-gdb:					316040:0x40071e	=>	316040:0x400722:	register	0	(ax)
undodb-gdb:					316040:0x400722	=>	316040:0x400725:	memory	0x7ffd23ec11fc..+4
undodb-gdb:					316040:0x400725	=>	316040:0x400728:	register	0	(ax)
undodb-gdb:					316039:0x400735	=>	316040:0x400728:	memory	0x7ffd23ec1214..+4
undodb-gdb:					316040:0x400722	=>	316041:0x400732:	memory	0x7ffd23ec11fc..+4
undodb-gdb:					316040:0x400728	=>	316041:0x400735:	memory	0x7ffd23ec1214..+4
undodb-gdb:					316041:0x400732	=>	316041:0x400735:	register	0	(ax)
undodb-gdb:					316041:0x400735	=>	316042:0x400747:	memory	0x7ffd23ec1214..+4
(undodb-gdb)
In	development:	Dataflow	Analysis.
Questions?
In	development:	Dataflow	Analysis.
What	could	we	do	with	reverse	dataflow?
Easier	tracing	of	the	ultimate	cause	of	incorrect	data.
Test-case	synthesis	-	find	what	inputs	would	be	required	to	make	programme	take	a	particular	path.
Issues:
Currently	only	detect	reads/writes	to	general	purpose	registers.
Consider:	add	%eax,-0xc(%rbp)
Reads	memory	at	%bp-0xc.
Does	this	also	read	register	%bp	?
(Currently	we	say:	No.)
Currently	slow.
EOF.
http://undo.io/

More Related Content

Similar to Next Generation Debugging

Fundamentals of DevOps and CI/CD
Fundamentals of DevOps and CI/CDFundamentals of DevOps and CI/CD
Fundamentals of DevOps and CI/CDBatyr Nuryyev
 
Why Software Test Performance Matters
Why Software Test Performance MattersWhy Software Test Performance Matters
Why Software Test Performance MattersSolano Labs
 
Intro to Continuous Integration
Intro to Continuous IntegrationIntro to Continuous Integration
Intro to Continuous IntegrationTal Mor (Moshayov)
 
Enabling Agile Testing Through Continuous Integration Agile2009
Enabling Agile Testing Through Continuous Integration Agile2009Enabling Agile Testing Through Continuous Integration Agile2009
Enabling Agile Testing Through Continuous Integration Agile2009sstolberg
 
UIAutomation_Testing
UIAutomation_TestingUIAutomation_Testing
UIAutomation_TestingKrunal Soni
 
[Srijan Wednesday Webinar] Mastering Mobile Test Automation with Appium
[Srijan Wednesday Webinar] Mastering Mobile Test Automation with Appium[Srijan Wednesday Webinar] Mastering Mobile Test Automation with Appium
[Srijan Wednesday Webinar] Mastering Mobile Test Automation with AppiumSrijan Technologies
 
Rapid deployment models for uPortal
Rapid deployment models for uPortalRapid deployment models for uPortal
Rapid deployment models for uPortalTom Freestone
 
Gatling - Bordeaux JUG
Gatling - Bordeaux JUGGatling - Bordeaux JUG
Gatling - Bordeaux JUGslandelle
 
Automated testing san francisco oct 2013
Automated testing san francisco oct 2013Automated testing san francisco oct 2013
Automated testing san francisco oct 2013Solano Labs
 
Agile Engineering Best Practices by Richard Cheng
Agile Engineering Best Practices by Richard ChengAgile Engineering Best Practices by Richard Cheng
Agile Engineering Best Practices by Richard ChengExcella
 
Selenium conference, 2016
Selenium conference, 2016Selenium conference, 2016
Selenium conference, 2016Pooja Shah
 
Continuous Deployment at Etsy — TimesOpen NYC
Continuous Deployment at Etsy — TimesOpen NYCContinuous Deployment at Etsy — TimesOpen NYC
Continuous Deployment at Etsy — TimesOpen NYCMike Brittain
 
Unit Testing Fundamentals
Unit Testing FundamentalsUnit Testing Fundamentals
Unit Testing FundamentalsRichard Paul
 
Google test training
Google test trainingGoogle test training
Google test trainingThierry Gayet
 

Similar to Next Generation Debugging (20)

Evergreen build
Evergreen buildEvergreen build
Evergreen build
 
Fundamentals of DevOps and CI/CD
Fundamentals of DevOps and CI/CDFundamentals of DevOps and CI/CD
Fundamentals of DevOps and CI/CD
 
DevOps and AWS
DevOps and AWSDevOps and AWS
DevOps and AWS
 
Why Software Test Performance Matters
Why Software Test Performance MattersWhy Software Test Performance Matters
Why Software Test Performance Matters
 
NYC MeetUp 10.9
NYC MeetUp 10.9NYC MeetUp 10.9
NYC MeetUp 10.9
 
Intro to Continuous Integration
Intro to Continuous IntegrationIntro to Continuous Integration
Intro to Continuous Integration
 
Enabling Agile Testing Through Continuous Integration Agile2009
Enabling Agile Testing Through Continuous Integration Agile2009Enabling Agile Testing Through Continuous Integration Agile2009
Enabling Agile Testing Through Continuous Integration Agile2009
 
Unit Testing
Unit TestingUnit Testing
Unit Testing
 
UIAutomation_Testing
UIAutomation_TestingUIAutomation_Testing
UIAutomation_Testing
 
Gatling
GatlingGatling
Gatling
 
[Srijan Wednesday Webinar] Mastering Mobile Test Automation with Appium
[Srijan Wednesday Webinar] Mastering Mobile Test Automation with Appium[Srijan Wednesday Webinar] Mastering Mobile Test Automation with Appium
[Srijan Wednesday Webinar] Mastering Mobile Test Automation with Appium
 
The Test way
The Test wayThe Test way
The Test way
 
Rapid deployment models for uPortal
Rapid deployment models for uPortalRapid deployment models for uPortal
Rapid deployment models for uPortal
 
Gatling - Bordeaux JUG
Gatling - Bordeaux JUGGatling - Bordeaux JUG
Gatling - Bordeaux JUG
 
Automated testing san francisco oct 2013
Automated testing san francisco oct 2013Automated testing san francisco oct 2013
Automated testing san francisco oct 2013
 
Agile Engineering Best Practices by Richard Cheng
Agile Engineering Best Practices by Richard ChengAgile Engineering Best Practices by Richard Cheng
Agile Engineering Best Practices by Richard Cheng
 
Selenium conference, 2016
Selenium conference, 2016Selenium conference, 2016
Selenium conference, 2016
 
Continuous Deployment at Etsy — TimesOpen NYC
Continuous Deployment at Etsy — TimesOpen NYCContinuous Deployment at Etsy — TimesOpen NYC
Continuous Deployment at Etsy — TimesOpen NYC
 
Unit Testing Fundamentals
Unit Testing FundamentalsUnit Testing Fundamentals
Unit Testing Fundamentals
 
Google test training
Google test trainingGoogle test training
Google test training
 

Recently uploaded

Try MyIntelliAccount Cloud Accounting Software As A Service Solution Risk Fre...
Try MyIntelliAccount Cloud Accounting Software As A Service Solution Risk Fre...Try MyIntelliAccount Cloud Accounting Software As A Service Solution Risk Fre...
Try MyIntelliAccount Cloud Accounting Software As A Service Solution Risk Fre...MyIntelliSource, Inc.
 
Alluxio Monthly Webinar | Cloud-Native Model Training on Distributed Data
Alluxio Monthly Webinar | Cloud-Native Model Training on Distributed DataAlluxio Monthly Webinar | Cloud-Native Model Training on Distributed Data
Alluxio Monthly Webinar | Cloud-Native Model Training on Distributed DataAlluxio, Inc.
 
Unveiling Design Patterns: A Visual Guide with UML Diagrams
Unveiling Design Patterns: A Visual Guide with UML DiagramsUnveiling Design Patterns: A Visual Guide with UML Diagrams
Unveiling Design Patterns: A Visual Guide with UML DiagramsAhmed Mohamed
 
Asset Management Software - Infographic
Asset Management Software - InfographicAsset Management Software - Infographic
Asset Management Software - InfographicHr365.us smith
 
The Evolution of Karaoke From Analog to App.pdf
The Evolution of Karaoke From Analog to App.pdfThe Evolution of Karaoke From Analog to App.pdf
The Evolution of Karaoke From Analog to App.pdfPower Karaoke
 
Building Real-Time Data Pipelines: Stream & Batch Processing workshop Slide
Building Real-Time Data Pipelines: Stream & Batch Processing workshop SlideBuilding Real-Time Data Pipelines: Stream & Batch Processing workshop Slide
Building Real-Time Data Pipelines: Stream & Batch Processing workshop SlideChristina Lin
 
Advancing Engineering with AI through the Next Generation of Strategic Projec...
Advancing Engineering with AI through the Next Generation of Strategic Projec...Advancing Engineering with AI through the Next Generation of Strategic Projec...
Advancing Engineering with AI through the Next Generation of Strategic Projec...OnePlan Solutions
 
Implementing Zero Trust strategy with Azure
Implementing Zero Trust strategy with AzureImplementing Zero Trust strategy with Azure
Implementing Zero Trust strategy with AzureDinusha Kumarasiri
 
MYjobs Presentation Django-based project
MYjobs Presentation Django-based projectMYjobs Presentation Django-based project
MYjobs Presentation Django-based projectAnoyGreter
 
Building a General PDE Solving Framework with Symbolic-Numeric Scientific Mac...
Building a General PDE Solving Framework with Symbolic-Numeric Scientific Mac...Building a General PDE Solving Framework with Symbolic-Numeric Scientific Mac...
Building a General PDE Solving Framework with Symbolic-Numeric Scientific Mac...stazi3110
 
办理学位证(UQ文凭证书)昆士兰大学毕业证成绩单原版一模一样
办理学位证(UQ文凭证书)昆士兰大学毕业证成绩单原版一模一样办理学位证(UQ文凭证书)昆士兰大学毕业证成绩单原版一模一样
办理学位证(UQ文凭证书)昆士兰大学毕业证成绩单原版一模一样umasea
 
Cloud Management Software Platforms: OpenStack
Cloud Management Software Platforms: OpenStackCloud Management Software Platforms: OpenStack
Cloud Management Software Platforms: OpenStackVICTOR MAESTRE RAMIREZ
 
EY_Graph Database Powered Sustainability
EY_Graph Database Powered SustainabilityEY_Graph Database Powered Sustainability
EY_Graph Database Powered SustainabilityNeo4j
 
What is Fashion PLM and Why Do You Need It
What is Fashion PLM and Why Do You Need ItWhat is Fashion PLM and Why Do You Need It
What is Fashion PLM and Why Do You Need ItWave PLM
 
software engineering Chapter 5 System modeling.pptx
software engineering Chapter 5 System modeling.pptxsoftware engineering Chapter 5 System modeling.pptx
software engineering Chapter 5 System modeling.pptxnada99848
 
KnowAPIs-UnknownPerf-jaxMainz-2024 (1).pptx
KnowAPIs-UnknownPerf-jaxMainz-2024 (1).pptxKnowAPIs-UnknownPerf-jaxMainz-2024 (1).pptx
KnowAPIs-UnknownPerf-jaxMainz-2024 (1).pptxTier1 app
 
Russian Call Girls in Karol Bagh Aasnvi ➡️ 8264348440 💋📞 Independent Escort S...
Russian Call Girls in Karol Bagh Aasnvi ➡️ 8264348440 💋📞 Independent Escort S...Russian Call Girls in Karol Bagh Aasnvi ➡️ 8264348440 💋📞 Independent Escort S...
Russian Call Girls in Karol Bagh Aasnvi ➡️ 8264348440 💋📞 Independent Escort S...soniya singh
 
Professional Resume Template for Software Developers
Professional Resume Template for Software DevelopersProfessional Resume Template for Software Developers
Professional Resume Template for Software DevelopersVinodh Ram
 

Recently uploaded (20)

Try MyIntelliAccount Cloud Accounting Software As A Service Solution Risk Fre...
Try MyIntelliAccount Cloud Accounting Software As A Service Solution Risk Fre...Try MyIntelliAccount Cloud Accounting Software As A Service Solution Risk Fre...
Try MyIntelliAccount Cloud Accounting Software As A Service Solution Risk Fre...
 
Alluxio Monthly Webinar | Cloud-Native Model Training on Distributed Data
Alluxio Monthly Webinar | Cloud-Native Model Training on Distributed DataAlluxio Monthly Webinar | Cloud-Native Model Training on Distributed Data
Alluxio Monthly Webinar | Cloud-Native Model Training on Distributed Data
 
Unveiling Design Patterns: A Visual Guide with UML Diagrams
Unveiling Design Patterns: A Visual Guide with UML DiagramsUnveiling Design Patterns: A Visual Guide with UML Diagrams
Unveiling Design Patterns: A Visual Guide with UML Diagrams
 
Asset Management Software - Infographic
Asset Management Software - InfographicAsset Management Software - Infographic
Asset Management Software - Infographic
 
The Evolution of Karaoke From Analog to App.pdf
The Evolution of Karaoke From Analog to App.pdfThe Evolution of Karaoke From Analog to App.pdf
The Evolution of Karaoke From Analog to App.pdf
 
Building Real-Time Data Pipelines: Stream & Batch Processing workshop Slide
Building Real-Time Data Pipelines: Stream & Batch Processing workshop SlideBuilding Real-Time Data Pipelines: Stream & Batch Processing workshop Slide
Building Real-Time Data Pipelines: Stream & Batch Processing workshop Slide
 
Advancing Engineering with AI through the Next Generation of Strategic Projec...
Advancing Engineering with AI through the Next Generation of Strategic Projec...Advancing Engineering with AI through the Next Generation of Strategic Projec...
Advancing Engineering with AI through the Next Generation of Strategic Projec...
 
Implementing Zero Trust strategy with Azure
Implementing Zero Trust strategy with AzureImplementing Zero Trust strategy with Azure
Implementing Zero Trust strategy with Azure
 
MYjobs Presentation Django-based project
MYjobs Presentation Django-based projectMYjobs Presentation Django-based project
MYjobs Presentation Django-based project
 
Building a General PDE Solving Framework with Symbolic-Numeric Scientific Mac...
Building a General PDE Solving Framework with Symbolic-Numeric Scientific Mac...Building a General PDE Solving Framework with Symbolic-Numeric Scientific Mac...
Building a General PDE Solving Framework with Symbolic-Numeric Scientific Mac...
 
办理学位证(UQ文凭证书)昆士兰大学毕业证成绩单原版一模一样
办理学位证(UQ文凭证书)昆士兰大学毕业证成绩单原版一模一样办理学位证(UQ文凭证书)昆士兰大学毕业证成绩单原版一模一样
办理学位证(UQ文凭证书)昆士兰大学毕业证成绩单原版一模一样
 
Cloud Management Software Platforms: OpenStack
Cloud Management Software Platforms: OpenStackCloud Management Software Platforms: OpenStack
Cloud Management Software Platforms: OpenStack
 
EY_Graph Database Powered Sustainability
EY_Graph Database Powered SustainabilityEY_Graph Database Powered Sustainability
EY_Graph Database Powered Sustainability
 
What is Fashion PLM and Why Do You Need It
What is Fashion PLM and Why Do You Need ItWhat is Fashion PLM and Why Do You Need It
What is Fashion PLM and Why Do You Need It
 
Call Girls In Mukherjee Nagar 📱 9999965857 🤩 Delhi 🫦 HOT AND SEXY VVIP 🍎 SE...
Call Girls In Mukherjee Nagar 📱  9999965857  🤩 Delhi 🫦 HOT AND SEXY VVIP 🍎 SE...Call Girls In Mukherjee Nagar 📱  9999965857  🤩 Delhi 🫦 HOT AND SEXY VVIP 🍎 SE...
Call Girls In Mukherjee Nagar 📱 9999965857 🤩 Delhi 🫦 HOT AND SEXY VVIP 🍎 SE...
 
software engineering Chapter 5 System modeling.pptx
software engineering Chapter 5 System modeling.pptxsoftware engineering Chapter 5 System modeling.pptx
software engineering Chapter 5 System modeling.pptx
 
KnowAPIs-UnknownPerf-jaxMainz-2024 (1).pptx
KnowAPIs-UnknownPerf-jaxMainz-2024 (1).pptxKnowAPIs-UnknownPerf-jaxMainz-2024 (1).pptx
KnowAPIs-UnknownPerf-jaxMainz-2024 (1).pptx
 
Russian Call Girls in Karol Bagh Aasnvi ➡️ 8264348440 💋📞 Independent Escort S...
Russian Call Girls in Karol Bagh Aasnvi ➡️ 8264348440 💋📞 Independent Escort S...Russian Call Girls in Karol Bagh Aasnvi ➡️ 8264348440 💋📞 Independent Escort S...
Russian Call Girls in Karol Bagh Aasnvi ➡️ 8264348440 💋📞 Independent Escort S...
 
Hot Sexy call girls in Patel Nagar🔝 9953056974 🔝 escort Service
Hot Sexy call girls in Patel Nagar🔝 9953056974 🔝 escort ServiceHot Sexy call girls in Patel Nagar🔝 9953056974 🔝 escort Service
Hot Sexy call girls in Patel Nagar🔝 9953056974 🔝 escort Service
 
Professional Resume Template for Software Developers
Professional Resume Template for Software DevelopersProfessional Resume Template for Software Developers
Professional Resume Template for Software Developers
 

Next Generation Debugging