https://seL4.systems	
Gernot	Heiser	|	Microkernel	Dude	
Gernot.Heiser@data61.csiro.au	|	@GernotHeiser	
	
LCA’18	
Flying	Autonomous	Aircraft	
Mixed-Criticality	Support	in	seL4
Why	Should	You	Listen	To	This?	
In	this	talk	I’ll	explain:	
•  what	mixed-criticality	system	(MCS)	are,	and	why	are	they	important	
•  what	their	certification	needs	are	
•  what	MCS	need	from	the	OS:	spatial	and	temporal	isolation	
•  how	we	support	MCS	in	seL4,	the	world’s	most	secure	OS	
•  what	we	are	using	it	for	
Flying	autonomous	aircraft:	Mixed-criticality	support	in	seL4	|	LCA'18	2		|
Cyberphysical	Systems	Software	Challenge	
Flying	autonomous	aircraft:	Mixed-criticality	support	in	seL4	|	LCA'18	3		|	
•  Growing	functionality	
•  Much	safety-critical	functionality	
•  Expensive	safety	assurance	processes	
•  Cost	at	least	linear	in	LoC	
8	MSLOC	
120	MSLOC
Traditional	Approach:	Physical	Separation	
Flying	autonomous	aircraft:	Mixed-criticality	support	in	seL4	|	LCA'18	4		|
Example:	Microcontroller	in	a	Car	
Flying	autonomous	aircraft:	Mixed-criticality	support	in	seL4	|	LCA'18	5		|	
Electronic	control	unit	(ECU)	must	
•  be	water	proof	
•  be	dust	proof	
•  be	grease	proof	
•  be	acid	proof	
•  be	highly	vibration	resistant	
•  operate	-30°C	to	80°C
Traditional	Approach:	Physical	Separation	
Flying	autonomous	aircraft:	Mixed-criticality	support	in	seL4	|	LCA'18	6		|	
Too	limited:	
•  Scalability:	100s	of	
microcontrollers	create	
space,	weight	and	power	
(SWaP)	problem	
•  Sensor	fusion:	functions	
require	multiple	sensors,	
same	sensors	required	for	
multiple	functions
Processor	Consolidation	
Flying	autonomous	aircraft:	Mixed-criticality	support	in	seL4	|	LCA'18	7		|	
•  Reduces	SWaP	
	reduced	cost	
•  Improves	integration	
	richer	functionality	
•  Essential	for	
autonomous	vehicles	
Challenge:		
•  Loss	of	physical	isolation	
 huge	assurance	
problem
Safety-Critical	System	Assurance	
•  Every	part	of	a	safety-critical	system	must	be	certified	
•  Certification	asserts	that	certifier	is	convinced	system	will	behave	safely	
•  Assurance	process	exists	to	convince	certifier	
•  extensive	specs,	development	documentation	
•  extensive	testing	&	its	documentation	
•  extensive	code	inspection	
•  tracing	of	requirements	to	code	
•  convincing	argument	that	no	out-of-spec	behaviour	exists	
•  At	highest	safety	levels,	cost	is	prohibitive	for	code	bases	exceeding	a	few	kLOC	
Flying	autonomous	aircraft:	Mixed-criticality	support	in	seL4	|	LCA'18	8		|
How	Certify	a	Consolidated	System?	
Flying	autonomous	aircraft:	Mixed-criticality	support	in	seL4	|	LCA'18	9		|	
Sensor	
driver	
Actuator	
driver	
Control	
Sensor	
driver	
Actuator	
driver	
Control	
Sensor	
driver	
Actuator	
driver	
Control	
Software	
isolation!	
Operating	System
Operating	System	
Reality:	Most	OSes	are	Hopless	at	Isolation	
Flying	autonomous	aircraft:	Mixed-criticality	support	in	seL4	|	LCA'18	10		|	
Sensor	
driver	
Actuator	
driver	
Control	
Sensor	
driver	
Actuator	
driver	
Control	
Sensor	
driver	
Actuator	
driver	
Control	
Reality	check:	Everything	
depends	on	everything!	
Trusted	computing	base	is	huge	
–	no	help	for	certification!
DO-178B	Design	Assurance	(Criticality)	Levels	
Criticality,	
development	
cost,	
assurance	
cost	
Avionics	
safety	
standard	
HAZARDOUS	
MAJOR	
MINOR	
CATASTROPHIC	
No	Effect
Mixed-Criticality	System	(MCS)	
•  Multiple	components	with	different	criticalities	on	same	system	
•  Idea:	Can	be	cost-effective,	if	certify	most	critical	stuff	in	isolation	
•  Requirement:	Nothing	must	depend	on	anything	less	critical!	
Operating	System	
Sensor	
driver	
Actuator	
driver	
Control	
Sensor	
driver	
Actuator	
driver	
Control	
Sensor	
driver	
Actuator	
driver	
Control
MCS:	Microkernel	Considered	Essential	
•  Multiple	components	with	different	criticalities	on	same	system	
•  Idea:	Can	be	cost-effective,	if	certify	most	critical	stuff	in	isolation	
•  Requirement:	Nothing	must	depend	on	anything	less	critical!	
High-Assurance	Microkernel	Operating	System	
Sensor	
driver	
Actuator	
driver	
Control	
Sensor	
driver	
Actuator	
driver	
Control	
Sensor	
driver	
Actuator	
driver	
Control
seL4	Microkernel:	Strong	Isolation	
High-Assurance	Microkernel	Operating	System	
Sensor	
driver	
Actuator	
driver	
Control	
Sensor	
driver	
Actuator	
driver	
Control	
Sensor	
driver	
Actuator	
driver	
Control	
︎	
•  Isolation	by	default	
•  Communication	where	
explicitly	enabled
Core	Security	Mechanism:	Capability	
Flying	autonomous	aircraft:	Mixed-criticality	support	in	seL4	|	LCA'18	15		|	
	Any	system	call	is	invoking	a	capability:	
err	=	method(	cap,	args	);	
Obj	reference	
Access	rights	
Capability	=	Access	Token:	
Prima-facie	evidence	of	privilege	
Eg.	read,	
write,	send,	
execute…	
Capabilities	provide:	
•  Fine-grained	access	
control	
•  Reasoning	about	
information	flow	
Eg.	thread,	
address	space	
Object
Capability-Protected	Objects	
•  Thread-control	blocks	(TCBs)	
•  Address	spaces	(page	table	objects:	PDs,	PTs)	
•  Endpoints	(IPC)	
•  Notifications	(binary	semaphores)	
•  Capability	spaces	(CNodes)	
•  Frames	
•  Interrupt	objects	(architecture	specific)	
•  Untyped	(free)	memory,	re-typeable	
Flying	autonomous	aircraft:	Mixed-criticality	support	in	seL4	|	LCA'18	16		|	
Capabilities	provide:	
•  Fine-grained	access	
control	
•  Reasoning	about	
information	flow
Abstract	
Model	
Integrity	
C	Imple-
mentation	
Confidentiality	 Availability	
Binary	code	
Proof	Proof	Proof	
Functional	
correctness	
Isolation	properties	
Translation	
correctness	
Exclusions	(at	present):	
• 	Initialisation	
• 	Privileged	state	&	caches	
• 	Multicore	
Worst-case	
execution	time	
World’s	fastest		
microkernel!	
Provable	Security	Enforcement	
Flying	autonomous	aircraft:	Mixed-criticality	support	in	seL4	|	LCA'18	17		|	
SPATIAL ISOLATION ONLY!
Temporal	Isolation	for	MCS	
Flying	autonomous	aircraft:	Mixed-criticality	support	in	seL4	|	LCA'18	18		|	
	High	 	Low	
Affect	execution	speed:	
Integrity	violation
New	Scheduling	Model:	
Enforcing	Temporal	Integrity
•  256	hard	priorities	(0–255)	
•  Priorities	are	strictly	observed,	suitable	for	real	time	
•  The	scheduler	will	always	pick	the	highest-prio	runnable	thread	
•  Round-robin	scheduling	within	prio	level	
•  Thread	scheduling	parameters:	
•  Priority	
•  Time	slice	
Flying	autonomous	aircraft:	Mixed-criticality	support	in	seL4	|	LCA'18	
Classical	L4	Scheduling	
Present	(Verified)	seL4	Master	Branch	
20		|	
prio0 255
Issue:		
•  Highest-prio	can	monopolise	CPU	
•  Priority	=	“importance”
Issue	with	Priority	=	Importance	
Flying	autonomous	aircraft:	Mixed-criticality	support	in	seL4	|	LCA'18	21		|	
Runs	every	100	ms	
for	few	millisecods	
Runs	frequently	but	for	
short	time	(order	of	µs)		
Control	
loop	Sensor	
readings	
NW		
driver	
NW	
interrupts	
NW	driver	must	preempt	control	loop	
• 	…	to	avoid	packet	loss	
• 	Driver	must	run	at	high	prio	
• 	Driver	must	be	trusted	not	to	monopolise	CPU
Critical	Sections	as	Shared	Servers	
Hoare-style	monitor	
Suitable	intra-core	
Semaphore	synchronisation	
Suitable	inter-core	
Messages	
Events	
Client1	
Client2	
Server1	 Server2	
server_1()	{	
			…	
				wait(										);	
				while	(1)	{	
								/*	critical	section	*/	
								Reply&wait(											);	
	}	
}	
	
client()	{	
				while	(1)	{	
										…	
	call(										);	
	…	
	signal(										);	
	…	
	wait(											);	
	}	
}	
	
server_2()	{	
			…	
				while	(1)	{	
	wait(											);	
	/*	critical	section	*/	
	signal(											);	
	}	
}	
	
Flying	autonomous	aircraft:	Mixed-criticality	support	in	seL4	|	LCA'18	22		|
Shared	Intra-Core	Servers	Implement	
Priority	Ceiling	Protocol	(IPCP)	
Flying	autonomous	aircraft:	Mixed-criticality	support	in	seL4	|	LCA'18	23		|	
IPCP:		
PS	=	max	(P1,	P2)	+	1	
Immediate	Priority	Ceiling:	
•  Requires	correct	priority	
configuration	
•  Deadlock-free	
•  Easy	to	implement	
•  Good	worst-case	blocking	times	
Client1	
P1	
Server	
PS	
Client2	
P2
Problem	With	Servers	As	Threads	
Flying	autonomous	aircraft:	Mixed-criticality	support	in	seL4	|	LCA'18	24		|	
Running	
Running	
Shared	server	has	
highest	prio,	runs	as	
long	as	it	has	work	
Has	used	no	time,	
Keeps	running	
Can	effectively	DoS		
same-prio	threads,	
no	temporal	isolation!	
Client1	
P1	
Server	
PS	
Client2	
P2
Requirements	for	MCS	
•  Certifiable	spatial	isolation	
•  Certifiable	temporal	isolation:	
•  Ability	to	guarantee	deadlines	without	trusting	low-criticality,	high-priority	processes	
•  Ability	to	share	resources	(servers)	safely,	even	across	criticalities	
•  Ability	to	re-use	all	slack	for	low-criticality	processes	
•  Desirable	for	seL4:	capabilities	for	time	control	
Flying	autonomous	aircraft:	Mixed-criticality	support	in	seL4	|	LCA'18	25		|
Scheduling	Contexts:	Caps	for	Time	
Classical	thread	attributes	
•  Priority	
•  Time	slice	
New	thread	attributes	
•  Priority	
•  Scheduling	context	capability	
Flying	autonomous	aircraft:	Mixed-criticality	support	in	seL4	|	LCA'18	26		|	
Not	
runnable	
if	null	
Not	
runnable	
if	null	
Scheduling	context	object	
•  T:	period	
•  C:	budget	(≤	T)	
Limits	CPU	
access!	
SchedControl	capability	
conveys	right	to	assign	
budgets	(i.e.	perform	
admission	control)	
C	=	2	
T	=	3	
C	=		250	
T	=	1000	
Capability	
for	time
Scheduling	Guarantees	
•  Kernel	will	run	highest-priority	runnable	thread	with	non-zero	budget	
•  Thread	with	no	budget	cannot	run	until	next	period	
•  Within	priority,	threads	are	scheduled	round-robin	
Flying	autonomous	aircraft:	Mixed-criticality	support	in	seL4	|	LCA'18	27		|	
Criticality	 Period	 Budget Utilisation	 Priority Deadlines
Medium	 10	 1	 10%	 high	 budget	enfored	
High	 100	 50	 50%	 medium	 DL	guaranteed	
Low	 1000	 N/A	 100%	 low	 no	guarantee
Client1						P1							
Shared	Server	w.	Scheduling	Contexts	
Flying	autonomous	aircraft:	Mixed-criticality	support	in	seL4	|	LCA'18	28		|	
Server	
Running	
Running	
Server	runs	on	
client’s	scheduling	
context	
Client	is	
charged	for	
server’s	time	
Budget	expiry	
during	server	
execution?	
Client2						P2
Budget	Expiry	Options	
•  Multi-threaded	servers	(COMPOSITE	[Parmer	‘10])	
•  Model	allows	this	
•  Forcing	all	servers	to	be	thread-safe	is	policy	!	
	
•  Bandwidth	inheritance	with	“helping”	(Fiasco	[Steinberg	‘10])	
•  Ugly	dependency	chains	!	
•  Wrong	thread	charged	for	recovery	cost	!	
•  Use	timeout	exceptions	to	trigger	one	of	several	possible	actions:	
•  Provide	emergency	budget	
•  Cancel	operation	&	roll-back	server	
•  Change	criticality	
•  Implement	priority	inheritance	(if	you	must…)	
Flying	autonomous	aircraft:	Mixed-criticality	support	in	seL4	|	LCA'18	29		|	
Mechanism for
implementing other
models, e.g. earliest-
deadline first (EDF)
Cost	of	Isolation	
Operation	 Mainline	 MCS	 Overhead	
IPC	Call	(client)	 307	 307	 0%	
IPC	ReplyRecv	(server)	 320	 333	 4%	
IRQ	latency	 1597	 1776	 11%	
Signal	semaphore	 138	 144	 4%	
schedule	 878	 1048	 19%	
Flying	autonomous	aircraft:	Mixed-criticality	support	in	seL4	|	LCA'18	30		|	
Microbenchmark	latencies	in	cycles	on	1	GHZ	ARM	A9
Isolation	in	Action	
•  High-prio	CPU	hog,	budget	limited,	10ms	period	
•  Lower-prio	UDP	echo	server,	10ms	period	
Flying	autonomous	aircraft:	Mixed-criticality	support	in	seL4	|	LCA'18	31		|	
0
5
10
15
20
25
30
35
1 2 3 4 5 6 7 8 9 10
0
20
40
60
80
100
Latency(ms)
CPUutilisation(%)
Budget (ms)
Max
Mean
Budget
CPU %
Implementing	EDF	at	User	Level	
Flying	autonomous	aircraft:	Mixed-criticality	support	in	seL4	|	LCA'18	32		|	
•  EDF	scheduling	implemented	in	user-level	on	seL4	
•  Compared	against	kernel-level	EDF	scheduler	in	LITMUSRT	(Linux	testbed)	
0
0.5
1
1.5
2
2.5
3
1 2 3 4 5 6 7 8 9 10
Time(µs)
Number of threads
seL4 user-level LITMUS kernel
Critical	Systems:	DARPA	HACMS	
Flying	autonomous	aircraft:	Mixed-criticality	support	in	seL4	|	LCA'18	33		|	
Retrofit	
existing	
system!	
Retrofit	
existing	
system!	
Develop	
technology
Example:	SMACCMcopter	
HACMS	Research	UAV	
Flying	autonomous	aircraft:	Mixed-criticality	support	in	seL4	|	LCA'18	34		|	
Flight	Control	Board	
HW	
Sensors
ARM
M3
Radio
Motors
SW
Control
Monitor
Mission
Plan
Sensor
Filtering
eChronos RTOS
CAN
CAN	Bus	
trusted
untrusted
Mission	Board	
HW	
C&C Radio CameraARM A15
SW
Image
Processing
Command
& Control Linux VMCAN
USB
SMACCMcopter:	
Mission	Computer	Architecture	
Flying	autonomous	aircraft:	Mixed-criticality	support	in	seL4	|	LCA'18	35		|	
UART	
Rx	
UART	
Rdy	
UART	in	
200Hz	
UART	out	
200Hz	
Server	
200Hz	
CAN	
Rx	
CAN	
Tx	
UART	
Tx	
CAN	
200Hz	
Server	
Event-
triggered	
Task	
Periodic	
Task	
Critical	
Section	 CAN	
Rx	
CAN	
Tx	
CAN	
200Hz	
Gateway	
200Hz	
Linux	VM	
camera	
20Hz
New	Mixed-Criticality	Kernel	
•  Meets	requirements	of	MCS	
•  Performance	very	close	to	old	(non-isolation)	kernel	
•  Certifiable,	presently	undergoing	formal	verification	
•  Capabilities	for	reasoning	about	time	
•  Flexible	model,	fixed-prio	based	but	supports	user-level	EDF	implementation	
•  Usable	for	real-world	systems	
Flying	autonomous	aircraft:	Mixed-criticality	support	in	seL4	|	LCA'18	36		|
MCS	Features	are	Invasive	and	Some	
Details	Experimental	
Flying	autonomous	aircraft:	Mixed-criticality	support	in	seL4	|	LCA'18	37		|	
Master	
Developer	
branches	
Developer	
branches	
Developer	
branches	
MCS	branch	 Stage	branch	
Developer	
branches	
Verified	on	
specific	platforms	
Experiment	&	
evaluate	
Mature	MCS	
features	plus	all	
mainline	features	
Developer	
branches	
Developer	
branches
Thanks,	Trustworthy	Systems	Team!	
Flying	autonomous	aircraft:	Mixed-criticality	support	in	seL4	|	LCA'18	38		|	
Thank	you,	LCA	audience!

Flying Autonomous Aircraft: Mixed-Criticality Support in seL4