Performing simulation based, real time
decision making with cloud HPC
Zack Smocha, April 2016
7	 Rescale	confiden-al	–	please	do	not	distribute	under	any	circumstances
Rescale	confiden-al	–	please	do	not	distribute	under	any	circumstances	2	
Agenda
•  Rescale overview
•  Evolution of simulation
•  Simulation in service
•  F1 simulations
•  Manor Racing case study
Rescale	confiden-al	–	please	do	not	distribute	under	any	circumstances	3	
HQ San Francisco, USA , Japan office
rapid growth
Global simulation cloud HPC platform
30+ data centers, 120 simulation software
Over 100 leading enterprises -
automotive, aerospace, energy and life
sciences
Headquarters
Technology
Customers
Investors
Rescale - Company Overview
Peter ThielJeff Bezos Richard Branson
... and several other
industry leaders,
technology experts, and
experienced executives
Rescale	confiden-al	–	please	do	not	distribute	under	any	circumstances	4	7	 Rescale	confiden-al	–	please	do	not	distribute	under	any	circumstances	4	
Rescale Cloud HPC Enterprise Simulation Platform
So#ware	
120+ software
packages
Mul,-clouds	
30 varied location
and HW availability
Workflow	
Administra,on	
Security	
Compliant, data
and user
Manage usage
access and cost
Experienced team
seamless workflow
5	
Simulations in Industry
Aerospace
Automotive
LifeSciences
Oil&Gas
Industrials
Semiconductor
•  Complex	turbine	
•  Wing	designs		
•  Modelling	
propulsion	
•  Crash	simula-on	
•  Engine	
computa-onal	
fluid	dynamics	
•  Reservoir	
simula-on	
workflows	
•  Hydrocarbon	
traps	
•  Gene-c	
engineering		
•  Isola-on	of	
gene-c	traits	
Rescale	confiden-al	–	please	do	not	distribute	under	any	circumstances
Rescale	confiden-al	–	please	do	not	distribute	under	any	circumstances	6	
Evolution of Automotive SimulationComplexity
1960 1970 1980 1990 2000 2010 Today
Vehicle
Dynamics
Crash Analysis
FEA
Multiphysics
High-fidelity
Ensemble analysis
CFD & HPC
Rescale	confiden-al	–	please	do	not	distribute	under	any	circumstances	7	
Simulation in the Product Life Cycle
Predict behavior
without actually
testing it in real life
 
Validate and optimize
the design of parts
and manufacturing
Using real data to
help make real time
decision
Engineering Design Manufacturing In Service/Production
Rescale	confiden-al	–	please	do	not	distribute	under	any	circumstances	8	7	 Rescale	confiden-al	–	please	do	not	distribute	under	any	circumstances	8	
Simulation In Service - Time Spectrum
Real	-me	“next	
move”	gaming	
analysis	
Threads	 CPU	 GPU	 ELO	
24 764 112 3,079
40 1,202 176 3,140
64 1,920 280 3,168
Maintenance	
and	abnormal	
behavior	
Using	real	-me	
track	side	data	
for	race	strategy	
Make	sure	you	
don’t	crash	the	
bus	
Days/hours		 Minutes	 Micro	seconds	Seconds
Rescale	confiden-al	–	please	do	not	distribute	under	any	circumstances	9	
F1 the Art of Race Strategy
Rescale	confiden-al	–	please	do	not	distribute	under	any	circumstances	10	
F1 Results Australia - Many Strategies
Superso#	 So#	 Medium	Hard	 Wet	 Intermedium	 Used	=	Pit	Stop
•  Based	in	Banbury	UK	
•  Partners	with	Mercedes-Benz	engine	technology		
•  Williams	Advanced	Engineering	for	transmissions		
Rescale	confiden-al	–	please	do	not	distribute	under	any	circumstances	11	
Manor Racing
Rescale	confiden-al	–	please	do	not	distribute	under	any	circumstances	12	
Manor use case - Goals
•  Best	-me	to	take	a	pit	stop		
•  What	-res	to	fit	for	the	next	stage	of	the	race.				
•  Second	guess	the	compe--on	to	try	and	gain	race	
posi-on	through	be^er	pit	stop	-me	
For Manor Racing it is about meticulous attention to detail, eking out every
single opportunity to find every single gap. Car and driver, factory and team
Rescale	confiden-al	–	please	do	not	distribute	under	any	circumstances	13	
Manor use case - Users
Dave	Ryan	-	Racing	Director	
	
	
	
	
James	Knapton	-	Head	of	
Vehicle	Science	
	
Strategy	engineers	who	advise	the	race	engineers	
on	the	op-mum	strategy	as	the	race	is	developing
Rescale	confiden-al	–	please	do	not	distribute	under	any	circumstances	14	
HPC Simulation in Service - Requirements
•  Collect the data in real time?
•  Insert the data into the system?
•  Upload the data to the cloud HPC?
•  Best HW for fast simulation?
•  Download and access the data?
Rescale	confiden-al	–	please	do	not	distribute	under	any	circumstances	15	
Manor Cloud HPC Architecture
Cloud HPC Cluster
Head Node
HPC Scheduler
Compute Nodes
Nodes are joined to the
HPC Scheduler
Virtual Network LAN
IPSec
VPN
Manor application GUI
•  For optimization jobs directly interact with the HPC cluster
•  Clients running jobs join the head node domain and mount the shared file system
Rescale	confiden-al	–	please	do	not	distribute	under	any	circumstances	16	
Input Parameters and Live Data
•  Parameters: lap time, tire
degradation rate for each tire
compound, expected car
performance as fuel level
reduces
Make a live decision
based on the
simulation results and
enter actual track
side results
Collect live track side
data and run the
simulation
Make a live decision
based on the
simulation results and
enter actual track
side results
Collect live track side
data and run the
simulation
•  Example of live Input data:
Actual lap time, tire degradation
Rescale	confiden-al	–	please	do	not	distribute	under	any	circumstances	17	
Input Parameters
•  How do I collect the data input in real time
– Data is available from the track side
•  Insert the data to the system
– User enters the data into the Manor
application interface, application generates
input size files ~500kB
•  Upload the data
– Data is uploaded to the head cluster node
from the user laptop
Rescale	confiden-al	–	please	do	not	distribute	under	any	circumstances	18	
Simulation Benchmark - Best HW for Fast Simulation
•  Simulations based on Monte-Carlo methods
•  Response < 45-50 sec
•  Run thousands of race simulations per minute, repeat
this process over and over throughout the race
	#cars	 #cores	 Strategies	 Permuta-ons	 Itera-ons	 Running	-me	on	the	cluster	
1	Car	 500	 30	 100	 100	 32.27	
1	Car	 500	 30	 300	 20	 69.58	
1	Car	 500	 30	 150	 20	 31.69	
1	Car	 500	 90	 20	 20	 31.82	
1	Car	 500	 90	 20	 100	 35.61	
1	Car	 750	 30	 100	 100	 31.80	
1	Car	 1500	 30	 100	 100	 30.75	
2	Cars	 750	(each)	 30	 100	 100	 35.03
Rescale	confiden-al	–	please	do	not	distribute	under	any	circumstances	19	
Running the Simulation
•  Clusters are running
the whole race
•  3000 tasks
•  Hardware:
•  1500 cores
•  16 CPU per node
•  98 nodes
•  Haswell CPU
Rescale	confiden-al	–	please	do	not	distribute	under	any	circumstances	20	
The Results
•  Output: Optimum race time
•  Results size is 5MB
•  Results are download to the user PC
•  User views results in the Manor GUI App
•  Using the results in practice: Decide when would be the
best time for a pit stop
Rescale	confiden-al	–	please	do	not	distribute	under	any	circumstances	21	
The Results
•  Each curve is a different tire choices
•  Each # represent a pit stop and the lap to stop
•  POA: Prime/Option/Alternate : Hardest to the softest
Op,mum	race	,me	–	Sensi,vity	to	,re	strategy
Rescale	confiden-al	–	please	do	not	distribute	under	any	circumstances	
Simulation defines our future,
join us in helping build a better world.
Rescale	confiden-al	–	please	do	not	distribute	under	any	circumstances	23	
Simulations in F1
•  Wind	tunnel,	aerodynamic	
•  CFD	and	FEA	
•  2014	FIA	regula-ons
Rescale	confiden-al	–	please	do	not	distribute	under	any	circumstances	24	
F1 the Art of Tire Change
•  15-19	people	
•  Stop	below	3	sec	
•  Do	other	adjustments
Appendix		
Rescale	confiden-al	–	please	do	not	distribute	under	any	circumstances	25

Performing Simulation-Based, Real-time Decision Making with Cloud HPC