The document discusses high performance computing (HPC) demands and strategies at BMW. It notes that BMW's HPC usage and simulation models are increasing in complexity, driving the need for more computing power. BMW has established an HPC cluster to meet these demands, and is taking steps to improve efficiency and cost-effectiveness through HPC co-location and exploring public and private cloud options. This includes relocating some workloads to a data center in Iceland and testing moving an elementary HPC cell to a cloud platform.

1. HIGH PERFORMANCE COMPUTING @ BMW
Wolfgang Burke, BMW Group, 22.09.2014

2. AGENDA.
1
BMW Group – The company
2
HPC demands
3
HPC Cluster @ BMW
4
HPC Colocation
5
HPC Cloud
Page 2
High Performance Computing @ BMW, 22.9.2014

3. High Performance Computing @ BMW, 22.9.2014
Page 3
BMW GROUP – THE COMPANY.
KEY FIGURES.
The BMW Group is one of the ten largest industrial enterprises in Germany.
Brands: BMW, MINI, Rolls-Royce, BMW Motorrad.
Sales BM W Group 2013: 76,06 bn. Euro.
Deliveries of 1.963.798 cars in 2013.
Deliveries of 115.215 motorcycles in 2013.
29 manufacturing and assembly plants in 14 countries.
43 sales companies worldwide.
Presence in more than 140 countries.
110.351 employees at 31.12.2013.

4. Page 4
Moses Lake
Spartanburg
Shenyang
Chennai Rayong
Kulim
Jakarta
Araquari Rosslyn
Production Plants Manaus
Assembly Plants
Engines / Components
BMW Group
Joint Venture
Contract Production
Regensburg
Dingolfing
Berlin
Eisenach Leipzig
Wackersdorf
Landshut
Munich
Kaliningrad
Born
(2014)
Steyr
Graz
Hams Hall
Oxford
Swindon
Goodwood
San Luis Potosí (2019)
State 07/2014
Cairo
BMW GROUP PRODUCTION NETWORK WORLDWIDE.
29 LOCATIONS IN 14 COUNTRIES.

5. High Performance Computing @ BMW, 22.9.2014
Page 5
BMW GROUP – THE COMPANY.
POSITIVE IMPULSES BY NEW MODELS.
Mini Hatch 5 doors
BMW i8
BMW X6
BMW 2 Series Active Tourer
BMW X4
BMW M4 Cabrio

6. AGENDA.
1
BMW Group – The company
2
HPC demands
3
HPC Cluster @ BMW
4
HPC Colocation
5
HPC Cloud
Page 6
High Performance Computing @ BMW, 22.9.2014

7. High Performance Computing @ BMW, 22.9.2014
Page 7
HPC DEMANDS
CHALLENGES BY INCREASING HPC USAGE
2007-07
2008-07
2009-07
2010-07
2011-07
2012-07
2013-07
2014-07
Core hours
Computation time demands per month

8. High Performance Computing @ BMW, 22.9.2014
Page 8
HPC DEMANDS
INCREASING NUMBER OF APPLICATION AREAS USING SIMULATION
Aerodynamics
Stiffness, structural dynamics
Passive Safety
Operating strength
Powertrain
Production

9. High Performance Computing @ BMW, 22.9.2014
Page 9
HPC APPLICATIONS
AERODYNAMIC SIMULATION

10. High Performance Computing @ BMW, 22.9.2014
Page 10
HPC APPLICATIONS
FUEL INJECTION SIMULATION.

11. High Performance Computing @ BMW, 22.9.2014
Page 11
HPC APPLICATIONS. SIMULATION OF PRODUCTION PROCESSES.
Forming limit diagram of an experimental tool
Simulation of the deep drawing process

12. High Performance Computing @ BMW, 22.9.2014
Page 12
HPC DEMANDS
SIMULATION MODELS GETTING MORE DETAILED

13. High Performance Computing @ BMW, 22.9.2014
Page 13
APPLICATIONS ON HPC. PREDICTIVENESS IN MATERIAL BEHAVIOR.

14. High Performance Computing @ BMW, 22.9.2014
Page 14
HPC DEMANDS. SIMULATION REDUCES PROTOTYPES IN THE DESIGN PROCESS.
design stage 2
design stage 1
industrialisation
1
2
3
Reduced prototype driven design process (schematically):
“close to production” cars
design stage 2
design stage 1
industrialisation
1
2
3
Old prototype driven design process (schematically):
prototypes “design evaluation”
prototypes “design validation”
“close to production” cars
prototypes “design validation” (only in case of high innovations)
Benefits of the new process:
Straightforward design process.
Reduced number of prototypes and cost savings.

15. High Performance Computing @ BMW, 22.9.2014
Page 15
HPC DEMANDS
CRASH SIMULATION: FRONTAL IMPACT
Experimental model for method development

16. High Performance Computing @ BMW, 22.9.2014
Page 16
HPC DEMANDS. NUMBER OF LEGAL REQUIREMENTS INCREASE.
Heute
1990
1997
2005
FMVSS 208 old
FMVSS 208 new
Phase I
FMVSS 208 new
Phase II
FMVSS 214 static
FMVSS 214 static + dynamic
FMVSS 214 new
MDB + Pole
FMVSS 301 old rear crash
FMVSS 301 new Offset rear crash
FMVSS 216 roof crush test
FMVSS 216 new
Occupant Containm.
ECE R95 side crash
ECE R95 AE-MDB
Advanced European
Movable Barrier
2003/102/EC
PDB or TRL Barrier
NHTSA Front to Front
BMW model
USA
ECE
USA
ECE
USA
ECE
USA
USA
ECE
Frontal impact
Side impact
Rear impact
Fuel tightness
Rollover
Pedestrian safety
Compatibility
ECE R12 steering column displacement
ECE R94 Offset frontal crash
2011

17. High Performance Computing @ BMW, 22.9.2014
Page 17
BMW product portfolio (status June 2014)
 Increasing number of models, variants, facelifts
 Diversity per model (e.g. left/right hand drive, motorization)
BMW market introduction 2014
HPC DEMANDS
INCREASING NUMBER OF MODELS AND VARIANTS

18. AGENDA.
1
BMW Group – The company
2
HPC demands
3
HPC Cluster @ BMW
4
HPC Colocation
5
HPC Cloud
Page 18
High Performance Computing @ BMW, 22.9.2014

19. High Performance Computing @ BMW, 22.9.2014
Page 19
HPC CLUSTER @ BMW
HPC SERVICE WITH AN EXTENSIBLE RESOURCE POOL.
Shared, extensible resource pool
Optimized computational architecture & technologies
Transparent access
Jobflow & workload management control
Overall operations framework (e.g. installation, administration, monitoring)

20. High Performance Computing @ BMW, 22.9.2014
Page 20
HPC CLUSTER @ BMW
HPC SERVICE WITH AN EXTENSIBLE RESOURCE POOL.
Workload-Management
HPC Cluster
Engineers, Virtual Dev.

21. High Performance Computing @ BMW, 22.9.2014
Page 21
HPC CLUSTER @ BMW
IMPACT OF HIGHER VIRTUAL SIMULATION USAGE
2007
2008
2009
2010
2011
2012
2013
2014
2015
2016
Floor space of HPC installation
Goals of the BMW HPC decisions
enable a scalable model for data center space requirements
deal with the increasing electricity prices
address carbon footprint caused by higher energy consumption
2008
2010
2012
2014
2016
2018
2020
2022
2024
Electricity price
2007-07
2008-07
2009-07
2010-07
2011-07
2012-07
2013-07
2014-07
Core-h per month

22. AGENDA.
1
BMW Group – The company
2
HPC demands
3
HPC Cluster @ BMW
4
HPC Colocation
5
HPC Cloud
Page 22
High Performance Computing @ BMW, 22.9.2014

23. High Performance Computing @ BMW, 22.9.2014
Page 23
HPC CLUSTER @ BMW
HPC CLUSTER DEVELOPMENT STEPS.
HPC private cloud
•On-demand resource reservation
•Transparent access (self-service portal)
•Standardized applications (services)
•Fully automated
•Zero downtime approach
•Cluster capacity scalability
•Maximum cluster utilization
•Cluster technological adaptability
•Cluster vendor independence
•Cluster best price purchasing

24. High Performance Computing @ BMW, 22.9.2014
Page 24
HPC CLUSTER @ BMW
HPC CLUSTER DEVELOPMENT STEPS.
HPC public cloud
•Flexible capacity scalability
•Best price purchasing
•“HPC as a service”
Challenges:
•Remote location
•Unclear cloud model (IaaS, PaaS, SaaS, …)
•Cloud integration
•Data security
•Organizational aspects
•…
HPC private cloud
•On-demand resource reservation
•Transparent access (self-service portal)
•Standardized applications (services)
•Fully automated
•Zero downtime approach
•Cluster capacity scalability
•Maximum cluster utilization
•Cluster technological adaptability
•Cluster vendor independence
•Cluster best price purchasing

25. High Performance Computing @ BMW, 22.9.2014
Page 25
HPC CLUSTER @ BMW
HPC CLUSTER DEVELOPMENT STEPS.
HPC public cloud
•Flexible capacity scalability
•Best price purchasing
•“HPC as a service”
Challenges:
•Remote location
•Unclear cloud model (IaaS, PaaS, SaaS, …)
•Cloud integration
•Data security
•Organizational aspects
•…
HPC Co-Location
•Space capacity scalability
•DC technology adaptability
•DC best price purchasing
Challenges:
•Remote location
•Data security
•Organizational aspects
HPC private cloud
•On-demand resource reservation
•Transparent access (self-service portal)
•Standardized applications (services)
•Fully automated
•Zero downtime approach
•Cluster capacity scalability
•Maximum cluster utilization
•Cluster technological adaptability
•Cluster vendor independence
•Cluster best price purchasing

26. High Performance Computing @ BMW, 22.9.2014
Page 26
HPC COLOCATION
ICELAND FOR HPC COLOCATION
Data center operation with 100% CO2 emission free, renewable energy (geothermal and hydro) Free air cooling nearly the whole year Fixed low power cost for up to 20 years High scalability in data center floor space High bandwidth network connections available Power Usage Effectiveness (PUE) < 1,2
1
1,2
1,43
1,8
0
0,5
1
1,5
2
Average PUE
BMW Data Center PUE
Iceland Colocation PUE
Best physical reachable PUE

27. High Performance Computing @ BMW, 22.9.2014
Page 27
HPC COLOCATION
RELOCATION OF ELEMENTARY CELLS TO ICELAND.
Workload-Management
HPC Cluster
Engineers, Virtual Dev.

28. High Performance Computing @ BMW, 22.9.2014
Page 28
HPC COLOCATION
RELOCATION OF ELEMENTARY CELLS TO ICELAND.
Workload-Management
HPC Cluster
Engineers, Virtual Dev.

29. High Performance Computing @ BMW, 22.9.2014
Page 29
HPC COLOCATION
CHALLENGE LATENCY
Batch jobs: Only one big bulk (result) data transfer
Optimization of TCP stack parameters
 Communication performance for bulk data transfer sufficient
Distance Munich - Keflavik: 2850 km
Latency BMW local 0,3 ms roundtrip
Latency BMW – Iceland 53 ms roundtrip
Transfer-Server
Cluster
Workload
Management
Storage system
Job submit
Result (bulk) data transfer
WAN connection
WAN connection

30. High Performance Computing @ BMW, 22.9.2014 Page 30
HPC COLOCATION
TRANSFER TIMES AND OVERALL RUNTIMES FOR CRASH JOBS
Transfer time
EZ IS
0,73%
EZ IS
0,86%
EZ MUC EZ MUC
0,41% 0,41%
EZ MUC
0,41%
Problem size of crash job 100% 98% 97% 97% 110%
HPC stages
local stages

31. High Performance Computing @ BMW, 22.9.2014
Page 31
HPC COLOCATION
INCREMENTAL EXTENSION OF HPC APPLICATIONS IN ICELAND
including Crash
HPC load share
successful
not successful
open
not tested
November 2012 Initial application tests in Iceland
HPC load share
released
not release
not successful
open
not tested
February 2013
Start of productive calculations in Iceland
HPC load share
released
not successful
open
Current situation
Only a few applications open or not released
Crash

32. High Performance Computing @ BMW, 22.9.2014
Page 32
HPC CLOUD COLOCATION
ODCA CARBON FOOTPRINT AND ENERGY EFFICIENCY USAGE MODEL
Application of the ODCA Carbon Footprint and Energy Efficiency Usage Model
Execution of an ODCA Carbon Footprint and Energy Efficiency Usage Model Proof of Concept

33. High Performance Computing @ BMW, 22.9.2014
Page 33
1148
673
2,1
0
200
400
600
800
1000
1200
1400
Coal Fired Grid
Natural Gas Grid
Iceland Data Center
Carbon Usage Effectiveness (CUE)
HPC COLOCATION
DATA CENTER ICELAND: CARBON POC
g/kWh

34. AGENDA.
1
BMW Group – The company
2
HPC demands
3
HPC Cluster @ BMW
4
HPC Colocation
5
HPC Cloud
Page 34
High Performance Computing @ BMW, 22.9.2014

35. High Performance Computing @ BMW, 22.9.2014
Page 35
HPC CLUSTER @ BMW
HPC CLUSTER DEVELOPMENT STEPS.
HPC public cloud
•Flexible capacity scalability
•Best price purchasing
•“HPC as a service”
Challenges:
•Remote location
•Unclear cloud model (IaaS, PaaS, SaaS, …)
•Cloud integration
•Data security
•Organizational aspects
•…
HPC Co-Location
•Space capacity scalability
•DC technology adaptability
•DC best price purchasing
Challenges:
•Remote location
•Data security
•Organizational aspects
HPC private cloud
•On-demand resource reservation
•Transparent access (self-service portal)
•Standardized applications (services)
•Fully automated
•Zero downtime approach
•Cluster capacity scalability
•Maximum cluster utilization
•Cluster technological adaptability
•Cluster vendor independence
•Cluster best price purchasing

36. High Performance Computing @ BMW, 22.9.2014
Page 36
HPC CLOUD
MOVE ELEMENTARY CELLS IN A HPC CLOUD.
Workload-Management
HPC Cluster
Engineers,
Virtual Dev.
HPC colocation provider
HPC on premise

37. High Performance Computing @ BMW, 22.9.2014
Page 37
HPC CLOUD
MOVE ELEMENTARY CELLS IN A HPC CLOUD.
Workload-Management
HPC Cluster
Engineers, Virtual Dev.
Stepwise approach
Reference performance benchmarks
HPC colocation provider
HPC cloud provider
HPC on premise

38. Cloud platform
High Performance Computing @ BMW, 22.9.2014
Page 38
HPC CLOUD
MOVE ELEMENTARY CELLS IN A HPC CLOUD.
HW provisioning performed by cloud platform
Servers
File service (scratch)
High speed network (Infiniband)
•Gateway-Nodes
•~132 Compute- Nodes:
o Bignodes
o Stdnodes
•High Speed Cluster Interconnect
•Admin Interconnect
EZ-Scratch
Building Block: Elementarzelle
WAN connection
HW administration provided by cloud platform
On-demand provisioning of servers in a cell

39. Cloud platform
High Performance Computing @ BMW, 22.9.2014
Page 39
HPC CLOUD
MOVE ELEMENTARY CELLS IN A HPC CLOUD.
•Gateway-Nodes
•~132 Compute- Nodes:
o Bignodes
o Stdnodes
•High Speed Cluster Interconnect
•Admin Interconnect
EZ-Scratch
Building Block: Elementarzelle
WAN connection
1. Step: Bare metal provisioning
HW server
OS provisioning
HPC provisioning
cloud service operation
Reference for performance benchmarks

40. Cloud platform
High Performance Computing @ BMW, 22.9.2014
Page 40
HPC CLOUD
MOVE ELEMENTARY CELLS IN A HPC CLOUD.
•Gateway-Nodes
•~132 Compute- Nodes:
o Bignodes
o Stdnodes
•High Speed Cluster Interconnect
•Admin Interconnect
EZ-Scratch
Building Block: Elementarzelle
WAN connection
1. Step: Bare metal provisioning
HW server
OS provisioning
HPC provisioning
cloud service operation
2. Step: Virtual server provisioning
cloud service operation
HPC provisioning
Virtual server
provisioning
Reference for performance benchmarks

41. High Performance Computing @ BMW, 22.9.2014
Page 41
HPC CLOUD
MOVE ELEMENTARY CELLS IN A HPC CLOUD.
Seamless integration of HPC cloud into BMW HPC cluster
First test results on bare metal show comparable performance to current cluster
Basis for a HPC service specification
Open questions to be clarified

42. High Performance Computing @ BMW, 22.9.2014
Page 42
HPC CLOUD
DEVELOPMENT OF A HPC CLOUD MODEL.
ODCA usage models might be helpful on different layers
•Data Security
Security Workgroup
•Long Distance Migration
Manageability & Services Workgroup
•ODCA Interop Across Clouds Guide
•Compute Infrastructure as a Service
Infrastructure Workingroup

43. Page 43
THANK YOU VERY MUCH FOR YOUR ATTENTION
Imprint: Published by: BMW Group Corporate and Governmental Affairs 80788 Munich No reproduction, in whole or in part, without the express permission of Bayerische Motoren Werke Aktiengesellschaft, München ©2014 Bayerische Motoren Werke Aktiengesellschaft