OpenNebula TechDay Boston 2015 - HA HPC with OpenNebula
•
1 like•1,018 views
This document discusses high performance computing (HPC) and its uses. It provides examples of how HPC is used for physics simulations like lattice quantum chromodynamics and supernovae, planetary science like hurricane modeling, life sciences like molecular dynamics, engineering applications, machine learning, and big data. It then describes Microway's test drive HPC cluster that uses OpenNebula for infrastructure management across CPU and GPU nodes with InfiniBand networking. Virtualizing the cluster provides flexibility for administrators and users while incurring minimal performance penalties.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (10)
Similar to OpenNebula TechDay Boston 2015 - HA HPC with OpenNebula
Similar to OpenNebula TechDay Boston 2015 - HA HPC with OpenNebula (20)
More from OpenNebula Project
More from OpenNebula Project (20)
Recently uploaded
Recently uploaded (20)
OpenNebula TechDay Boston 2015 - HA HPC with OpenNebula
- 1. HA HPC with OpenNebula Eliot Eshelman - Microway 2015-06-29
- 2. HPC - what is it good for? Lattice Quantum ChromoDynamics (QCD) RBC/UKQCD collaboration; Research Team: Dirk Broemmel, Thomas Rae, Ben Samways, Investigators: Jonathan Flynn Physics
- 3. HPC - what is it good for? Tech-X VORPAL for the DOE and NNSA Physics
- 4. HPC - what is it good for? Astrophysics Simulation of a supernova Courtesy of Oak Ridge National Laboratory, U.S. Dept. of Energy
- 5. HPC - what is it good for? Planetary Science WRF 0.5km simulation of Hurricane Sandy NCAR CISL VAPOR visualizations
- 6. HPC - what is it good for? Life Science NAMD & GROMACS; visualized with VMD
- 7. HPC - what is it good for? https://www.nersc.gov/assets/Trinity--NERSC-8-RFP/ Documents/NERSCWorkloadAnalysisFeb2013.pdf All Science!
- 8. HPC - what is it good for? ALTAIR AcuSolve Engineering: FEA CFD Multi-Physics
- 9. HPC - what is it good for? Machine Learning NVIDIA DIGITS with Caffe from UC Berkeley
- 10. HPC - what is it good for? Big Data
- 11. First: a discussion of scale What types of HPC systems do we design? ● up to ~512 nodes ● budgets of $50K to $3M Most leadership-class HPC sites use similar designs, but source from the big vendors.
- 12. 10,000-foot view of an HPC cluster
- 13. HPC clusters ready to ship
- 14. Microway's Test Drive cluster ● Owned and maintained by Microway ● Used by customers for benchmarking ● Used by employees for testing, replicating customer issues & software development ● Not actually mission-critical, but designed to emulate those that are...
- 15. The Hardware ● (3) OpenNebula hosts ● (4) Parallel storage servers ● (6) Bare-metal CPU + GPU compute nodes ● Gigabit Ethernet ● 56Gbps FDR InfiniBand
- 19. Compute Nodes ● Remaining bare metal for now ○ Virtualizing GPUs has caveats ● Virtualizing the nodes does give a lot more flexibility to the admins and the users ○ HPC users have very specific software needs ○ VMs can enable reproducibility ○ Some sites are trying out containers (Docker)
- 20. End Goal ● Each employee/customer can be assigned their own private HPC cluster ● Multiple cluster instances for: ○ Development ○ QA ○ Production
- 21. What we gain Flexibility: ● Easy backups ● Easy restores ● Easy upgrades ● Easy rollbacks ● Faster software development Customer sees: ● Better uptime ● Quicker upgrades ● Fewer bugs ● Better performance
- 22. What we lose Not much! ● a little bit of performance (~1% on CPU; up to 10% on I/O) ● no more direct access to InfiniBand (HPC folks like having access to bare metal)
- 23. Other tools to investigate...
- 24. What's next? ● Got a project in mind? ● Inspired to speak at our next meetup? Get in touch! eliot@microway.com