Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

OpenNebula out in the Open, Ander Astudillo, SURFsara

195 views

Published on

Where OpenNebula is typically used by system administrators to manage data centers, SURFsara’s HPC Cloud exposes OpenNebula’s web interface crudely to end users. SURFsara’s end users are researchers. They all have diverse needs and most of them are no system experts. It turns out that OpenNebula’s interface is easy enough to use, enabling researchers to build complex compute infrastructures for their analysis workflows. This presentation discusses the features our researchers and we appreciate in OpenNebula, it pictures a selection of succesful projects and it also explains caveats we have come across while keeping the service in the air.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

OpenNebula out in the Open, Ander Astudillo, SURFsara

  1. 1. — IaaS in the real world OpenNebula out in the Open Ander Astudillo <ander.astudillo@surfsara.nl> Ede (The Netherlands), 19th September 2017 OpenNebula Tech Day
  2. 2. Ander Astudillo <ander.astudillo@surfsara.nl> Science Park, Amsterdam The SURF cooperative
  3. 3. What do we (SURFsara) want to offer? 3 Services for scientists …scientists systems gurus … complex users’ problems • Data: big, dirty, non-structured… • Computation: complex (e.g.: modeling, simulation) • Libraries nightmare • 3rd party, incompatibility, maintenance… … flexibility … trial and error … test … cooperate … share … show … scratch … privacy
  4. 4. Where does the HPC Cloud fit? 4
  5. 5. Agenda 1.- SURFsara & HPC Cloud 2.- The real world Appendix.- SURFsara’s implementation
  6. 6. HPC Cloud IaaS ≡ Your craft 61.- SURFsara & HPC Cloud Your application may need more… Scale up Scale outvs.
  7. 7. The prelude 71.- SURFsara & HPC Cloud Claudia: Exploring the concept • Couple of borrowed nodes • 16 cores • 1Gbps network • Local storage • Self-built UI • vs. OpenNebula 1.x !
  8. 8. Calligo 81.- SURFsara & HPC Cloud HPC • Big nodes (19×) • 32 cores, 256GB RAM • Fast interconnect • Low-latency 40 Gbps • NFS storage • Web interface • Groups Cloud • Multi-purpose versatility • Shape elasticity • Self-service on-demand • Project-based • Private network Service ! 3.x
  9. 9. The real world
  10. 10. Rounding the system off 102.- The real world • Documentation • System load • Support • Accounting Cloud light • Smaller nodes • CPU overcommit • (Interactive) services …scientists sysadmins • Image upload • Network filters • DHCP • Dynamic DNS • Wizard tweaks
  11. 11. Storage 112.- The real world • Fast I/O • More nodes Local SSD VirDir • Per-project NFS share • Image upload & download • Backups MySQL aaS Archive Data Ingest (DIS)
  12. 12. Incidents & eventualities 122.- The real world Datacenter power outage • Storage collapsed • Helped discover power redundancy was faulty Enthusiastic users • Component down • Uncovered lenient quotas ‘s glory Delete images • Deleted users’ data • Uncovered bug VNC ports • Ran out of ports • Uncovered limitation Too many tweaks • Could not upgrade • Uncovered our optimism
  13. 13. The present 132.- The real world • Latest release of OpenNebula (v.5.2) • Accounting • Ceph storage; expansion • Distributed object store and file system • Cope with increasing load • GPU processing • Highly parallel structure • Program specifically to use it Oort: Consolidating the concept
  14. 14. 14 The present… eventualities 2.- The real world SSD disks • Consumer grade • Helped sharpen our tenders ‘s upgrade glory Inter-version UI changes • Users’ VM’s logs • Terminate vs. Delete • ACL! Resources bookkeeping • Double VNC ports assignment • Double GPU’s assignment Datacenter move • Remember cables! • Cloud-init?
  15. 15. User applications 152.- The real world • Flexible software mix • Big VMs • Elasticity • Provide their own service to their own users • Software that requires licenses • Set up, test and deploy workflows • Deliver training; courses • Intensive computing • …from diverse fields: • Biology • Genetics • Informatics • Chemistry • Ecology • Linguistics • Robotics • Business • Social sciences • Engineering • Humanities • Water management • … Users like& leverage… ‘s glory
  16. 16. 162.- The real world
  17. 17. SURFsara’s HPC Cloud implementation ppendix
  18. 18. Hardware 183.- Implementation
  19. 19. Ander Astudillo <ander.astudillo@surfsara.nl> Credits Images: Wikipedia, Science Park, RRZE icons, NIST, nVidia, Ceph, Pixabay Slides: SURFsara colleagues

×