INTRODUCTION TO HPC CLOUD COMPUTING
SURF Research Bootcamp
Nuno Ferreira, Ander Astudillo / 2018.04.10
Agenda: Compute Track
10:15 - 11:30 Introduction to UNIX
12:30 - 14:30 Introduction to Cluster Computing
15:00 - 17:00 Introduction to HPC Cloud Computing
15:00 Presentation: HPC Cloud Intro
15:30 Hands-on
Access until the 17th April
SURFsara services
Why different services?
Supercomputer Hadoop Grid
HPC Cloud
What is Cloud Computing?
Essential Characteristics
• On-demand self-service
• Network access
• Resource pooling
• Elasticity
• Measured service
Service Models
• Software as a Service
• Platform as a Service
• Infrastructure as a Service
“Ask 10 people what the cloud is, get 11 answers.” [?]
“Cloud computing is a model for enabling ubiquitous, convenient, on-
demand network access to a shared pool of configurable computing
resources (e.g., networks, servers, storage, applications, and services)
that can be rapidly provisioned and released with minimal
management effort or service provider interaction.”
“Say Cloud one more time …”
Pizza as a Service
Traditional
On-Premise
Dining Table
Drinks
Electric / Gas
Oven
Fire
Pizza Dough
Tomato Sauce
Toppings
Cheese
Home Made
Software as a
Service
Dining Table
Drinks
Electric / Gas
Oven
Fire
Pizza Dough
Tomato Sauce
Toppings
Cheese
Restaurant
Platform as a
Service
Dining Table
Drinks
Electric / Gas
Oven
Fire
Pizza Dough
Tomato Sauce
Toppings
Cheese
Pizza Delivery
Infrastructure
as a Service
Dining Table
Drinks
Electric / Gas
Oven
Fire
Pizza Dough
Tomato Sauce
Toppings
Cheese
Take & Bake
YouManageVendorManages
A. Barron, IBM
Cloud Service Models
Traditional
On-Premise
Applications
Data
Runtime
Middleware
OS
Virtualization
Servers
Storage
Networking
Software as a
Service
Applications
Data
Runtime
Middleware
OS
Virtualization
Servers
Storage
Networking
Platform as a
Service
Applications
Data
Runtime
Middleware
OS
Virtualization
Servers
Storage
Networking
Infrastructure
as a Service
Applications
Data
Runtime
Middleware
OS
Virtualization
Servers
Storage
Networking
YouManageVendorManages
Infrastructure
as a Service
Applications
Data
Runtime
Middleware
OS
Virtualization
Servers
Storage
Networking
Infrastructure as a Service
YouManageVendorManages
The Scientific Method
Prediction
Experiment
Results
Refine Model
Hypothesis
“Failure is a data point!”
HPC Cloud Projects
Research fields
• Biology
• Genetics
• Informatics
• Chemistry
• Ecology
• Linguistics
• Robotics
• Business
• Social sciences
• Engineering
• Humanities
• …
Use cases
• Flexible software mix
• Big VMs
• Elasticity
• Provide a service to peers
• Software requiring licenses
• Set up, test and deploy workflows
• Training courses
• Intensive computing
• …
HPC Cloud Benefits
General benefits
• Data & Computing in Dutch soil
• Data privacy inside your VM
• Unrestricted Internet access
• Collaborative work
Technical benefits
• No overcommitting
• Tailor made your VM to your needs (flexibility)
• Root access!
• Controlled environment : choose your OS & packages
• Fast private network between VM’s
• No maximum wall time!
HPC Cloud Shortcomings
• No Service Level Agreement
• No 24/7 Helpdesk service support
• You maintain everything in your VM
• You are responsible for all of your VM’s behavior
• You must protect yourself against threats from the Internet
• Accounting on VM uptime, not just compute time (like gas, light)
• No automatic backups
• Your laptop is faster than a 1 core VM
?
How to obtain an HPC Cloud account?
Resource
Request 1
Technical
Evaluation
Allocation
Resources
Access
Granted
Time : hours to a few days
1 Resource Request form
2 helpdesk@surfsara.nl
Helpdesk 2
HPC Cloud Resources
Compute Nodes
Intel(R) Xeon(R) CPU E5-2698 v3 @ 2.30GHz 21 64 256GB 3.2TB
Intel(R) Xeon(R) CPU E5-2698 v4 @ 2.20GHz 10 80 512GB 3.5TB
Intel(R) Xeon(R) Gold 6148 CPU @ 2.40GHz 7 80 576GB 3.0TB
Intel(R) Xeon(R) CPU E5-2640 v3 @ 2.60GHz 11 32 256GB 800GB 4 (GRID K2)
Intel(R) Xeon(R) CPU E5-2609 v4 @ 1.70GHz 2 16 128GB 800GB 4 (Tesla P100)
Storage Nodes
• 900 TB Ceph net * 3 redundancy : 2.7 PB total
Network
• 10 Gbit
#
Top 10 beautiful data centers, Datacenter Dynamics, 2017.06.05
“GPU” icon by Misha Petrishchev from the Noun Project
“SSD” icon by mikicon from the Noun Project
Interacting with the HPC Cloud
Image(s) Template
AppMarket
Hands-On
Access
HPC Cloud UI : https://ui.hpccloud.surfsara.nl/
Username : ;-)
Password : ;-|
WWW : https://doc.hpccloud.surfsara.nl/bootcamp-20180410/

Introduction to HPC Cloud Computing

  • 1.
    INTRODUCTION TO HPCCLOUD COMPUTING SURF Research Bootcamp Nuno Ferreira, Ander Astudillo / 2018.04.10
  • 2.
    Agenda: Compute Track 10:15- 11:30 Introduction to UNIX 12:30 - 14:30 Introduction to Cluster Computing 15:00 - 17:00 Introduction to HPC Cloud Computing 15:00 Presentation: HPC Cloud Intro 15:30 Hands-on Access until the 17th April
  • 3.
  • 4.
  • 5.
    What is CloudComputing? Essential Characteristics • On-demand self-service • Network access • Resource pooling • Elasticity • Measured service Service Models • Software as a Service • Platform as a Service • Infrastructure as a Service “Ask 10 people what the cloud is, get 11 answers.” [?] “Cloud computing is a model for enabling ubiquitous, convenient, on- demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications, and services) that can be rapidly provisioned and released with minimal management effort or service provider interaction.”
  • 6.
    “Say Cloud onemore time …”
  • 7.
    Pizza as aService Traditional On-Premise Dining Table Drinks Electric / Gas Oven Fire Pizza Dough Tomato Sauce Toppings Cheese Home Made Software as a Service Dining Table Drinks Electric / Gas Oven Fire Pizza Dough Tomato Sauce Toppings Cheese Restaurant Platform as a Service Dining Table Drinks Electric / Gas Oven Fire Pizza Dough Tomato Sauce Toppings Cheese Pizza Delivery Infrastructure as a Service Dining Table Drinks Electric / Gas Oven Fire Pizza Dough Tomato Sauce Toppings Cheese Take & Bake YouManageVendorManages A. Barron, IBM
  • 8.
    Cloud Service Models Traditional On-Premise Applications Data Runtime Middleware OS Virtualization Servers Storage Networking Softwareas a Service Applications Data Runtime Middleware OS Virtualization Servers Storage Networking Platform as a Service Applications Data Runtime Middleware OS Virtualization Servers Storage Networking Infrastructure as a Service Applications Data Runtime Middleware OS Virtualization Servers Storage Networking YouManageVendorManages
  • 9.
  • 10.
    The Scientific Method Prediction Experiment Results RefineModel Hypothesis “Failure is a data point!”
  • 11.
    HPC Cloud Projects Researchfields • Biology • Genetics • Informatics • Chemistry • Ecology • Linguistics • Robotics • Business • Social sciences • Engineering • Humanities • … Use cases • Flexible software mix • Big VMs • Elasticity • Provide a service to peers • Software requiring licenses • Set up, test and deploy workflows • Training courses • Intensive computing • …
  • 12.
    HPC Cloud Benefits Generalbenefits • Data & Computing in Dutch soil • Data privacy inside your VM • Unrestricted Internet access • Collaborative work Technical benefits • No overcommitting • Tailor made your VM to your needs (flexibility) • Root access! • Controlled environment : choose your OS & packages • Fast private network between VM’s • No maximum wall time!
  • 13.
    HPC Cloud Shortcomings •No Service Level Agreement • No 24/7 Helpdesk service support • You maintain everything in your VM • You are responsible for all of your VM’s behavior • You must protect yourself against threats from the Internet • Accounting on VM uptime, not just compute time (like gas, light) • No automatic backups • Your laptop is faster than a 1 core VM ?
  • 14.
    How to obtainan HPC Cloud account? Resource Request 1 Technical Evaluation Allocation Resources Access Granted Time : hours to a few days 1 Resource Request form 2 helpdesk@surfsara.nl Helpdesk 2
  • 15.
    HPC Cloud Resources ComputeNodes Intel(R) Xeon(R) CPU E5-2698 v3 @ 2.30GHz 21 64 256GB 3.2TB Intel(R) Xeon(R) CPU E5-2698 v4 @ 2.20GHz 10 80 512GB 3.5TB Intel(R) Xeon(R) Gold 6148 CPU @ 2.40GHz 7 80 576GB 3.0TB Intel(R) Xeon(R) CPU E5-2640 v3 @ 2.60GHz 11 32 256GB 800GB 4 (GRID K2) Intel(R) Xeon(R) CPU E5-2609 v4 @ 1.70GHz 2 16 128GB 800GB 4 (Tesla P100) Storage Nodes • 900 TB Ceph net * 3 redundancy : 2.7 PB total Network • 10 Gbit # Top 10 beautiful data centers, Datacenter Dynamics, 2017.06.05 “GPU” icon by Misha Petrishchev from the Noun Project “SSD” icon by mikicon from the Noun Project
  • 16.
    Interacting with theHPC Cloud Image(s) Template AppMarket
  • 18.
    Hands-On Access HPC Cloud UI: https://ui.hpccloud.surfsara.nl/ Username : ;-) Password : ;-| WWW : https://doc.hpccloud.surfsara.nl/bootcamp-20180410/