Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
High Performance   Cloud Computing                                 Deepak Singh         P r i n c i p a l   P r o d u c t ...
Via butteryflysha under a CC-BY license
Image: Simon Cockell under CC-BY
High Scale Computing
using a large number ofcomputers at the sametime to solve a problem
2
1    High Throughput       Computing
scale out
“embarassingly   parallel”
constraints
constrained by   capacity
More molecules    Bigger systemsconstrained by   capacity    More simulations    More dimensions
constrained by time
Upcoming conference       Grant submissionsconstrained by time          Impatience!     Exploratory “spike” run
EC2
EC2Elastic Compute Cloud
elastic
programmatic
ec2-run-instances
AWS CloudFormation
EC2 instance types
s             pe           ty           ce        an            standard “m1”     st  in  2EC              high cpu “c1”  ...
s             pe           ty           ce        an            standard “m1”     st  in  2EC              high cpu “c1”  ...
ec2-terminate-instances
rapid provisioning
10K in 45 minutes
design patterns
optimize forthroughput
TasksInstances
TasksQueueInstances
TasksQueueInstances
vertical scaling
Tasks            Queue            Instances Increaseinstance   size
Tasks            Queue            Instances Increaseinstance   size
horizontal scaling
Tasks            Queue            Instances Increaseinstance  count
TasksQueueInstancesResultsStore
TasksQueueOn-premiseInstancesResultsStore
TasksQueueOn-premiseInstancesResultsStore
TasksQueueOn-premiseInstancesResultsStore
optimize for cost
optimize for cost maximize bang for buck
on-demand instances
reserved instances
spot instances
ideal for batch
persistent requests
all or nothing
use cases galore
Credit: Angel Pizzaro, U. Penn
2     Cluster    Computing
tightly coupled
MPI
Dua  l Intel          23GB RA  X 5570        GPGPU                              M“Neha   lem”                             ...
10 gig E           Cluster Compute
Placement  Group
Placement                    groupCluster Compute
231
450
Cores      7040R   max           41.82R   peak           82.51
GPGPU
2 x Tesla   M2050
Getting Started
http://aws.amazon.com/hpc
4 steps
15 minutes
http://aws.amazon.com/ec2
performance
WIEN2K Parallel                                                                    Performance                            ...
customer examples
Example Use Case #1Computational Fluid Dynamics      Dynamic Clusters   40-180 CC1 instances
Example Use Case #2    Molecular Dynamics       Steady Usage   32-40 CG1 instances
Example Use Case #3     Machine Learning    Spiky, Experimental    8-20 CG1 instances
Customer Case Study: Bioproximity          http://aws.amazon.com/solutions/case-studies/bioproximity/
Customer Case Study: cyclopic energy                           OpenFOAM®         http://aws.amazon.com/solutions/case-stud...
Customer Case Study: PSR                  Stochastic Dual Dynamic Programming 44,000 CPU hrs in Oct 2010             http:...
familiar tools
Oracle Grid Engine
MIT StarCluster
LSF
Moab/Torque
Condor
StackIQ Rocks+
Slurm
deesingh@amazon.com                                                             Twitter:@mndoci                           ...
High Performance Cloud Computing
High Performance Cloud Computing
High Performance Cloud Computing
High Performance Cloud Computing
High Performance Cloud Computing
High Performance Cloud Computing
High Performance Cloud Computing
High Performance Cloud Computing
High Performance Cloud Computing
High Performance Cloud Computing
High Performance Cloud Computing
High Performance Cloud Computing
High Performance Cloud Computing
High Performance Cloud Computing
High Performance Cloud Computing
High Performance Cloud Computing
High Performance Cloud Computing
Upcoming SlideShare
Loading in …5
×

High Performance Cloud Computing

2,107 views

Published on

Talk given at a customer site recently

Published in: Technology, Business
  • Be the first to comment

  • Be the first to like this

High Performance Cloud Computing

  1. 1. High Performance Cloud Computing Deepak Singh P r i n c i p a l P r o d u c t M a n a g e r
  2. 2. Via butteryflysha under a CC-BY license
  3. 3. Image: Simon Cockell under CC-BY
  4. 4. High Scale Computing
  5. 5. using a large number ofcomputers at the sametime to solve a problem
  6. 6. 2
  7. 7. 1 High Throughput Computing
  8. 8. scale out
  9. 9. “embarassingly parallel”
  10. 10. constraints
  11. 11. constrained by capacity
  12. 12. More molecules Bigger systemsconstrained by capacity More simulations More dimensions
  13. 13. constrained by time
  14. 14. Upcoming conference Grant submissionsconstrained by time Impatience! Exploratory “spike” run
  15. 15. EC2
  16. 16. EC2Elastic Compute Cloud
  17. 17. elastic
  18. 18. programmatic
  19. 19. ec2-run-instances
  20. 20. AWS CloudFormation
  21. 21. EC2 instance types
  22. 22. s pe ty ce an standard “m1” st in 2EC high cpu “c1” high memory “m2” http://aws.amazon.com/ec2/instance-types/
  23. 23. s pe ty ce an standard “m1” st in 2EC high cpu “c1” high memory “m2” http://aws.amazon.com/ec2/instance-types/
  24. 24. ec2-terminate-instances
  25. 25. rapid provisioning
  26. 26. 10K in 45 minutes
  27. 27. design patterns
  28. 28. optimize forthroughput
  29. 29. TasksInstances
  30. 30. TasksQueueInstances
  31. 31. TasksQueueInstances
  32. 32. vertical scaling
  33. 33. Tasks Queue Instances Increaseinstance size
  34. 34. Tasks Queue Instances Increaseinstance size
  35. 35. horizontal scaling
  36. 36. Tasks Queue Instances Increaseinstance count
  37. 37. TasksQueueInstancesResultsStore
  38. 38. TasksQueueOn-premiseInstancesResultsStore
  39. 39. TasksQueueOn-premiseInstancesResultsStore
  40. 40. TasksQueueOn-premiseInstancesResultsStore
  41. 41. optimize for cost
  42. 42. optimize for cost maximize bang for buck
  43. 43. on-demand instances
  44. 44. reserved instances
  45. 45. spot instances
  46. 46. ideal for batch
  47. 47. persistent requests
  48. 48. all or nothing
  49. 49. use cases galore
  50. 50. Credit: Angel Pizzaro, U. Penn
  51. 51. 2 Cluster Computing
  52. 52. tightly coupled
  53. 53. MPI
  54. 54. Dua l Intel 23GB RA X 5570 GPGPU M“Neha lem” HVM 1.7TB scratch Cluster Compute
  55. 55. 10 gig E Cluster Compute
  56. 56. Placement Group
  57. 57. Placement groupCluster Compute
  58. 58. 231
  59. 59. 450
  60. 60. Cores 7040R max 41.82R peak 82.51
  61. 61. GPGPU
  62. 62. 2 x Tesla M2050
  63. 63. Getting Started
  64. 64. http://aws.amazon.com/hpc
  65. 65. 4 steps
  66. 66. 15 minutes
  67. 67. http://aws.amazon.com/ec2
  68. 68. performance
  69. 69. WIEN2K Parallel Performance H size 56,000 (25GB) Runtime (16x8 processors) Local (Infiniband) 3h:48 Cloud (10Gbps) 1h:30 ($40) 1200 atom unit cell; SCALAPACK+MPI diagonalization, matrix size 50k-100kCredit: K. Jorissen, F. D. Villa, and J. J. Rehr (U. Washington)
  70. 70. customer examples
  71. 71. Example Use Case #1Computational Fluid Dynamics Dynamic Clusters 40-180 CC1 instances
  72. 72. Example Use Case #2 Molecular Dynamics Steady Usage 32-40 CG1 instances
  73. 73. Example Use Case #3 Machine Learning Spiky, Experimental 8-20 CG1 instances
  74. 74. Customer Case Study: Bioproximity http://aws.amazon.com/solutions/case-studies/bioproximity/
  75. 75. Customer Case Study: cyclopic energy OpenFOAM® http://aws.amazon.com/solutions/case-studies/cyclopic-energy/
  76. 76. Customer Case Study: PSR Stochastic Dual Dynamic Programming 44,000 CPU hrs in Oct 2010 http://aws.amazon.com/solutions/case-studies/psr/
  77. 77. familiar tools
  78. 78. Oracle Grid Engine
  79. 79. MIT StarCluster
  80. 80. LSF
  81. 81. Moab/Torque
  82. 82. Condor
  83. 83. StackIQ Rocks+
  84. 84. Slurm
  85. 85. deesingh@amazon.com Twitter:@mndoci http://slideshare.net/mndoci http://mndoci.com Inspiration and ideas from Matt Wood, James Hamilton & Larry LessigCredit” Oberazzi under a CC-BY-NC-SA license

×