Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

High Performance Cloud Computing

2,118 views

Published on

Talk given at a customer site recently

Published in: Technology, Business
  • Be the first to comment

  • Be the first to like this

High Performance Cloud Computing

  1. 1. High Performance Cloud Computing Deepak Singh P r i n c i p a l P r o d u c t M a n a g e r
  2. 2. Via butteryflysha under a CC-BY license
  3. 3. Image: Simon Cockell under CC-BY
  4. 4. High Scale Computing
  5. 5. using a large number ofcomputers at the sametime to solve a problem
  6. 6. 2
  7. 7. 1 High Throughput Computing
  8. 8. scale out
  9. 9. “embarassingly parallel”
  10. 10. constraints
  11. 11. constrained by capacity
  12. 12. More molecules Bigger systemsconstrained by capacity More simulations More dimensions
  13. 13. constrained by time
  14. 14. Upcoming conference Grant submissionsconstrained by time Impatience! Exploratory “spike” run
  15. 15. EC2
  16. 16. EC2Elastic Compute Cloud
  17. 17. elastic
  18. 18. programmatic
  19. 19. ec2-run-instances
  20. 20. AWS CloudFormation
  21. 21. EC2 instance types
  22. 22. s pe ty ce an standard “m1” st in 2EC high cpu “c1” high memory “m2” http://aws.amazon.com/ec2/instance-types/
  23. 23. s pe ty ce an standard “m1” st in 2EC high cpu “c1” high memory “m2” http://aws.amazon.com/ec2/instance-types/
  24. 24. ec2-terminate-instances
  25. 25. rapid provisioning
  26. 26. 10K in 45 minutes
  27. 27. design patterns
  28. 28. optimize forthroughput
  29. 29. TasksInstances
  30. 30. TasksQueueInstances
  31. 31. TasksQueueInstances
  32. 32. vertical scaling
  33. 33. Tasks Queue Instances Increaseinstance size
  34. 34. Tasks Queue Instances Increaseinstance size
  35. 35. horizontal scaling
  36. 36. Tasks Queue Instances Increaseinstance count
  37. 37. TasksQueueInstancesResultsStore
  38. 38. TasksQueueOn-premiseInstancesResultsStore
  39. 39. TasksQueueOn-premiseInstancesResultsStore
  40. 40. TasksQueueOn-premiseInstancesResultsStore
  41. 41. optimize for cost
  42. 42. optimize for cost maximize bang for buck
  43. 43. on-demand instances
  44. 44. reserved instances
  45. 45. spot instances
  46. 46. ideal for batch
  47. 47. persistent requests
  48. 48. all or nothing
  49. 49. use cases galore
  50. 50. Credit: Angel Pizzaro, U. Penn
  51. 51. 2 Cluster Computing
  52. 52. tightly coupled
  53. 53. MPI
  54. 54. Dua l Intel 23GB RA X 5570 GPGPU M“Neha lem” HVM 1.7TB scratch Cluster Compute
  55. 55. 10 gig E Cluster Compute
  56. 56. Placement Group
  57. 57. Placement groupCluster Compute
  58. 58. 231
  59. 59. 450
  60. 60. Cores 7040R max 41.82R peak 82.51
  61. 61. GPGPU
  62. 62. 2 x Tesla M2050
  63. 63. Getting Started
  64. 64. http://aws.amazon.com/hpc
  65. 65. 4 steps
  66. 66. 15 minutes
  67. 67. http://aws.amazon.com/ec2
  68. 68. performance
  69. 69. WIEN2K Parallel Performance H size 56,000 (25GB) Runtime (16x8 processors) Local (Infiniband) 3h:48 Cloud (10Gbps) 1h:30 ($40) 1200 atom unit cell; SCALAPACK+MPI diagonalization, matrix size 50k-100kCredit: K. Jorissen, F. D. Villa, and J. J. Rehr (U. Washington)
  70. 70. customer examples
  71. 71. Example Use Case #1Computational Fluid Dynamics Dynamic Clusters 40-180 CC1 instances
  72. 72. Example Use Case #2 Molecular Dynamics Steady Usage 32-40 CG1 instances
  73. 73. Example Use Case #3 Machine Learning Spiky, Experimental 8-20 CG1 instances
  74. 74. Customer Case Study: Bioproximity http://aws.amazon.com/solutions/case-studies/bioproximity/
  75. 75. Customer Case Study: cyclopic energy OpenFOAM® http://aws.amazon.com/solutions/case-studies/cyclopic-energy/
  76. 76. Customer Case Study: PSR Stochastic Dual Dynamic Programming 44,000 CPU hrs in Oct 2010 http://aws.amazon.com/solutions/case-studies/psr/
  77. 77. familiar tools
  78. 78. Oracle Grid Engine
  79. 79. MIT StarCluster
  80. 80. LSF
  81. 81. Moab/Torque
  82. 82. Condor
  83. 83. StackIQ Rocks+
  84. 84. Slurm
  85. 85. deesingh@amazon.com Twitter:@mndoci http://slideshare.net/mndoci http://mndoci.com Inspiration and ideas from Matt Wood, James Hamilton & Larry LessigCredit” Oberazzi under a CC-BY-NC-SA license

×