Intel Theater Presentation - SC11

3,984 views
3,905 views

Published on

Published in: Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
3,984
On SlideShare
0
From Embeds
0
Number of Embeds
259
Actions
Shares
0
Downloads
0
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Intel Theater Presentation - SC11

  1. 1. HPC with Amazon EC2 Deepak Singh @mndoci P r i n c i p a l P r o d u c t M a n a g e r
  2. 2. Amazon Web Services
  3. 3. 4
  4. 4. 2
  5. 5. 1. Infrastructure
  6. 6. ec2-run-instances
  7. 7. on demand global secure
  8. 8. programmable
  9. 9. elastic
  10. 10. instance types
  11. 11. t1.micro standard (m1)high memory (m2) high CPU (c1)
  12. 12. high performance
  13. 13. “Our 40-instance (m2.2xlarge) cluster canscan, filter, and aggregate 1 billion rows in950 milliseconds.”Mike Driscoll - Metamarkets
  14. 14. cluster computing
  15. 15. MPI
  16. 16. bandwidth intensive
  17. 17. Cluster Compute Instance
  18. 18. 2*Intel Xeon 5570 8 cores w/HT 23 GB RAM 1.7 TB disk HVM cc1.4xlarge
  19. 19. 10 gig E
  20. 20. Placement Group
  21. 21. ull- on f i ectbis Placement group
  22. 22. linpack
  23. 23. Cores 7040R max 41.82R peak 82.51
  24. 24. 231November 2010
  25. 25. 451June 2011
  26. 26. WIEN2K Parallel Performance H size 56,000 (25GB) Runtime (16x8 processors) Local (Infiniband) 3h:48 Cloud (10Gbps) 1h:30 ($40) 1200 atom unit cell; SCALAPACK+MPI diagonalization, matrix size 50k-100kCredit: K. Jorissen, F. D. Villa, and J. J. Rehr (U. Washington)
  27. 27. New Cluster Compute Instance
  28. 28. 2*Intel Xeon16 cores w/HT60.5 GB RAM 3.4 TB disk HVM cc2.8xlarge
  29. 29. linpack
  30. 30. Cores 17024R max 240.09R peak 354.12
  31. 31. 42November 2011
  32. 32. optimizing costs
  33. 33. on-demand
  34. 34. reserved
  35. 35. spot
  36. 36. 30,472 cores
  37. 37. $1279/hr
  38. 38. 2. Orchestration
  39. 39. AWS CloudFormation
  40. 40. bootstrap
  41. 41. Cloud Init
  42. 42. #cloud-configpackages:! - httpd!runcmd:! - /etc/init.d http start! - echo "<h1>hello,world"</h1> ! ! > /var/www/html/index.html
  43. 43. #!/bin/shec2-run-instancesami-8c1fece5 ! -n 1 ! -t m1.small ! -g deesinghdemo-SG ! -k deesinghdemo-keypair ! --user-data-file .cloudconfig.txt
  44. 44. chef/puppet
  45. 45. familiar tools
  46. 46. LSF
  47. 47. Grid Engine
  48. 48. BrightCluster Manager
  49. 49. combining worlds
  50. 50. MIT Starcluster
  51. 51. $ starcluster start mycluster$ starcluster listclusters
  52. 52. http://www.bioteam.net/2011/03/dude-you-got-some-chef-in-my-starcluster/
  53. 53. Provisions Cluster Shared Storage Monitoring Bootstraps StarCluster Includes 200 GB Public DatasetProvisioned Stack = Submit jobs to Grid Engine
  54. 54. Image: Chris Dagdigian

×