0
Utility HPC:Right Systems, Right Scale,Right ScienceJason Stowe, CEO@jasonastowe, @cyclecomputing
I’m here to recruit you,for a cause
We believeutility access to compute powermakes impossible science,possible.
Dynamic, utility access tocompute poweris as important as uptime
(that’s why coded infrastructureis critical)
Skeptical?Flickr:	  Tourist	  on	  Earth	  
In prior years (today?)Researchers/engineers waitedfor computing
For	  the	  horsepower	  
For	  the	  place	  	  to	  put	  it	  
For	  it	  to	  be	  	  Configured..	  Flickr: vaxomatic
Yesterday, high performanceengineering, science clusterswere…Too smallwhen you need it most,Too largeevery other time.
The Innovation Bottleneck:Researchers/Scientists/EngineersForced to size questions to theinfrastructure you have
 Multi-­‐tenant	  systems	  create	  float	  capacity	  That	  is	  critical	  to	  innovation	  	  
The60’sThe70’sThe80’sThe90’sThe00’sFrom centralized to decentralized, collaborative to independentand right back again!The...
Ask aQuestion Hypothesize PredictExperiment /Test Analyze Final Results	  	  	  	  The Scientific MethodTest and Analyze s...
Ask aQuestion Hypothesize PredictExperiment /Test Analyze Final Results	  	  	  	  The Scientific MethodAny improvements t...
A Challenge Across Industries— 3 of Top 5 Insurance— 6 of Top 8 Pharmaceutical— 2 of Top 3 Banks— 2 of Top 3 Genomics ...
Utility HPC in the NewsWSJ, NYTimes, Wired, Bio-IT World BusinessWeek
To accelerate science, we needautomation
Management SoftwareCC1/CCGInstancesEBSS3SharedFSEBSUtility	  HPC	  Cluster	  -­‐ Scales	  to	  50,000+	  cores	  -­‐ Data	...
50,000-core CycleCloudUsing Chef and AWSChefConf 2012
10,600-instance clusteragainst cancer targetChefConf 2013
Created in 2 hoursConfigured with Search,with Data bags
one Chef 11 server
We make software tools to easily orchestrate complexworkloads and data access across Utility HPCToday is a survey of use c...
Dynamic, utility access tocompute poweris as important as uptime
Why?
#1: “Better” Science =“Answer the question we want toask”, not constrained to what fitson local compute power
#2 “Faster” Science =Run this “better” science,that would have takenmonths or yearsin hours or days
Survey of Use Casesþ  Drug Designþ  CAD/CAMþ  Genomics…
Life Sciences & Compute?ComputeData/BandwidthGenomicsMolecularModelingCAD/CAMAll SampleAnalysisProteomicsBiomarker/Image A...
Why is this important?
(W.H.O./Globocan 2008)
~2 million Type 2 diabetics,~200k Type 1
Every day iscrucial and costly
Before:Trade-off compute time vs.accuracyNow:Accurate analysis, fewer falsenegatives, fasterInitialCoarseScreenHigherQuali...
Big 10 PharmaBuilt 10,600 instance cluster($44M) in 2 hours, ran40 years of sciencein 11 hours for $4,372
Most Recent Utility Supercomputerserver count:
AWS Console view:
Cycle’s view of this cluster:One Chef 11 Server
Earlier Drug DesignNovartis discussed at BioIT2012— Needed—  Push-button Utility Supercomputer for molecularmodeling— C...
—  Capacity is no longer an issue—  Hardware = software—  Testing (error handling, unit testing, etc.)e.g. Cycle spent ...
 Servers	  are	  not	  	  house	  plants	  	  
 Servers	  are	  wheat	  	  
Survey of Use Casesþ  Drug Designþ  CAD/CAMþ  Genomics…
Nuclear Power Plant simulation
We don’t’ know what they’rerunning, but it has “Safety”
600-core CAD/CAM3 Quarters of a year wait became 3 weeksSiteDataCorporateFirewall3 Weeks insteadOf 3 QuartersSecureHPCClus...
Survey of Use Casesþ  Drug Designþ  CAD/CAMþ  Genomics…
Gene Expression AnalysisMorgridge Institute for ResearchRun holistic comparison of all 78 terabyte stem cellRNA samples to...
78 TB of Stem Cell RNA
1 Million compute hours,115 years of computing in1 week for $19,555
Gene Expression AnalysisMorgridge Institute for Research— Cluster details—  5,000 to 10,000 cores for a week—  Very lon...
Survey of Use Casesþ  Drug Designþ  CAD/CAMþ  Genomics…
Code can accelerate Science
Ask aQuestion Hypothesize PredictExperiment /Test Analyze Final Results	  	  	  	  The Scientific Method on Utility HPCYie...
Dynamic, utility access tocompute poweris as important as uptime
I’m here to recruit you,for a cause
Contribute to Chef.Make the community better.And you will help Cyclemake impossible science,possible.
2013 BigScience Challenge$10,000 of free computing to sciencebenefitting humanity2012 winner: 115yr Genomic analysisEnter ...
Thank You! Questions?
Utility HPC: Right Systems, Right Scale, Right Science
Upcoming SlideShare
Loading in...5
×

Utility HPC: Right Systems, Right Scale, Right Science

1,377

Published on

Published in: Technology, Education
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,377
On Slideshare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
11
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Transcript of "Utility HPC: Right Systems, Right Scale, Right Science"

  1. 1. Utility HPC:Right Systems, Right Scale,Right ScienceJason Stowe, CEO@jasonastowe, @cyclecomputing
  2. 2. I’m here to recruit you,for a cause
  3. 3. We believeutility access to compute powermakes impossible science,possible.
  4. 4. Dynamic, utility access tocompute poweris as important as uptime
  5. 5. (that’s why coded infrastructureis critical)
  6. 6. Skeptical?Flickr:  Tourist  on  Earth  
  7. 7. In prior years (today?)Researchers/engineers waitedfor computing
  8. 8. For  the  horsepower  
  9. 9. For  the  place    to  put  it  
  10. 10. For  it  to  be    Configured..  Flickr: vaxomatic
  11. 11. Yesterday, high performanceengineering, science clusterswere…Too smallwhen you need it most,Too largeevery other time.
  12. 12. The Innovation Bottleneck:Researchers/Scientists/EngineersForced to size questions to theinfrastructure you have
  13. 13.  Multi-­‐tenant  systems  create  float  capacity  That  is  critical  to  innovation    
  14. 14. The60’sThe70’sThe80’sThe90’sThe00’sFrom centralized to decentralized, collaborative to independentand right back again!The10’sMainframes VAX   The  PC   Beowulf Clusters Central  Clouds  100% 60% 0% 40% ??? %SHARING  ~  0Mbit   ~ 1Mbit ~ 10Mbit ~  1000  Mbit   ~ 10,000 MbitBigger, better but further and further away from the scientist’s lab
  15. 15. Ask aQuestion Hypothesize PredictExperiment /Test Analyze Final Results        The Scientific MethodTest and Analyze stagesrequire the most time,compute, and data
  16. 16. Ask aQuestion Hypothesize PredictExperiment /Test Analyze Final Results        The Scientific MethodAny improvements to thiscycle yield multiplicativebenefits
  17. 17. A Challenge Across Industries— 3 of Top 5 Insurance— 6 of Top 8 Pharmaceutical— 2 of Top 3 Banks— 2 of Top 3 Genomics Sequencing— 1 of Top 2 FPGA
  18. 18. Utility HPC in the NewsWSJ, NYTimes, Wired, Bio-IT World BusinessWeek
  19. 19. To accelerate science, we needautomation
  20. 20. Management SoftwareCC1/CCGInstancesEBSS3SharedFSEBSUtility  HPC  Cluster  -­‐ Scales  to  50,000+  cores  -­‐ Data  Scheduling  -­‐ Workload  portability  Data &ApplicationAwareMovementTraditionalSchedulerMassive ScaleBased upon workloadSecure, HPCClusterUserHPCReporting &Audit
  21. 21. 50,000-core CycleCloudUsing Chef and AWSChefConf 2012
  22. 22. 10,600-instance clusteragainst cancer targetChefConf 2013
  23. 23. Created in 2 hoursConfigured with Search,with Data bags
  24. 24. one Chef 11 server
  25. 25. We make software tools to easily orchestrate complexworkloads and data access across Utility HPCToday is a survey of use cases…10,600 instanceLife ScienceMolecularModeling600 coreManufacturingNuclear PowerPlant for safetysimulationGenomicAnalysisRNA forStem Cells
  26. 26. Dynamic, utility access tocompute poweris as important as uptime
  27. 27. Why?
  28. 28. #1: “Better” Science =“Answer the question we want toask”, not constrained to what fitson local compute power
  29. 29. #2 “Faster” Science =Run this “better” science,that would have takenmonths or yearsin hours or days
  30. 30. Survey of Use Casesþ  Drug Designþ  CAD/CAMþ  Genomics…
  31. 31. Life Sciences & Compute?ComputeData/BandwidthGenomicsMolecularModelingCAD/CAMAll SampleAnalysisProteomicsBiomarker/Image AnalysisSensor Data ImportCreating fakeCharts, withFake Data
  32. 32. Why is this important?
  33. 33. (W.H.O./Globocan 2008)
  34. 34. ~2 million Type 2 diabetics,~200k Type 1
  35. 35. Every day iscrucial and costly
  36. 36. Before:Trade-off compute time vs.accuracyNow:Accurate analysis, fewer falsenegatives, fasterInitialCoarseScreenHigherQualityAnalysisBestQualityProcess for Drug DesignHigherQualityAnalysisBestQuality
  37. 37. Big 10 PharmaBuilt 10,600 instance cluster($44M) in 2 hours, ran40 years of sciencein 11 hours for $4,372
  38. 38. Most Recent Utility Supercomputerserver count:
  39. 39. AWS Console view:
  40. 40. Cycle’s view of this cluster:One Chef 11 Server
  41. 41. Earlier Drug DesignNovartis discussed at BioIT2012— Needed—  Push-button Utility Supercomputer for molecularmodeling— Created—  30,000 core run across US/EU Cloud (AWS)—  10 years of compute in 8 hours for $10,000—  Found 3 compounds now in the wetlab as a result
  42. 42. —  Capacity is no longer an issue—  Hardware = software—  Testing (error handling, unit testing, etc.)e.g. Cycle spent ~$1M dollars on AWS over 5 years—  The only way to do this is to automateLessons learned
  43. 43.  Servers  are  not    house  plants    
  44. 44.  Servers  are  wheat    
  45. 45. Survey of Use Casesþ  Drug Designþ  CAD/CAMþ  Genomics…
  46. 46. Nuclear Power Plant simulation
  47. 47. We don’t’ know what they’rerunning, but it has “Safety”
  48. 48. 600-core CAD/CAM3 Quarters of a year wait became 3 weeksSiteDataCorporateFirewall3 Weeks insteadOf 3 QuartersSecureHPCClusterTBs FSExternal Cloud  ~600 CPU clusterScheduledDataEngineer
  49. 49. Survey of Use Casesþ  Drug Designþ  CAD/CAMþ  Genomics…
  50. 50. Gene Expression AnalysisMorgridge Institute for ResearchRun holistic comparison of all 78 terabyte stem cellRNA samples to build a unique gene expressiondatabaseMake it easier to replicate disease in petri dishes w/induced stem cells
  51. 51. 78 TB of Stem Cell RNA
  52. 52. 1 Million compute hours,115 years of computing in1 week for $19,555
  53. 53. Gene Expression AnalysisMorgridge Institute for Research— Cluster details—  5,000 to 10,000 cores for a week—  Very long individual analysis were check-pointed =Spot instance usage possible
  54. 54. Survey of Use Casesþ  Drug Designþ  CAD/CAMþ  Genomics…
  55. 55. Code can accelerate Science
  56. 56. Ask aQuestion Hypothesize PredictExperiment /Test Analyze Final Results        The Scientific Method on Utility HPCYield “Better”, “Faster”Research for less $
  57. 57. Dynamic, utility access tocompute poweris as important as uptime
  58. 58. I’m here to recruit you,for a cause
  59. 59. Contribute to Chef.Make the community better.And you will help Cyclemake impossible science,possible.
  60. 60. 2013 BigScience Challenge$10,000 of free computing to sciencebenefitting humanity2012 winner: 115yr Genomic analysisEnter at:http://cyclecomputing.com/big-science-challenge/enter
  61. 61. Thank You! Questions?
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×