AWS re:Invent - Accelerating Research
Upcoming SlideShare
Loading in...5
×
 

AWS re:Invent - Accelerating Research

on

  • 1,181 views

20 slides for a 10 minute talk! ...

20 slides for a 10 minute talk!
Short presentation for the 2012 Amazon Web Services re:Invent conference. This briefly covers some Computer Aided Engineering (CAE) simulation work done on Amazon using CST Studio software. Interesting Linux/Windows/Cloudbursting use case.

Statistics

Views

Total Views
1,181
Views on SlideShare
1,181
Embed Views
0

Actions

Likes
3
Downloads
20
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

AWS re:Invent - Accelerating Research AWS re:Invent - Accelerating Research Presentation Transcript

  • Accelerating Research10-minute Vignette for 2012 Amazon re:Invent Conference chris@bioteam.net 1 1
  • I’m Chris.I’m an infrastructure/bio geek.I work for the BioTeam. Twitstuff: @chris_dag 2 2
  • BioTeam - Not your standard AWS userWho, what & why ‣ Independent consulting shop ‣ Run by scientists forced to learn IT, SW & HPC to get our own genetics research done ‣ 12+ years bridging the “gap” between science, IT & high performance computing ‣ www.bioteam.net 3 3
  • Meta-IssuesWhat you don’t know WILL hurt you. 4 4
  • Lets Be BluntA common theme across all our biotech, pharma, EDU andresearch clients ... ‣ It’s almost 2013 ‣ AWS is comprehendible in minute:hours; pilot projects and proof-of-concepts can be done in hours:days. Really. ‣ The TECHNICAL bits are easy or at least manageable ‣ The POLICY & PROCEDURE bits will destroy your soul ‣ If I could only give 1 piece of advice to researchers ... 5 5
  • One Piece of Advice ...For anyone doing research HPC or “science” on the cloud ‣ The technical issues surrounding cloud-based scientific research are easy or at least surmountable ‣ The hard part is sorting out the internal policies and procedures that govern cloud use ‣ Ignore this at your peril 6 6
  • Five Minute Case StudyComputer Aided Engineering (CAE) on AWS 7 7
  • NMR Probehead Simulation on AWSNext Generation Nuclear Magnetic Resonance ‣ CAE Simulation Project ‣ via www.hpcexperiment.com ‣ Software: CST Studio 2012 ‣ My role: Volunteer HPC Mentor 8 8
  • Why this was an interesting projectSimulating next-generation NMR probeheads ‣ Frontend interface is graphics heavy and requires Windows ‣ Studio ‘solvers’ run Linux or Windows; support GPUs and MPI task distribution ‣ Simultaneous use of local and cloud-based solvers actually works ‣ flexLM license server involved ‣ Non-trivial security and geo- location requirements 9 9
  • Initial Architecture 10 10
  • At experiment end it looked like this ... 11 11
  • Amazon FTWAWS makes the easy stuff fun and the hard stuff possible 12 12
  • Nightmare on any other cloudWhy this work was ‘easy’ on Amazon AWS ... ‣ Tick Tock ... ‣ Don’t have the time to get into the CAE simulation details ‣ Lets discuss why this simulation workload would be much, much harder to do on some other cloud platform ... 13 13
  • Nightmare on any other cloudWhy this work was ‘easy’ on Amazon AWS ...‘Brand X’ Cloud AWS ‣ EC2, S3, EBS, RDS, SNS,1. Virtual Servers SQS, SWS, GPUs, SSDs,2. Block Storage CloudFormation, VPC, ENIs, SecurityGroups, 10GbE3. Object Storage DirectConnect, Reserved4. ... and maybe some other Instances, ImportExport, stuff if I’m lucky Spot Market ‣ And ~25 other products and service features with more added monthly 14 14
  • One very specific exampleEasy on AWS; much harder elsewhere ‣ The widely used FLEXlm license server uses NIC MAC addresses when generating license keys ‣ Different MAC? Science stops. Screwed. ‣ VPC ENIs allow separation of MAC address from Network Interface. Badass. 15 15
  • A few other examples ...Why this work was ‘easy’ on Amazon AWS ... Incredibly powerful. Actually useful. VPC Approachable even if you are not an IPSEC or BGP routing god. Compelling economics. Once you start you’ll likely never run anywhere else.Spot Market The competition can’t compete. cc* & cg* Fat nodes with bidirectional 10GbE bandwidth.ec2 instance And don’t get me started on SSD or Provisioned- types performance EBS volumes. 16 16
  • Science on AWSWrapping up ... ‣ It’s not rocket science; yeah we can do this just about anywhere ‣ It’s just easier and faster on AWS because they have more building blocks; I can spend more time on actual science problems You mean I get paid for this? 17 17
  • Science on AWS, cont.Wrapping up ... ‣ There is no one-size-fits-all research design pattern ... ‣ Very few of us have a single pipeline or workflow that we can devote endless engineering effort to ‣ We are not going to toss out hundreds of legacy codes and rewrite everything for GPUs or MapReduce ‣ For research HPC it’s all about the I love my job building blocks { and how we can effectively use/deploy them } 18 18
  • One last plug ‣ Want more info or details? ‣ Watch http://bioteam.net for upcoming posts that document in detail the CST Studio on AWS effort ‣ And by the way ... ‣ hpcexperiment.com round #2 is starting soon; They need volunteers, resource providers and new use cass 19 19
  • Thanks!Slides online at: http://slideshare.net/chrisdag/ Twitter: @chris_dag 20 20