20 slides for a 10 minute talk! Short presentation for the 2012 Amazon Web Services re:Invent conference. This briefly covers some Computer Aided Engineering (CAE) simulation work done on Amazon using CST Studio software. Interesting Linux/Windows/Cloudbursting use case.

1. Accelerating Research
10-minute Vignette for 2012 Amazon re:Invent Conference
chris@bioteam.net
1
1

2. I’m Chris.
I’m an infrastructure/bio geek.
I work for the BioTeam.
Twitstuff: @chris_dag 2
2

3. BioTeam - Not your standard AWS user
Who, what & why
‣ Independent consulting shop
‣ Run by scientists forced to
learn IT, SW & HPC to get our
own genetics research done
‣ 12+ years bridging the “gap”
between science, IT & high
performance computing
‣ www.bioteam.net
3
3

4. Meta-Issues
What you don’t know WILL hurt you.
4
4

5. Lets Be Blunt
A common theme across all our biotech, pharma, EDU and
research clients ...
‣ It’s almost 2013
‣ AWS is comprehendible in minute:hours; pilot projects
and proof-of-concepts can be done in hours:days. Really.
‣ The TECHNICAL bits are easy or at least manageable
‣ The POLICY & PROCEDURE bits will destroy your soul
‣ If I could only give 1 piece of advice to researchers ...
5
5

6. One Piece of Advice ...
For anyone doing research HPC or “science” on the cloud
‣ The technical issues surrounding cloud-based scientific
research are easy or at least surmountable
‣ The hard part is sorting out the internal policies and
procedures that govern cloud use
‣ Ignore this at your peril
6
6

7. Five Minute Case Study
Computer Aided Engineering (CAE) on AWS
7
7

8. NMR Probehead Simulation on AWS
Next Generation Nuclear Magnetic Resonance
‣ CAE Simulation Project
‣ via www.hpcexperiment.com
‣ Software: CST Studio 2012
‣ My role: Volunteer HPC Mentor
8
8

9. Why this was an interesting project
Simulating next-generation NMR probeheads
‣ Frontend interface is graphics
heavy and requires Windows
‣ Studio ‘solvers’ run Linux or
Windows; support GPUs and MPI
task distribution
‣ Simultaneous use of local and
cloud-based solvers actually works
‣ flexLM license server involved
‣ Non-trivial security and geo-
location requirements
9
9

10. Initial Architecture 10
10

11. At experiment end it looked like this ... 11
11

12. Amazon FTW
AWS makes the easy stuff fun and the hard stuff possible
12
12

13. Nightmare on any other cloud
Why this work was ‘easy’ on Amazon AWS ...
‣ Tick Tock ...
‣ Don’t have the time to get into the CAE simulation details
‣ Lets discuss why this simulation workload would be
much, much harder to do on some other cloud
platform ...
13
13

14. Nightmare on any other cloud
Why this work was ‘easy’ on Amazon AWS ...
‘Brand X’ Cloud AWS
‣ EC2, S3, EBS, RDS, SNS,
1. Virtual Servers SQS, SWS, GPUs, SSDs,
2. Block Storage CloudFormation, VPC, ENIs,
SecurityGroups, 10GbE
3. Object Storage DirectConnect, Reserved
4. ... and maybe some other Instances, ImportExport,
stuff if I’m lucky Spot Market
‣ And ~25 other products and
service features with more
added monthly 14
14

15. One very specific example
Easy on AWS; much harder elsewhere
‣ The widely used FLEXlm
license server uses NIC
MAC addresses when
generating license keys
‣ Different MAC? Science
stops. Screwed.
‣ VPC ENIs allow separation
of MAC address from
Network Interface.
Badass.
15
15

16. A few other examples ...
Why this work was ‘easy’ on Amazon AWS ...
Incredibly powerful. Actually useful.
VPC Approachable even if you are not an IPSEC or BGP
routing god.
Compelling economics. Once you start you’ll likely
never run anywhere else.
Spot Market
The competition can’t compete.
cc* & cg* Fat nodes with bidirectional 10GbE bandwidth.
ec2 instance And don’t get me started on SSD or Provisioned-
types performance EBS volumes.
16
16

17. Science on AWS
Wrapping up ...
‣ It’s not rocket science; yeah
we can do this just about
anywhere
‣ It’s just easier and faster on
AWS because they have more
building blocks; I can spend
more time on actual science
problems You mean I get paid for this?
17
17

18. Science on AWS, cont.
Wrapping up ...
‣ There is no one-size-fits-all
research design pattern ...
‣ Very few of us have a single
pipeline or workflow that we can
devote endless engineering effort to
‣ We are not going to toss out
hundreds of legacy codes and
rewrite everything for GPUs or
MapReduce
‣ For research HPC it’s all about the I love my job
building blocks { and how we can
effectively use/deploy them }
18
18

19. One last plug
‣ Want more info or details?
‣ Watch http://bioteam.net for upcoming posts that
document in detail the CST Studio on AWS effort
‣ And by the way ...
‣ hpcexperiment.com round #2 is starting soon; They need
volunteers, resource providers and new use cass
19
19

20. Thanks!
Slides online at: http://slideshare.net/chrisdag/
Twitter: @chris_dag
20
20