this is the deck from my keynote talk at the IEEE Cloud Conference in New York 2015. Survey of cloud architecture, cloud programming and more

1. gannon@Indiana.edu
Dennis.gannon@outlook.com

2. gannon@Indiana.edu
Dennis.gannon@outlook.com

6. Every research field is now a data
science field

7. Last
few decades
Thousand
years ago
Today and the FutureLast few
hundred years
2
2
2.
3
4
a
cG
a
a










Simulation of
complex phenomena
Newton’s laws,
Maxwell’s equations…
Description of natural
phenomena
Unify theory, experiment and
simulation with large
multidisciplinary Data
Using data exploration and
data mining
(from instruments, sensors,
humans…)
Distributed Communities

10. The Long Tail of Science

11. Programming tools: Scala, IPython, Azure ML, …
Frameworks: Spark, Hadoop, Yarn, HDInsight, Reef, Twister, Brisk
Software Defined Storage
Software Defined Networks
Hardware Abstraction/Virtualization
Container and Distributed Cluster OS (Docker, Mesosphere, K8S)

12. http://tce.technion.ac.il/files/2012/06/Scott-shenker.pdf
www.opennetsummit.org/pdf/2013/presentations/albert_greenberg.pdf
http://www.cs.princeton.edu/~jrex/papers/pyretic-login13.pdf

17. The IPython notebook deployed on a VM.
1. Create a coreos VM and open the https port 443
2. Login and issue this one command
$docker run -d -p 443:8888 -e "PASSWORD=****" ipython/scipyserver
3. Go to https://yourVMaddress and log in.

18. compute
compute
compute
compute
compute
compute
compute
compute
compute
compute
compute
compute

23. Vowpal Wabbit? Datumbox?

25. Marathon
Master
node
Master
Backup
Worker
node
Worker
node
Worker
node
Mesos
Zookeeper

28. Cloud
SDN
NIH data
commons

29. • Many Examples
• Challenges:
• I only talked about the Analysis … but it there is more
• Sustainability
• Sharing
• Reproducible Science
Data
Acquisition &
modelling
Collaboration
and
visualisation
Analysis &
data mining
Dissemination
& sharing
Archiving and
preserving