Downloaded 21 times
Uploaded byMicrosoft Azure for Research
ieee cloud 2015 keynote talk
The document discusses the evolution of research fields into data science domains, emphasizing the integration of theory, experiment, and simulation using large datasets. It highlights various programming tools, frameworks, and the challenges of sustainability, sharing, and reproducibility in data science. Furthermore, it outlines the importance of collaboration in data acquisition, modeling, and analysis.
More Related Content
Similar to ieee cloud 2015 keynote talk
![Getting Started with Apache Spark: Big Data Made Simple [Free Meetup]](https://cdn.slidesharecdn.com/ss_thumbnails/apachesparkgettingstarted-260203175547-8361bcc3-thumbnail.jpg?width=640&height=640&fit=bounds)
ieee cloud 2015 keynote talk
- 1.
- 2.
- 6. Every research fieldis now a data science field
- 7. Last few decades Thousand years ago Todayand the FutureLast few hundred years 2 2 2. 3 4 a cG a a Simulation of complex phenomena Newton’s laws, Maxwell’s equations… Description of natural phenomena Unify theory, experiment and simulation with large multidisciplinary Data Using data exploration and data mining (from instruments, sensors, humans…) Distributed Communities
- 10. The Long Tailof Science
- 11. Programming tools: Scala,IPython, Azure ML, … Frameworks: Spark, Hadoop, Yarn, HDInsight, Reef, Twister, Brisk Software Defined Storage Software Defined Networks Hardware Abstraction/Virtualization Container and Distributed Cluster OS (Docker, Mesosphere, K8S)
- 12.
- 17. The IPython notebookdeployed on a VM. 1. Create a coreos VM and open the https port 443 2. Login and issue this one command $docker run -d -p 443:8888 -e "PASSWORD=****" ipython/scipyserver 3. Go to https://yourVMaddress and log in.
- 18.
- 23.
- 25.
- 28.
- 29. • Many Examples •Challenges: • I only talked about the Analysis … but it there is more • Sustainability • Sharing • Reproducible Science Data Acquisition & modelling Collaboration and visualisation Analysis & data mining Dissemination & sharing Archiving and preserving