Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
1© Copyright 2015 Pivotal. All rights reserved.
@ianhuston
1© Copyright 2014 Pivotal. All rights reserved. 1© Copyright 20...
2© Copyright 2015 Pivotal. All rights reserved.
@ianhuston
Who am I?
Ÿ  Ian Huston
Ÿ  @ianhuston
Ÿ  github.com/ihuston
...
3© Copyright 2015 Pivotal. All rights reserved.
@ianhuston
What is Cloud Foundry?
http://cloudfoundry.org
Open Source Clou...
4© Copyright 2015 Pivotal. All rights reserved.
@ianhuston
How can CF help data scientists?
Ÿ  Jamie is a data scientist ...
5© Copyright 2015 Pivotal. All rights reserved.
@ianhuston
Cloud Foundry is a Platform
You bring the apps, the rest
is tak...
6© Copyright 2015 Pivotal. All rights reserved.
@ianhuston
Cloud Foundry Foundation: Industry Standard
Gold
Silver
7© Copyright 2015 Pivotal. All rights reserved.
@ianhuston
CF for data scientists & developers
Easily deploy your web app
...
8© Copyright 2015 Pivotal. All rights reserved.
@ianhuston
Simple Python Flask App Demo
Ÿ  Simple one page “Hello World” ...
9© Copyright 2015 Pivotal. All rights reserved.
@ianhuston
Official Language Support on Cloud Foundry
Go
Ruby
10© Copyright 2015 Pivotal. All rights reserved.
@ianhuston
Buildpacks – How to Customise CF
Use Heroku-like buildpacks to...
11© Copyright 2015 Pivotal. All rights reserved.
@ianhuston
Comparison with Docker
runtime layer
OS image
application laye...
12© Copyright 2015 Pivotal. All rights reserved.
@ianhuston
Services now available
These are available on Pivotal CF (Pivo...
13© Copyright 2015 Pivotal. All rights reserved.
@ianhuston
Data Services
Ÿ  Easy to bind and scale services
–  Databases...
14© Copyright 2015 Pivotal. All rights reserved.
@ianhuston
What can you build with CF?
Some ideas:
Ÿ  Interactive data e...
15© Copyright 2015 Pivotal. All rights reserved.
@ianhuston
Examples
Ÿ  Simple HTML + JS
app
Ÿ  Uses data in S3
Ÿ  Scal...
16© Copyright 2015 Pivotal. All rights reserved.
@ianhuston
Examples
Ÿ  RShiny app
Ÿ  Interactive controls
with very lit...
17© Copyright 2015 Pivotal. All rights reserved.
@ianhuston
Send me your examples!
Show off your data science related Clou...
18© Copyright 2015 Pivotal. All rights reserved.
@ianhuston
Try Cloud Foundry today!
Pivotal Web Services: http://run.pivo...
19© Copyright 2015 Pivotal. All rights reserved.
@ianhuston
More info and resources
Ÿ  Material for this talk: http://tin...
Upcoming SlideShare
Loading in …5
×

Cloud Foundry for Data Science

2,272 views

Published on

As a data scientist I frequently need to create web apps to provide interactive functionality, deliver data APIs or simply publish results. It is now easier than ever to deploy your data driven web app by using cloud based application platforms to do the heavy lifting. Cloud Foundry (http://cloudfoundry.org) is an open source public and private cloud platform that enables simple app deployment, scaling and connectivity to data services like PostgreSQL, MongoDB, Redis and Cassandra.

Resources: http://www.ianhuston.net/2015/01/cloud-foundry-for-data-science-talk/

Published in: Technology
  • Be the first to comment

Cloud Foundry for Data Science

  1. 1. 1© Copyright 2015 Pivotal. All rights reserved. @ianhuston 1© Copyright 2014 Pivotal. All rights reserved. 1© Copyright 2015 Pivotal. All rights reserved. 1© Copyright 2015 Pivotal. All rights reserved. 1© Copyright 2014 Pivotal. All rights reserved. Cloud Foundry for Data Science Ian Huston, Data Scientist
  2. 2. 2© Copyright 2015 Pivotal. All rights reserved. @ianhuston Who am I? Ÿ  Ian Huston Ÿ  @ianhuston Ÿ  github.com/ihuston Ÿ  www.ianhuston.net Ÿ  Talk resources: http://tinyurl.com/cf4ds-talk Ÿ  Data Scientist Ÿ  Previously a theoretical physicist using Python for numerical simulations & HPC Ÿ  Use Python/R/SQL/… for predictive analytics and machine learning
  3. 3. 3© Copyright 2015 Pivotal. All rights reserved. @ianhuston What is Cloud Foundry? http://cloudfoundry.org Open Source Cloud Platform Simple App Deployment, Scaling & Availability No Cloud Provider Lock In
  4. 4. 4© Copyright 2015 Pivotal. All rights reserved. @ianhuston How can CF help data scientists? Ÿ  Jamie is a data scientist who has just finished some analysis. They want to put up a simple internal web app with Javascript visualisations connected to internal data stores. Ÿ  Sam is a data engineer who wants to set up a REST API to expose a production machine learning model as a service. Ÿ  Alex is a data scientist who has an existing RShiny or Python app that they want to make available with multiple instances.
  5. 5. 5© Copyright 2015 Pivotal. All rights reserved. @ianhuston Cloud Foundry is a Platform You bring the apps, the rest is taken care of! Source: Albert Barron (IBM), https://www.linkedin.com/pulse/20140730172610-9679881-pizza-as-a-service
  6. 6. 6© Copyright 2015 Pivotal. All rights reserved. @ianhuston Cloud Foundry Foundation: Industry Standard Gold Silver
  7. 7. 7© Copyright 2015 Pivotal. All rights reserved. @ianhuston CF for data scientists & developers Easily deploy your web app cf  push  myapp   Scale up and out quickly cf  scale  myapp  –i  5  –m  1G   Create and bind services cf  bind-­‐service  myapp  redis    
  8. 8. 8© Copyright 2015 Pivotal. All rights reserved. @ianhuston Simple Python Flask App Demo Ÿ  Simple one page “Hello World” web app Ÿ  Video: https://www.youtube.com/watch?v=QOfD6tnoAB8 Ÿ  Demonstrates: –  Installation of requirements –  Scaling properties Ÿ  Need to Provide: –  App files –  Dependencies listed in requirements.txt file –  Optional manifest.yml file with configuration for deployment
  9. 9. 9© Copyright 2015 Pivotal. All rights reserved. @ianhuston Official Language Support on Cloud Foundry Go Ruby
  10. 10. 10© Copyright 2015 Pivotal. All rights reserved. @ianhuston Buildpacks – How to Customise CF Use Heroku-like buildpacks to add additional runtimes: Ÿ  Community Buildpacks for Clojure, Haskell, .NET, Erlang,… Ÿ  R Buildpack –  https://github.com/alexkago/cf-buildpack-r –  Works with RShiny apps Ÿ  PyData Buildpack –  https://github.com/ihuston/python-conda-buildpack –  Uses conda for hard to install dependencies
  11. 11. 11© Copyright 2015 Pivotal. All rights reserved. @ianhuston Comparison with Docker runtime layer OS image application layer Docker system brings fixed host OS Kernel * Devs may bring a custom buildpack runtime layer* OS image application layer CF Buildpack system brings fixed host OS Kernel App container System Provides Dev Provides
  12. 12. 12© Copyright 2015 Pivotal. All rights reserved. @ianhuston Services now available These are available on Pivotal CF (Pivotal’s packaged Cloud Foundry offering). See http://run.pivotal.io for the services available on Pivotal Web Services.
  13. 13. 13© Copyright 2015 Pivotal. All rights reserved. @ianhuston Data Services Ÿ  Easy to bind and scale services –  Databases, NoSQL, message queues etc. $  cf  create-­‐service  rediscloud  PLAN_NAME  INSTANCE_NAME   $  cf  bind-­‐service  APP_NAME  INSTANCE_NAME   Ÿ  Details provided in VCAP_APP_SERVICES env variable: {  "rediscloud":  [          {  "name":  "rediscloud-­‐42",  "label":  "rediscloud",  "plan":  "20mb",              "credentials":  {  "port":  "6379",      "hostname":  "pub-­‐redis-­‐6379.us-­‐east-­‐1-­‐2.3.ec2.redislabs.com",                  "password":  "your_redis_password”    }    }  ]   }  
  14. 14. 14© Copyright 2015 Pivotal. All rights reserved. @ianhuston What can you build with CF? Some ideas: Ÿ  Interactive data exploration apps Ÿ  APIs for a machine learning model Ÿ  Internet of Things data collection and modelling tools Ÿ  Microservices architecture for data analysis Ÿ  Interface to a natural language system like IBM’s Watson
  15. 15. 15© Copyright 2015 Pivotal. All rights reserved. @ianhuston Examples Ÿ  Simple HTML + JS app Ÿ  Uses data in S3 Ÿ  Scale out when heavy load expected with auto load balancing http://ds-demo-transport.cfapps.io
  16. 16. 16© Copyright 2015 Pivotal. All rights reserved. @ianhuston Examples Ÿ  RShiny app Ÿ  Interactive controls with very little additional coding. Ÿ  Runs RandomForest model in background https://ak-insurance-demo.cfapps.io:4443/
  17. 17. 17© Copyright 2015 Pivotal. All rights reserved. @ianhuston Send me your examples! Show off your data science related Cloud Foundry apps: Twitter: @dsoncf http://dsoncf.com http://DataScienceOnCloudFoundry.com
  18. 18. 18© Copyright 2015 Pivotal. All rights reserved. @ianhuston Try Cloud Foundry today! Pivotal Web Services: http://run.pivotal.io IBM Bluemix: http://bluemix.net Anynines: http://anynines.com HP Helion: http://horizon.hpcloud.com
  19. 19. 19© Copyright 2015 Pivotal. All rights reserved. @ianhuston More info and resources Ÿ  Material for this talk: http://tinyurl.com/cf4ds-talk Ÿ  Cloud Foundry: http://cloudfoundry.org Ÿ  Python Flask Mega Meta Tutorial for DS: http://tinyurl.com/flaskDS Ÿ  Twelve Factor Apps: http://12factor.net/ Ÿ  Meetups: http://cloud-foundry.meetup.com/

×