Computing at Scale: Data Exploration Jerjou Cheng, Barry Brumitt
HELLO <ul><li>/ jur' jō/ </li></ul><ul><li>ɯoɔ˙ǝlƃooƃ@ noɾɹǝɾ </li></ul>MY NAME IS Developer Programs Engineer (Developer ...
Michael PIERROT http://www.freephotobank.org /v/sky-stars/cloud/Cloud-19.jpg.html
Computing at Scale: Data Exploration Jerjou Cheng, Barry Brumitt
Overview <ul><ul><li>Google App Engine </li></ul></ul><ul><ul><li>Google Storage for Developers </li></ul></ul><ul><ul><li...
Introductions
Who are these services for?
A World without Clouds <ul><ul><li>Build a web application </li></ul></ul><ul><ul><ul><li> Startup costs </li></ul></ul><...
Google App Engine <ul><li>Easy to  start </li></ul><ul><li>Easy to  maintain </li></ul><ul><li>Easy to  scale </li></ul>
Users
gigy  Socialize - traffic
Overview <ul><ul><li>Google App Engine </li></ul></ul><ul><ul><li>Google Storage for Developers </li></ul></ul><ul><ul><li...
Overview <ul><ul><li>Google App Engine </li></ul></ul><ul><ul><li>Google Storage for Developers </li></ul></ul><ul><ul><li...
A World without Clouds <ul><ul><li>Store data </li></ul></ul><ul><ul><li> Reliability </li></ul></ul><ul><ul><li>Sharing ...
Google Storage for Developers <ul><ul><li>Google infrastructure </li></ul></ul><ul><ul><li>You control access to your data...
Example <ul><li>  </li></ul>
Internal use cases <ul><ul><li>Content hosting  </li></ul></ul><ul><li>  </li></ul><ul><li>  </li></ul><ul><li>  </li></ul...
Overview <ul><ul><li>Google App Engine </li></ul></ul><ul><ul><li>Google Storage for Developers </li></ul></ul><ul><ul><li...
Prediction API <ul><ul><li>Cloud-hosted machine learning as service </li></ul></ul><ul><ul><li>Simple interface over compl...
The Prediction API finds relevant features  in the sample data during training. How does it work? The Prediction API later...
Prediction API 1. Upload 2. Train Upload your  training data to Google Storage  Build a model from your data Make new pred...
Example <ul><li>  </li></ul>
Prediction API <ul><ul><li>Google's machine learning algorithms </li></ul></ul><ul><ul><li>Available as RESTful HTTP servi...
Overview <ul><ul><li>Google App Engine </li></ul></ul><ul><ul><li>Google Storage for Developers </li></ul></ul><ul><ul><li...
<ul><ul><li>To request access and get more information, go to: </li></ul></ul><ul><ul><ul><li>http://code.google.com/appen...
Upcoming SlideShare
Loading in …5
×

Computing at scale

763 views

Published on

Talk given at Seattle Tech Forum on Dev 15, 2010 at Bellevue City Hall.

Published in: Technology, Business
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
763
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
16
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide
  • Few sample graphs from Gigya’s events Very hard to predict traffic demands in advance - Largest event is 10X compared to second largest - Their largest was estimated to be one of the smallest :)
  • Computing at scale

    1. 1. Computing at Scale: Data Exploration Jerjou Cheng, Barry Brumitt
    2. 2. HELLO <ul><li>/ jur' jō/ </li></ul><ul><li>ɯoɔ˙ǝlƃooƃ@ noɾɹǝɾ </li></ul>MY NAME IS Developer Programs Engineer (Developer Relations) Google Storage
    3. 3. Michael PIERROT http://www.freephotobank.org /v/sky-stars/cloud/Cloud-19.jpg.html
    4. 4. Computing at Scale: Data Exploration Jerjou Cheng, Barry Brumitt
    5. 5. Overview <ul><ul><li>Google App Engine </li></ul></ul><ul><ul><li>Google Storage for Developers </li></ul></ul><ul><ul><li>Prediction API </li></ul></ul><ul><ul><li>BigQuery </li></ul></ul>
    6. 6. Introductions
    7. 7. Who are these services for?
    8. 8. A World without Clouds <ul><ul><li>Build a web application </li></ul></ul><ul><ul><ul><li> Startup costs </li></ul></ul></ul><ul><ul><ul><li>Maintenance / reliability </li></ul></ul></ul><ul><ul><ul><li>Scaling </li></ul></ul></ul>Michael Scheltgen flickr.com/mscheltgen/
    9. 9. Google App Engine <ul><li>Easy to start </li></ul><ul><li>Easy to  maintain </li></ul><ul><li>Easy to scale </li></ul>
    10. 10. Users
    11. 11. gigy Socialize - traffic
    12. 12. Overview <ul><ul><li>Google App Engine </li></ul></ul><ul><ul><li>Google Storage for Developers </li></ul></ul><ul><ul><li>Prediction API </li></ul></ul><ul><ul><li>BigQuery </li></ul></ul>
    13. 13. Overview <ul><ul><li>Google App Engine </li></ul></ul><ul><ul><li>Google Storage for Developers </li></ul></ul><ul><ul><li>Prediction API </li></ul></ul><ul><ul><li>BigQuery </li></ul></ul>
    14. 14. A World without Clouds <ul><ul><li>Store data </li></ul></ul><ul><ul><li> Reliability </li></ul></ul><ul><ul><li>Sharing </li></ul></ul><ul><ul><li>Large objects </li></ul></ul>Michael Scheltgen flickr.com/mscheltgen/
    15. 15. Google Storage for Developers <ul><ul><li>Google infrastructure </li></ul></ul><ul><ul><li>You control access to your data </li></ul></ul><ul><ul><li>Store massive data in Google's cloud </li></ul></ul><ul><ul><li>Easy interface </li></ul></ul>
    16. 16. Example <ul><li>  </li></ul>
    17. 17. Internal use cases <ul><ul><li>Content hosting  </li></ul></ul><ul><li>  </li></ul><ul><li>  </li></ul><ul><li>  </li></ul><ul><ul><li>Sharing </li></ul></ul><ul><li>  </li></ul><ul><li>  </li></ul><ul><ul><li>Data Import  </li></ul></ul>Google BigQuery Google   Prediction API
    18. 18. Overview <ul><ul><li>Google App Engine </li></ul></ul><ul><ul><li>Google Storage for Developers </li></ul></ul><ul><ul><li>Prediction API </li></ul></ul><ul><ul><li>BigQuery </li></ul></ul>
    19. 19. Prediction API <ul><ul><li>Cloud-hosted machine learning as service </li></ul></ul><ul><ul><li>Simple interface over complex analysis </li></ul></ul><ul><ul><li>Predict results in real-time </li></ul></ul>
    20. 20. The Prediction API finds relevant features  in the sample data during training. How does it work? The Prediction API later searches for those features during prediction. &quot;english&quot; The quick brown fox jumped over the lazy dog. &quot;english&quot; To err is human, but to really foul things up you need a computer. &quot;spanish&quot; No hay mal que por bien no venga. &quot;spanish&quot; La tercera es la vencida. &quot;english&quot; To be or not to be, that is the question. &quot;spanish&quot; La  fe mueve montañas.
    21. 21. Prediction API 1. Upload 2. Train Upload your  training data to Google Storage  Build a model from your data Make new predictions prediction/v1.1/training?data={} POST : a training request prediction/v1.1/training/{}/predict GET : model info POST : a prediction request   Use the API, gsutil or any compatible utility to upload your data to Google Storage 3. Predict
    22. 22. Example <ul><li>  </li></ul>
    23. 23. Prediction API <ul><ul><li>Google's machine learning algorithms </li></ul></ul><ul><ul><li>Available as RESTful HTTP service </li></ul></ul><ul><ul><li>Predict results in real-time </li></ul></ul>
    24. 24. Overview <ul><ul><li>Google App Engine </li></ul></ul><ul><ul><li>Google Storage for Developers </li></ul></ul><ul><ul><li>Prediction API </li></ul></ul><ul><ul><li>BigQuery </li></ul></ul>
    25. 25. <ul><ul><li>To request access and get more information, go to: </li></ul></ul><ul><ul><ul><li>http://code.google.com/appengine </li></ul></ul></ul><ul><ul><ul><li>http://code.google.com/apis/bigquery </li></ul></ul></ul><ul><ul><ul><li>http://code.google.com/apis/predict </li></ul></ul></ul><ul><ul><ul><li>http://code.google.com/apis/storage </li></ul></ul></ul><ul><li>         </li></ul>GET /information HTTP/1.0

    ×