Web as a data resource

1,593 views

Published on

Tech Talk at Georgia Tech

Hack U, 8 March 2011

0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,593
On SlideShare
0
From Embeds
0
Number of Embeds
13
Actions
Shares
0
Downloads
8
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Web as a data resource

  1. 1. Get me my Data !<br />Use the web as a data source<br />
  2. 2. Hack U @ Georgia Tech - March 8 – 12 2011<br />2<br />
  3. 3. Subramanyan Murali<br />yahoo<br />Mail Engineer <br />Hacker, Photographer, Traveler <br />@rmsguhan<br />
  4. 4. Data and hacks <br />4<br />
  5. 5. Putting something together in 24 hours can be a real rush !<br />5<br />
  6. 6. Internet is a data exchange medium <br />
  7. 7. Need innovative ways to make the data into information that is useful <br />7<br />
  8. 8. Any resource on the internet is a source of data … <br />8<br />
  9. 9. … that means that there is a lot of data <br />9<br />
  10. 10. Loads and loads of data and data services <br />10<br />
  11. 11. Data that is complex and in many formats <br />11<br />
  12. 12. Navigating through all the complexity can be hard !#$%&<br />12<br />
  13. 13. How to solve the data problem ?<br />13<br />
  14. 14. http://pipes.yahoo.comsaves the day!<br />14<br />
  15. 15. Graphical interface to play with data !<br />15<br />
  16. 16. WYSIWYG = pipes<br />16<br />
  17. 17. Predefined presentations !RSS, JSON, Map, badge<br />17<br />
  18. 18. Pipes is a visual tool, where is my command line ??<br />18<br />
  19. 19. Put your hands up for YQL !<br />19<br />
  20. 20. Common data access language across the webdeveloper.yahoo.com/yql<br />20<br />
  21. 21. SQL like syntaxStandard and easy to use<br />21<br />
  22. 22. select {what} from {service} where {condition} in ({sub-query} )<br />22<br />
  23. 23. Several services can be chained using in ()<br />23<br />
  24. 24. YQL Console<br />24<br />
  25. 25. Lots of Integrations and predefined tablesanyone can be part of the interface<br />25<br />
  26. 26. Fetching data from a resource has never been simpler !<br />26<br />
  27. 27. http://query.yahooapis.com/v1/ public/yql?q={query} &format={format} <br />27<br />
  28. 28. Irrespective of the source, output can be standardized …<br />28<br />
  29. 29. … thus giving you a solid & consistent data foundation<br />29<br />
  30. 30. Input can be XML, HTML, CSV, micro-formats, Atom feeds, JSON, RSS<br />30<br />
  31. 31. Output can be JSON or XML<br />31<br />
  32. 32. Mix and match YQL and Pipes ?<br />32<br />
  33. 33. Pipes provides you with processing capabilities<br />33<br />
  34. 34. YQL is another feed source<br />34<br />
  35. 35. Interlinking YQL and Pipes = fewer lines of code<br />35<br />
  36. 36. Not limited to Yahoo! sources <br />
  37. 37. select * from google.translate where q in (select text from twitter.search where q="georgia tech") and target='fr';<br />
  38. 38. Fire off multiple queries at once <br />
  39. 39. select * from query.multi where queries=‘{query1};{query2}’;<br />
  40. 40. Create your own tables with Open tables www.datatables.org/<br />
  41. 41. http://www.yqlblog.net/<br />
  42. 42. Go explore !developer.yahoo.com/yql/console/<br />
  43. 43. Happy Hacking !!<br />Photos @ http://flickr.com/photos/rmsguhan<br />

×